Here's where the problem lies: If you are some sort of soul/non-material being, you must also be riding a p-zombie. We can trace motion back to signals sent to muscles; those signals back through the brain. And, given sufficient observation, trace those signals back through time, identifying the trillions upon trillions of origins for every minute detail of those signals. Which is to say, without violation of the laws of physics in a directly observable fashion (either through statistical anomalies or imbalanced equations of energy transfer or similar), interaction can be at best from your brain to your non-physical bits, and not the other way around. That rules out extra-universal souls or non-physical causes; making p-zombies an incoherent notion in those scenarios.
Then there's a purely material world of p-zombies. In this, the material soul-object is a measurable phenomena, much like the Ghost Line from the Ghost in the Shell series. In this case, you can reproduce it by simply using adequately meticulous duplication methods. Furthermore, if it turns out to be a relatively stable, self-reinforcing pattern (as most brain activity patterns turn out to be; give a simple neural structure mimicking a brain's, then an application of some current stimulating it, it will create activity patterns at least similar to that of a brain), even an imperfect replication will fall into a similar pattern. And again, everything is thus measurable; if it ends up being different, that is a detectable difference, preventing it from falling into p-zed classification, making it a matter of methods rather than philosophy.