I wasn't trying to imply any of that. Just saying that our sense of time isn't an accurate representation of what time is, the same way our five senses aren't an accurate representation of light, sound, electrostatic repulsion, and chemistry. Our senses are designed to help us navigate the world through the interpretation of certain data. Your sense of sight represents objects as solid when they're really mostly empty space, omits most of the spectrum of electromagnetic radiation, and only gives the most vague hints about the nature of lightwaves. Sight is such an abstraction that we're still working on figuring out what light is.
The way the universe is, and the way we first perceive it, are very, very different things. Our sense of time is no less distorted. It's not because our senses are trying to trick us, it's just that it makes it easier for the mind of a child to learn not to injure oneself or get eaten, which is more important in terms of individual survival than knowing the actual density of particles in an object or the exact wavelength, speed, direction, and amplitude of the light waves entering the eyes.
When I said before that we perceive time the way it is because that's the only way we can make sense of it, that wasn't really a complete description, as Starver pointed out. It's more of a result of the way entropy works. When an event happens, the energy associated with that event immediately begins dispersing into the surrounding environment in various ways (light, sound, movement of nearby objects, etc). You can think of this as similar to ripples on a pond. Anything outside the zone that the ripples have spread to (let's assume we're only talking about the light of the event here) is unaffected by and unaware of the event. Because entropy (think of entropy as how "spread out" a given amount of energy is) always increases in one direction in time, the ripples always spread outward from an event as we approach the future, never inward. The brain of an observer can only be aware of an event once those ripples have reached it.
To have time go in the other direction would be similar to Starver's stone-throwing pond; it would require the observer to know of the event in advance, and then forget it, and in the process actually help to cause it by converting those memories to nerve impulses traveling to the eyes, which then send out light beams that converge simultaneously with other beams from every direction and provide the energy to cause the event in reverse. So yes, if you saw time backwards, telekinesis would be real, but you'd have to forget things in order to make them happen, even though your memory of it would be clearest in the instant before you forgot it.
That actually sounds pretty cool.
So, that's why the brain can only remember events in the past (meaning when entropy is less, not what the observer considers the past) no matter which direction they'd percieve time in. And that's why we don't see the whole "tape" at once. Well, I guess at the last instant before you die (or, for those who live backward, at the instant you un-die), assuming you had a perfect memory, you actually could perceive the entire portion of the cosmic videotape that you directly experienced. It may sound like it but I'm actually not saying it's in any way related to the saying that your life flashes before your eyes when you die, the similarity is entirely a coincidence. (see what I mean though about myths echoing reality?)
But just because we can only remember things in one direction, that doesn't mean that everything in the other direction is undefined. This is apparent when you consider that for those who live backward (and just to be clear, I am NOT saying that such beings exist, just using them as a thought experiment to illustrate my point), everything that they directly experience is defined for them at the beginning of their lives. As they "age", they remember a given event more clearly, then perfectly, then forget it completely as they contribute energy to it. But, even though they forgot it, and they are completely unaffected by it for the "first time", that doesn't mean it didn't unhappen. The fact is that it did unhappen, even though nobody is aware of it. For us, we are not aware of what events lie in our future, but that doesn't mean they're undefined.
Is this idea testable? As far as I know, absolutely not. So, it can't be a true feature of physics. But it's self-consistent and works on paper, and it's possible that some day a physicist derives a hypothesis from this idea that can be tested, so it's worth keeping in mind. It's somewhat like string theory (that's a misnomer, should be string hypothesis) in that regard; currently untestable ideas that nevertheless do a good job of explaining otherwise unexplainable things, and through further development may eventually lead to hypotheses that can be tested.