This is kind of a theory question, and might even belong in the math thread. I'm trying to work something out for Dungeons and Dragons shenanigans, so obeying physical laws isn't strictly necessary (we're allowed to have stuff happen instantaneously, for instance, rather than having a minimum length of time).
If you have a recursive operation with no base case, instantaneous (or, in other words, length 0 time) execution of each iteration of that operation, and a finite amount of time over which it is executed (6 seconds in this case, but it should be unnecessary to specify), how many iterations execute by the end of that time?
Expanding on MadocComadrin's answer, if the casting happens and repeats instantaneously, in length zero time, you will reach infinity iterations in any nonzero time. This has some rather interesting implications. For starters, the mage hand triggering of the second trap probably involves some manner of movement, yes? I don't know how mage hands work, but I'm picturing an ethereal glowing hand moving up and down on a lever like it's trying to start a wank fire. Well, it DOES start a wank fire, and a big one at that. Since the hand is triggering the trap INFINITY times per second, it is necessarily moving infinitely fast, thus creating an infinite amount of friction, feeding an infinite amount of thermal energy into the system. What this would mean in real-world physics is something of a mystery, but it would presumably increase the temperature of the trigger so high that the concept of "temperature" would no longer exist, and the traps, the dungeon and the adventurers would all be broken into whatever particle-equivalents could exist at such energy levels.
Each of these particles, lets call them "doomlets", would have infinite energy, and they would expand in a sphere at the speed of light, never slowing down. Every particle of mundane matter they interact with, assuming some form of particle interaction is possible, would turn into more infinite-energy doomlets, thus destroying the entire Prime Material Plane, and everything connected to it in a manner that allows the passage of doomlets.
It's also quite possible that our doomlets will not be able to interact with normal matter. In this case, the trap's operational parts will simply vanish from existence, with or without a fireball of unknown size. Who knows.
This would also happen if the spell itself outputs any amount of energy. Does it form a physical bolt of some kind, that displaces matter or glows even slightly? Does it make even the faintest of sounds? That's also infinite energy. If the magic leaves some kind of trace that does not affect the physical world in any way but is detectable by other wizards, this trace is multiplied infinifold. I doubt the true nature of magic in DnD is clear enough for the results of this to be defined, but plausible effects might include:
- Total destruction of multiverse, as before
- Magic completely disappearing from multiverse
- Every magic user in the multiverse suddenly dying
- Every creature in the multiverse suddenly dying
- Every creature in the multiverse gaining the casting ability of a level infinity sorcerer
- Anything else. It's MAGIC for Mystra's sake!
So, before making things happen infinity times per turn, you should remember that the GM has many ways to screw you over if he so wishes. You'll only have yourself to blame for the resulting total party kill of cosmic proportions.