Something you all seem to be missing here is that the impressive endurance of the Earth Ponies isn't just physical.
They work harder, longer, and can maintain the focus needed to get the job done. They seem to be naturally disciplined. They are not stupid, either.
Pegasus ponies are fast and can fly, but they also seem fairly lazy and, mentally, a lot weaker than the earth ponies we've seen. They seem more interested in competing and playing around than actually getting work done. We haven't seen Pegasus ponies in many leadership positions, either - while earth ponies seem to be widespread in roles such as aristocrats, mayors, and even as leaders of the fashion world, where one would expect unicorn ponies to be on top.
Unicorn ponies, meanwhile, seem overly concerned with appearances. Twilight is really different than most of them - do recall how the other unicorns seemed upset that she would prefer working on her studies to socializing. Those we've seen with magical talents (again, aside from twilight), are quite focused, and having something come so easily to you must make it quite difficult to justify all that effort it would take to do something else. Especially when most of them seem to care far more for social approval than anything meaningful.
So who are the movers and shaker's of pony society? Earth ponies. They push out the frontiers, they do the work that needs to be done, they can be found leading industries and managing towns, because above all their defining trait seems to be the fact that they get. things. done.
If there was ever a war between the three types, well, I know which side I'd want to lead. While the unicorns were struggling to deal with their own trying to get out of military service, or patting themselves on the back for being such good sports and letting those earth ponies go free, or chatting up about how secure they are in their "impenetrable" magical fortress they made with a whole bunch of tricks that they've nicknamed the Titanic, the Earth Ponies would be figuring out exactly what they would need to do to win the war, and then they wouldn't stop until the war was won.
I can't even see the Pegasus pony's organizing an army without it breaking out into bickering, to be honest. Assuming they would even want to fight and not just sit things out after making an agreement with the earth ponies to continue getting... ya'know, food.
You might be able to train enough unicorns to pull together a small elite force, but most of them would wash out. Or whine. Or give up. Or get tired too easily to be worthwhile. And it would take time. And the whole time they would likely be questioning why they have to be the ones doing it, while the Earth Ponies are getting organized, kicking down the doors, and, oh yeah, being the only side that has any idea how to do the work that keeps society functioning. (like growing food).