To get more out of training follow this checklist:
1. Squads should be 3 or 4 dwarfs big, this increases the amount of time spent sparring the 3rd dwarf will do an individual combat drill in the case of 4 dwarfs being available by chance the two non sparring dwarfs will do demonstrations, since there's only two of them their teacher skill will level up faster than with 10 dwarfs / squad
2. Draft dwarfs who already have military skills (use Dwarf Therapist). The important skills are (in order of importance): dodger, shield user, armor user, fighter, weapon skill. Teacher/student/organizer/observer are a big plus but will ultimately be raised to reasonable levels in a few years. Bringing a dedicated militia commander on embark with a combination of mentioned skills appears to be a very viable strategy.
3. Make the best possible equipment and make at least a full set of armor, weapon and shield for every soldier early. A dwarf without a shield cannot gain shield user skill while sparring, same applies to weapons.
4. Equip the dwarfs via uniforms and make sure they all have flasks/waterskins and backpacks (disable woodcutting,mining,hunting for soldiers or you will run into a nasty bug). This will reduce the time they spend eating and drinking to very brief trips to fill their containers.
5. Using individual choice melee weapon for a weapon in your uniform is fine if the dwarf only has one weapon skill and you have enough weapons of that type so he will take one. If he has multiple weapon skills or no weapon skills assign the type of weapon you want him to train, else he might change his no quality sword for a superior quality axe training up multiple weapon skills and wasting his time.
6. Consider putting dwarfs without a weapon skill in an inactive squad but assign them a barracks for training and disable all their civilian labors. Do this with any useless migrant preferring the ones with good physical attributes who you don't need for hauling (assign at least a weapon preferably a shield too). They will do individual combat drills slowly raising their weapon skill (amongst others) without wasting your non recruits time by taking forever to organize a demonstration (which wouldn't do much anyway as the recruit doesn't have many skills).
Once UristMcRecruit becomes an axedwarf, put him in an actively training squad.
7. Deal with losses in a smart way: If a whole squad is killed don't replace them all with novice soldiers. Instead rearrange your squads into groups of 2 experienced and 2 new soldiers.
Same in the event of no losses. Add 2 novice soldiers to your first squad of 4 after a few years, combat training will now actually do something.
Some more bits not related to training
Know your enemies: Lashers are the death of every soldier without some defensive skills, so train up a squad of marksdwarfs (bone bolts, wood bolts or even metal if you have magma for the archery range) and rain death on them from behind fortifications.
Ranged attackers will also do well against anyone without high dodging and/or shield using skills, so use zig zag walls or tunnels.
Axes aren't "the best" weapon. While axe lords are pretty awesome, on low skill levels spears give a better chance of hitting a vital organ early on resulting in a quick death or insta-kill, and hammers will be better if an armored body part is hit.
fakeedit: not gonna read all replies before posting this if anything has been mentioned before - sorry