Make two squads of 5, or better yet three squads of 4, and have them train for 12 to 18 months. They will be Adequate to Skilled after about nine months of training (even if using training weapons instead of the real thing). The training goes faster if you have at least one dwarf already Skilled or better in the weapon the squad is assigned to use.
After 18 months, switch them to an on-training/on-duty/off-duty schedule if the majority are complaining about long patrol duties. In my current fort I have the three squads off four months, train four months, stand guard four months, so I always have one group on duty and one group sparing/training. The original ten are all Competent or better after a year, while the new recruits are gaining skill faster now than I have a competent cadre to train them.
If you have caged wild animals/kobolds/goblins you can have your dwarves battle them in an arena. They always gain skill fastest in live combat, and this gets more true as their skills improve.
For humor value, and longer "training" battles, ensure your troops are armed with training weapons. You could also leave the goblins their armor.
I'm sure others have better info, but the above (except the arena thing, which I haven't tried ... yet) is all stuff I've used in multiple forts with success.
[edit] As restricted said, letting the militia get unhappy enough to tantrum is a bad thing. The last thing you want is them to have the happy though "enjoyed a good slaughter lately" when you haven't encountered any hostiles in a season or three. For this reason I normally only do constant training for one year, then shift them to an on/off schedule. That initial year is to get them up to at least Adequate skill in their assigned weapon.