By default the training alert schedule keeps the squad training at the barracks all the time. Until the soldiers earn champion status that will result in unhappy thoughts. Go to the schedule for the squad via (m) -> (s) and delete (x) the training order from the squad for a few months (giving them time off for 4 months is usually enough). It is better to make their holidays multiple months long because they will get unhappy thoughts from changing to a militia dwarf from being a civilian too (which can get you unhappy dwarfs quickly with a schedule like train - no order - train - no order - etc).
The dwarfs will do individual combat drills in their free time too which increases their weapon skill, so if you start with completely green guys it is actually advisable to not have them on active/train all the time.
Make sure the schedule reads sleep in room at will (the default) and make good rooms for them (good quality bed, door, etc). Another trick is to make multiple paved roads (out of materials they like if you have easy access) on the way to the barracks so whenever they leave/enter the barracks they will get a happy thought.
Whenever you give the soldiers active orders, make sure you cancel them once the threat is dealt with.
If you need longer protection for stuff like woodcutting or outdoor constructions, consider using multiple squads.