The way I like to "hack" this is
a) make a custom stockpile of weapons, armor, backpacks, flasks, quivers and bolts either directly at, OR close by, my entrance where I usually want my militia to first assemble at when there is a threat.
b) keep some dogs just in front of the stockpile to spot thieves who might want to steal from this stockpile or enter the fortress
c) have my militia with dual roles anyway, no need to have pure military dwarves in the embark. Instead of hunter/marksdwarf, I would go wood cutter/marksdwarf, instead of wood cutter/axedwarf I would go miner/speardwarf. Helps to distinguish their modes, and keep tabs on equipment, as they will either strictly have a battle axe or a mining pick and be in civilian mode, or a military set and be in the military mode. This way all mining picks and battle axes are dedicated civilian mode tools while any spears and crossbows are dedicated for military and threat response. Make sure to create or bring the correct amount for things to roll smoothly.
The drawback here is I don't get to have a hunter that would participate effectively in militia since the only way he would be useful for defense is if the design wasn't so that he will run from far far away to pick up his OTHER crossbow from the frigging stockpile (reference a) ) which shouldn't be needed anyway, and then probably it was too frigging late. If hunter/marksdwarf would be able to follow commands without dropping his crossbow, only then he would be convenient... But we can't have that. So that a) b) c) above is how I'm rolling in the meantime instead.