Well. Here's the thing. Bows and crossbows are very, very OP against armor and very, very UP against well-trained dodge/shield user/weapon users. Any bolt should be able to pierce through virtually any armor like it was shot through a railgun. Wooden bolts can even maim or kill Dwarves in adamantine. On the downside, well-trained enemies will be able to dodge/block/parry ridiculous numbers of bolts. But that can be solved with more dakka... By giving every civilian a crossbow and some low-grade bolts.
Practically speaking, if Goblins are rampaging through your fortress, they won't be charging down long, narrow hallways with all your civilians lined up to fire at them all at once. They'll be coming down stairways, through troll-smashed doors, and around corners. They'll also come in force so shooting one or two won't slow them down much. Some of your militia will get good opportunities, but many might only get one or two shots off before they get mobbed, and the odds are those will miss. But if you have 100 militia Dwarves, and each gets just a shot off, you could easily end up wounding or killing 20 Goblins. They'll probably get more shots off by shooting Goblins while they murder friends and pets.
If you intentionally design your fortress with a lot of open space and long hallways, they'll do better. If you give them all breastplates and a helm made out of iron or bronze or copper, they'll do much better. And if you make sure to train all of your militia a bit, up to at least Proficient, before turning them back into civilians, you could do some serious damage.
Basically, while it might not stop an invasion by itself, it will definitely give civilians a defensive edge in emergencies and will make taking your fortress more costly for the enemy. Unless you go the full mile and give every single Dwarf some rudimentary equipment and some training. Then you have 100 semi-armored Dwarves with railguns of death and the skill to use them patrolling your fortress at every moment, looking badass and popping bolts in any Goblins that dare show their faces. That's a god damn Dwarf Fortress.