despite the fact that goblins are the second most technically adept creatures in the game (Pro Tip: Saying clearly doesn't make something true), and the fact that as far back as I remember, discussions about goblins have invariably focussed around the differences between those who think post Tolkien D&D and video games, and those who think classic northern European mythology.
This bothers me enough that I have to reply to it.
Dwarf Fortresses' goblins are, at least for the things relevant to this discussion, D&D / Tolkien goblins. Technically Tolkien didn't really have distinct goblins except as a different word for smaller orcs, but everyone else just used it for creatures like his orcs, but smaller and greener. I'll use "Tolkien" as a catchall term for everything derived from him, in any case.
Traditional goblins are primarily distinct from Tolkien's orcs and goblins in the following ways:
* First, and by far the most importantly, they didn't form armies. Generally they were purely individual. A goblin might cause mischief on the road or a small group of them cause mischief in an area, but they were essentially considered local spirits. They would never move about in large numbers or invade anyone in a military fashion. At best, they might have a king in the same way you might have a fairy-king or some such thing, who can be appealed to to settle disputes or break curses he's cast or whatever -- but he was a magical creature, not a secular king who raised armies and invaded people. The militaristic 'vile force of darkness' that Dwarf Fortress uses is, as it applies to Goblins, essentially something Tolkien invented whole-cloth.
* Second, goblins were not generally considered demonic themselves (they were considered a type of mischievous fairy, generally), and they did not serve demonic overlords; they served only their own nature and their whims. The horde of green warty creatures serving as an army for a shadowy demon overlord is, again, Tolkien's invention.
* Third, goblins did not generally use weapons or armor, and it would be
especially -- most particularly -- unusual to have them using iron or steel, which (aside from everything else) requires a degree of organization and cohesion that nobody ever ascribed to scattered, mischievous goblins until Tolkien rewrote them into his 'vile force of darkness.' There were exceptions to this (goblins sometimes overlapped with dwarves), but in those cases they tended towards dedicated crafting-spirits, not creatures that would forge swords and then run around stabbing people with them. Having goblins and their like acting like humans in that regard was entirely Tolkien's invention.
* Fourth, they were not universally evil. They were mischievous, certainly, and might cause trouble if you don't leave a plate of milk or some similar sacrifice out for them, but they wouldn't generally run around committing murder (and certainly not invading anyone -- although, of course, as I already explained they wouldn't invade anyone anyway because they were never considered to form armies.) If you treated them respectfully they might help you.
'Goblin' encompasses an awful lot, of course, so there are no doubt exceptions to the above, but the most important point (about goblin armies, their most distinctive trait in Dwarf Fortress and the one you are most concerned about in this thread) is, I think, universal -- that is pure Tolkien and D&D, with no mythological antecedents.
This is common sense when you think about it. People actually
believed in goblins. Nobody could seriously believe in a huge army of goblins laying siege to structures, or forming big societies or whatever. Therefore, they were small, mischievous things on which people blamed small troubles. Tolkien was able to turn them into a big scary army because he was writing in a purely fantasy universe, and only loosely drawing on myths and legends.
Dwarf Fortress, as you can probably see, uses Tolkien's goblins and completely ignores the earlier definition for nearly everything important that can currently be seen in the game. But, more than that (and here's what I'm really getting at),
you are using the Tolkien-D&D definition. You are. You are asking for goblins to be made even less small, mischievous things, and even more into huge vile Tolkienesque D&D armies who attack in mindless waves in order to provide a challenge for your characters.
I resisted saying this until now because it isn't really that important, but it bugs me every time you bring it up. You have it completely and utterly backwards. Organized armies of goblins armed with human-like metal weapons invading dwarves and elves as part of big military campaigns overseen by demons is pure D&D.
Tiny groups of mischievous goblins (or, really, individual mischievous goblins) who pose little threat to even one alert, armed military dwarf, but who skulk around curdling milk and stealing things and, occasionally, helping people if they leave out proper offerings and so forth is the earlier traditional goblin.
If you want traditional goblins, the first step is to ask for the complete and permanent removal of their ability to form armies or level sieges. In effect, they would be a bit more like kobolds.
That is all.