Currently, invaders from the first goblins to sully your newly-founded fortress to the goblins that attack your decades-old fortress filled with the finest in all things come to you barely out of whatever passes for goblin military academy, with barely the skill to hold a sword by the handle and hardly a stat increase to their names, their crudely-forged equipment barely clinging to their bodies.
These goblins, then, are sent against the legendary warriors of the dwarves, equipped with the finest polished steel, trained since the dawn of adulthood in the ways of war. It is no wonder, then, that the average "mature" fortress can afford but a handful of guards (a handful simply to reduce the chance that they are all eating, drinking, or asleep at the same time), since a single one can slay an entire goblin siege without breaking a sweat.
Thus, I propose that in addition to sending additional goblins, the skill of the warriors sent should also rise with time. More slowly to be sure, the first wave being nothing more than a scouting party. But soon, when word of the wealth and power of the fortress spreads, goblin generals will begin to plan more than simply rounding up the first eighty goblins they see and marching them off to war. More skilled goblins, trained under the harsh eye of their generals or drawn by tales of glory and plunder, will compose the armies. An advanced fortress might even expect to see a legendary goblin or two among their foes, warriors able to confidently challenge a dwarven warrior to single combat in melee.
The flowery language aside, it's kind of ridiculous that the gobins that siege and ambush you have less than a month of training (based on dwarven training rates). I propose that once the sieges cap out at 80, the skill levels of the invaders start rising instead, so that they continue to provide something other than comic relief once you have your first legendary solider. While it doesn't do anything to address the problems posed by traps, magma, drowning chambers, atom smashers, or engineered cave-ins, I think that it would do something to make goblin sieges more of an event.