It's possible, though it depends on how world population interacts with site population. My medium-sized world has a total of 64,424 goblins according to Legends Mode. The actual goblin civ I'm at war with, The Evils of Spray, has the following:
757 Goblins
42 Dwarves
27 Humans
22 Goblin Outcasts
14 Dwarf Prisoners
10 Elves
6 Human Prisoners
1 Elf Prisoners
I assume outcasts and prisoners are not combatants, so that's a total of 826 foes. They send big raiding parties once every 2 years. I usually kill 20 before the rest scatter, so with an average of 10 a year, that's 83 years before the Evils of Spray is depleted. At that point, I have no idea what the game does. Do more world pop goblins replenish their ranks? Does it select the next closest goblin civ? My current fort is 20 years old, which is my oldest one by far. The 100+ year forts I know of turned off invaders long ago to work on megaprojects, though there might be some succession forts that have destroyed an enemy civ.
**Edit**
I just checked my Legends file, approx. 2 years after the last one. Goblin population world population has GROWN by 31. I was not expecting that at all. I doubled checked to make sure I didn't switch my legends files. The site pop for the Evils of Spray hasn't changed at all, though they lost around 20 goblins recently.
Civilized World Population, c. 269
152963 Dwarves
125628 Humans
143501 Elves
64424 Goblins
2615 Kobolds
Total: 489131
Civilized World Population, c. 271
152996 Dwarves
125656 Humans
143512 Elves
64455 Goblins
2614 Kobolds
Total: 489233