I was saying sealing the entrance up might result in decreased speed, as it would discourage any goblin ambushers from invading your fortress and revealing themselves. If the entrances are all locked, outside goblins tend to just mill around, and don't bother making their way towards your entrances. However, if there's a way into your fortress, the goblins will usually make their way in as soon as they're ready, making it easier for you to deal with them accordingly.
A default size map, 6x6 I think, is pretty large, and allows a lot of room for ambushers to be wandering around outside, so I'm guessing there are probably quite a lot of goblins out there waiting to be slaughtered. I'm not sure what the best way to deal with them would be, other than expanding your outdoor territory, accumulating some additional dwarves for military training, and maybe throwing and locking some animals outdoors to act as search parties for goblins.
Yeah, it's common to have lots of stone lying around. When you say "a huge amount", do you mean, say, a few thousand, or a few hundred thousand? The most I've had is just over one hundred thousand (as checked in the Stocks screen after pressing [z]), and that fortress ran slowly and, I think, with stutters, but it could be just coincidence.
I haven't found very many underground rivers myself, but have you cleared out any creatures that may be living down there? Do you know if the stuttering started around the time when you tapped the river?
I'm sure other people may have more ideas that I haven't thought of yet.