First of all, let me say that I dont have any conclusive proof, but here are some thoughts.
If, indeed, every stone is looking to create a hauling job, if you will, then it's just logical it can decrease game speed. If it's just the dwarves creating jobs, a lot of stones could be problematic IF every hauler decides to check every available item. It should not play a role if the looking stops after the first appropriate item to be hauled is found.
Forbidding them should alleviate the problem in any case a bit, since a flag that makes all the possible hauling jobs not even consider that certain stone should be less CPU intensive than every other possible option.
However it's handled, a lot of stone probably tax the ram. I dont know how much stones you need for that to apply, but a wild guess says it's "a fucking metric shittonne".
And last but not least a little story:
I had my dwarves live in the soil layers and was preparing the real fortress. The main level was dug out, full of stones, so I designated the approximate center tile as dump zone and just let them dump all the stones in that level on that tile. The dump zone was in the future dining hall, accessible by 4 entrances, each of them 2 tiles wide (door wall door). Framerate around 30.
As a little experiment I dug out the wall tile of the entrances. Framerate climbed to 60.
So... pathfinding can be a problem without you noticing it. For instance assume you have some exploratory mining shafts that are 1 tile wide. Or future aquaeducts that are 1 tile wide. There are some rocks lying around there. Now you may not even have a dump zone or any dump designation, you may not even have a stone stockpile. But you PROBABLY will have some stone using jobs queued - I guess you guys see what I'm getting at....
Well, either way, I suffer from a mild case of OCD, and I tend to clear out my maps. (Hence the design of my waste disposal system in my sig) I can say with a considerable certainty that having less items on a map increases game speed. Especially as you get more dwarves all the "Might there be something that needs hauling" seems to take more CPU time. Coming to think of it... the "check for hauling jobs" routine, however it works, probably contains a bit of pathfinding anyway, no matter what.
So, tl;dr.... I think a gazillion of items lying around somewhat taxes gamespeed. But with only a handful of dwarves you will barely notice it. Once there are more guys running around, the effect becomes more and more noticeable.
Solution ? Train catapult users or turn the rocks into stonecrafts and use my automated waste disposal system! (stone crafts float
)
edit: another way to put it, since I feel i didnt make quite clear what I mean: a lot of items lying around probably make any possible shortcomings of the pathfinding more noticeable.