Its believed that jobs look for dwarves to do them, rather then dwarves looking for jobs. So there is speculation that each of your 76234897234879324 stones are looking for someone to haul them or stone stockpiles that need filling, or at least checking to see if they need to be looking for those things. Its all very plausible, but to the best of my knowledge not a whole lot of actual science or code analysis has been done on the issue.
IIRC back when job priorities is a common topic, it was said that Toady said jobs look for dwarves, and that's why implementing job priorities would be a bit difficult.
But I do not believe loose stones would cause job look ups - it doesn't make sense at all. The logical source of a hauling job would be an empty stockpile space that allows stone. Then a dwarf is selected, and a loose stone selected and claimed for the job. And I believe Toady made the same choice.
The worst a loose stone could do is causing a delay when you use the stone entry in stockpile screen, and a tiny tiny bit of memory used to store it, which is only accessed when used in some way and insignificant when compared to other things.
Personally, I think its very possible, but also possible that the empty space produced by mining causes lag (or more lag). Mining out an area generally causes pathing to have to check that area when trying to get somewhere above, below, or near it, if I understood that epic A* thread properly. Also since correlation =/= causation unless you're a politician it could just be that forts being big and old causes both lag and lots of mined out stone. Maybe I'll mine out some z levels and test it tomorrow?
It really depends on where that empty space is. At the end of a mining shaft that no one other than miners and stone haulers go, not so much.
In other places, the pathfinding algorithm is already looking at all nearby spaces - at the very least, it has to check if they are accessible (wall? another level but no stairs? etc.) or not. A dug space should give the path finder another possible space to consider only if it's accessible.
How to measure the performance though? I'm thinking about this:
1. Have a fort with a high number of dwarves, as few kids, animals and nobles as possible.
2. Make one big room filled with stones.
3. Make another room of equal dimension, but this one should be empty. This room should not be directly connected to the first room.
4. Save and backup.
5. Connect the two rooms with the path(s) you want to test.
6. Move all dwarves inside the rock-filled room, and seal it so the only path to the empty room is the newly dug test path(s).
7. Give everyone nothing but stone hauling labor.
8. Make a stone stockpile, or designate a dump zone and dump all stones.
9. Check FPS and time taken, then reload and test another path.