Could be a side-effect of your fort layout. There used to be an issue with the z-axis not being considered in path measuring for job items. I havent trawled the releasenotes of DF2014, so if this bug is gone then its recently gone. But you used to often see a mason running 100's of squares for a rock, and upon closer inspection you realise hes clearing the rocks directly beneath his workshop all the way to the magma before moving 1 step horizontally for a rock from the pile. By the time hes using the pile properly he can have cleared literally 1000 rocks that were beneath or above him and 'closer'. The reason you see this specifically with stone is of course the prevalence of it lying around un-piled compared to other items. When every tile dropped a boulder this problem was excessive.
If you really just want this to work then set the workshop to take ONLY from the pile you specify. Unfortunately wheelbarrows are only for traffic to piles, not for fetching items from piles (unless they are headed to yet another) or any jobs not involving piles.
Another excellent workaround is to make your stone pile 3x3 tiles directly above or below the workshop, the path to those stones will always be as short as possible, just have a handy stair nearby.
Edit: the other place you might see this is the smelter and metal ores, I always link my ore smelters to take ONLY from a designated pile. The only one not linked is my melt object smelter