- As a reminder, dwarves have a limited carrying capacity. They can only carry so much goblin shoes, depending on their force and containers. This will naturally limit the items we have to consider.
Oh, did I forget to mention that? I meant to. If you are only pathfinding once you have actually decided to try carrying something, then you don't need to pathfind any more items than you can actually carry.
- Grouping jobs: instead of striving for maximum time and resource efficiency (every free dwarf must haul if there's an item to be hauled), strive for organizational efficiency: all hauling jobs for a group of items are grouped, and assigned to a dwarf. That dwarf goes over there, takes what he can carry (preferring items for the same stockpile(s)), goes to stockpile it (visiting several stockpiles in a row and dropping of items, if necessary), and returns to pick up more. When he goes to sleep somebody else can be assigned to continue the group of jobs.
I was going to try to address Draco's post with this, but... I'm not sure if looking for things "on the way" is worth the pathfinding cost, actually. Basically, we could just go by absolute job starting and ending point definitions of "nearby". It means that maybe the haulers will go to a distant site, pick up half a wheelbarrow's worth of socks, and roll right over a sock on the way to the sock stockpile, but that at least prevents additional limited-range flood pathfinding along an already pathfound route.
Basically, I'd think it would come down to how likely it is that you would save trips and jobs and extra full pathfinding tasks by doing so compared to the costs of doing so. You'd have to test this with plenty of live forts to really get a sense of the average number of jobs you'd save for doing this.
If a goblin dies, and the stockpile for metal armor are clearly in another part of the map than the stockpile for fabric and leather clothing, then there should probably be completely different tasks for dwarves to jump for. If there are a large enough number of items that the dwarf cannot claim all the hauling jobs for that pile of crap, then the jobs that he/she cannot claim should still be left up on the jobs queue for other haulers to try to take.
- Ad hoc: a dwarf goes to an item to be hauled, picks it up and checks for a similar nearby item (using a limited floodfill like Draco suggested; the distance of the floodfill should be larger if the trip to the stockpile is longer (people think twice whether they have everything before leaving the store and driving back home, but if you're taking food out of the pantry it's not important if you have to return one more time)), if he finds one he picks it up and checks for more until his backpack is full and goes to the stockpile, if he doesn't he goes to the stockpile.
Actually, if we're dealing with wheelbarrows or even mine carts that are "mobile stockpiles", then wheelbarrow-hauler dwarves should have a much larger "nearby" definition, so that a single guy with a wheelbarrow might try to take up every single goblin-corpse-clothing-looting job that he can physically cram into his wheelbarrow. Provided that the siege goblins all died relatively near one another, you could set a ten or twenty-tile "nearby" radius or something similarly large, and use flood filling around the corpses to see a decent path between which corpses to visit. The flood-fill would be costly, but it should only take one or two dwarves to even do the entire task, as opposed to the hundreds of trips you'd be saving in the long run.
- Central depot stockpiles: for large quantities of diverse items (eg. battlefield remains) it makes sense to haul everything to a central location (inside your fortress, presumably) and sort it out later.
The way it could work is to flag a stockpile as central/depot: this stockpile would then get priority as a destination for non-stockpiled items. However, it would also be emptied ASAP by moving stuff to normal stockpiles. That way you could set up a central stockpile for everything to start, and later diversify with other central stockpiles when you don't want corpses to be brought into your central hall, or set up the ore processing facilities elsewhere.
This makes a lot of sense, but may be tricky to get coded.
Of course, this pretty much describes what the minecarts are supposed to be and do, anyway. You make a minecart track to near where the hauling jobs are getting created (battlefield, mining quarry, farm, underground tree factory), and then shove the mining cart back to the central distribution area, where it can be distributed to the workshop resource stockpiles.
Ah, three ninja attacks... I'll post and sort that out later.