Is he psychologically a skiver? (If so, maybe not your best choice for the position[1].)
I just tend to remove absolutely every 'proper' job from my broker's itinerary, suffer the call of food/booze/bed[2] and get him/her there. Once I see the broker is in the depot, I know I can (usually!) trust them to stay there for the duration, barring dehydration/starvation prompts to leave (I'm sure), even while I'm still hurrying to get those last few individual shale crafts there[3].
Much as the still-being-hauled item might be a process-stopper (as described, and with the resolution as given), is it possible that they were actually on their way to "store a sock", or something else that they were actually wearing? Apart from forbidding their
entire clothing range, I think the solution would be to forget about the burrowing and let them get on with it.
Or else unburrow for a moment, and see if they head off on a purpose. And then perhaps include in the "You must trade!" burrow also the location they're naturally heading for (e.g. their own personal coffer), to let that activity finish and then the
next job (the Trading) is selected and commenced, with minimum fuss.
[1] Although, not knowing who else you've got available, I could still see them being the best of a bad lot...
[2] Usually by ensuring that I'm not waiting until the "the Tradrs are leaving soon" message before arranging for him to stop his other work, so that he can get a good meal or two, drink or six and 40-to-80 winks in, if necessary, and still have time to wander over to the depot while it's still relevant.
[3] In the early days, when I've not already got wealth beyond the dreams of avarice waiting for swapping for the
entire caravan... Later on... well, not a problem.