0.40.22 (?) or thereabout changed the jobs handling quite a bit.
- One change is that wall (etc.) construction is now an unskilled labor rather than a carpenter/mason job.
- Another change is that DF goes to think on what job to take next, so e.g. pulling a lever on repeat gives a very uneven rate.
- Job selection is now partially based on proximity, so the stack based build order of walls is no longer a thing.
- Job priorities have been introduced for a number of jobs, so you can e.g. designate digging with different priorities to control the order they're performed in (that does not stop a second dorf from taking the next lower priority job after the first one, but getting it done faster, however).
I can't say I've had any great problems with the "thinking about what to do next" delay. Usually the dorfs move some 10 tiles before returning (or taking another job), not trek all the way to the meeting hall. However, the jobs rewrite made the idle count basically useless, since it includes "between jobs".
I did have great problems with workflow screwing up all kinds of jobs, so I've completely stopped using it, while still using DFHack. However, you say that's not it.
One major cause of problems for me has been bins. Anything that interrupts one repeat job cancels the job rather than just that particular job instance (trying to use items pushed around by the surf is great fun...). When a bin or barrel is accessed to put something in it or remove something from it the bin is blocked for all other access for a substantial amount of time, causing the items in it to be inaccessible, and thus cancel the jobs. That issue existed in 0.40.19 as well, though.