It happens with all dwarves & all jobs, actually. Depending on how your fort is laid out, a dwarf will go to the main food stockpile and say,
"Now, why should I eat my lunch in the perfectly good dining hall, which is literally right through that door, when I can easily carry my lunch through the dining hall, up four flights of stairs, past the trade depot, around the hospital, and finally eat it in the church where I'll get annoyed at being 'forced' to eat without a proper table?"
When choosing the destination of a Miner's next task, or the spot where a dwarf goes to eat, the game only checks the distance between the starting & (potential) ending points, not the path that the dwarf would have to take to actually get there. The game only calculates the pathing after the destination has been chosen.
So, to put my suggestion more clearly: When a Miner has dug out all tiles adjacent to his current spot, he should move to the work-adjacent tile with the shortest walkable path to his current position*. This would optimally be done by checking if there's a path to such a job site within 5 steps, and then (if the answer is no) within 10 steps, and then 15, 20, & finally a path of 25 steps, and if there are still no "hits" it finally defaults to its current algorithm--finding the job site the shortest distance away, regardless of pathing.
* Actually, there should be a modifier to prioritize squares that are adjacent to multiple tiles designated for digging, in order to maximize the work that can be done before the pathing calculation needs to be done again.