I only have seven miners. Two are level 3, two are level 8, and the rest are legendary. It did not matter who was digging. And once I shifted the burrow definition one square left to include the square they were standing on they dug it right out.
I was also seeing the same behavior with woodcutters cutting down trees. Chopping for a month until they got thirsty and left. I shifted the burrow border to be just to the left of the tree as well and it came right down.
I usually start with two miners, and by year 1055 (i.e. five years into my forts) I tend to accumulate a couple more from immigrants with decent enough mining skills that I can't afford not to draft into the team. The current fort I can easily check is on Summer 1055 and I have three Legendary+5 (=20) miners active, one Great (=10) Miner who spends part of his time in an 'irregular' military unit (had the auxiliary military skills for it, and you never know when a pick-axe and other arbitrary fighting skills might come in useful to assist with the regular military units) and six dabbling (=0) miners who've never been let near a pick and probably never will be.
So, I just set up a bit of an experiment. With a "tempMoreDitches" burrow (one I had already set up to get some surface defensive features channelled out over pre-existing dug-out underground drifts, so as not to leave ramp-access) and the Great miner currently tackling a north-south row of ditches I changed the burrow designation so that instead of being only over the to-be-channelled tiles: one third of them had the burrow designated also to the western-access tile (from which the Great Miner was already cutting my ditches); another third of them had the burrow also designated over the eastern-access tile; the remaining third stay as they originally were. This was alternating between the three, rather than runs of one type, then the other, then the other...
Not only does the miner not have any noticeable difference in speed as it progresses downwards (if I had done ten of each at a time, maybe I could see a subtle timing difference), he doesn't chop and change to dig from the RHS of the to-be-channelled tile, but sticks to the LHS of all of them, regardless of whether the burrow covers that.
But it might be different if I'm not channelling into a void. I don't have anywhere suitable for that test, though, in this fort, so I'll have to come back to this (or start up a new world/prepare to save-scum this one to undo my vandalism), so I'll put that forward only as a theory, for now. Right now, I've got some human traders to
foist my useless crafts upon trade with.