Nonono, I mean the killing the best ones specifically--sport hunters usually go after the bucks with the most magnificent, er, antlers? Horns? Which generally signify the deer's overall physical capabilities. If a disease starts taking out the deer, then leaving only females and the weakest of the males could lead to a localized extinction.
... thing is, we could honestly kinda' do with a fair number of localized extinctions right now, so far as I'm aware. It could improve things for a fair chunk of the country vis a vis deer population. Others migrate in once the local population is dead, usually, which thins out saturation and whatnot, and the deer population density is a pretty significant ecological problem in some areas. And for some idea of the scope of things, we have more deer than we have asians in this country. More deer than every non white, black or latino demographic combined. Nearly as many as we have black folks. We have a lot of deer, and best conditions can see the local population
double in
two years.
That said, it does seem like we're starting to come down a bit, which I actually wasn't aware of, heh. There's still more than enough room for some pretty massive die back, though.
... though beyond all that, the largest point bucks are usually relatively old. Not always, but it's pretty likely. They tend to spread the genes around just fine before sport hunters start going after them. Antler size also isn't really the best indication of health, particularly genetics wise. S'mostly just indicative of available diet, s'far as I'm aware.
And yes, folks, when the estimated carrying capacity (i.e. the number that can inhabit an area before it starts becoming more likely to cause damage) for deer is around eight per kilometer and there's regions in the US where the actual density approaches a
hundred per km, it is, in fact, something of a stateside political issue. Far from the largest, but it's definitely there.