I'd rather eliminate racism, period. Sometimes getting to a higher maximum means moving away from a local maximum for a time.
Plus I do have the studies that find that having a black name results in fewer callbacks from resume applications. Obviously, this can't say too much about the hiring process, but at the absolute minimum, it's an implication, and more likely it's reflective of actual trends. I don't think these trends are necessarily as massive as some political groups say they are, and I think a lot are more reflective of historical racism than current racism - and I need to explain what I mean by that but I'll get there in a sec - but I don't think deciding this stuff based on virtue ethics is the right way to go about things, which is what it sorta seems like you're saying we should do. Unless you think that a white person failing to get that job affects them as much as a black person, who tend to have fewer resources available to them to fall back on as a default.
That said, affirmative action is still the wrong way to go about things in my opinion, I think the much more productive thing to focus on is those remnants of historical racism. Now, what I mean by this is basically all the factors about economic conditions and environment left over from the much more explicitly racist period. Helping get past those obstacles and improve conditions in the communities is probably the single best way to help reduce the incidence rate of reduced achievement or what-have you. Plus, it helps more than just one racial group; it helps anyone and everyone in those shitty situations, which are shitty regardless of race. Race can exasperate it but that's not the point.
Unfortunately, that's also really hard to fix and it's much easier to blame your opponents. Whether it's saying 'well they're just less able people, can't blame employers for not wanting them' or saying 'well they're just racists, as soon as they're gone all your problems will be fixed'. Much easier to do that than to acknowledge that yes, there is racism, and yes, there's a lot more there too.
Life is complicated. Science is hard. It is rare that one side has it all right and the other all wrong. People aren't typically stupid, but using your intelligence without it being hijacked by other parts of your brain is really hard, because your conscious mind is basically built to be lied to by the rest of your head, for PR purposes (more convincing to tell a lie you actually believe).\
Although on the topic of affirmative action I did get information for the first time the other day that made me think there might be something to the whole 'math tests are sexist' argument. Not that they are on purpose, just that when you remove the penalty for answering multiple choice questions wrong, the gap between men and women disappears. Most of the time, if you have to guess, then you're more likely to get points than if you left it blank, as long as you can be sure it's not one of the answers beforehand. Women were more likely to leave questions blank then guess if they thought there was a penalty for getting it wrong; they felt like they had to be absolutely sure before they answered, essentially (according to the theory). And if I ever refer to systemic racism/sexism, that's the sort of thing I mean; subtle things that have totally unintentional side effects.