I'm a vegetarian myself, and fully support Vaftrudner's arguments.
I'd also like to point out that even if drinking milk is bad morally in some way, it's still better than drinking milk AND eating meat (unless you replace the meat totally with equivalent amounts of milk in your diet - equivalent in terms of ethical consequences). Personally, I've always consumed a lot of milk and becoming a vegetarian did not change that.
Drinking milk but being a vegetarian is not hypocrisy if you acknowledge the negative consequences of the former (if any). It's just a matter of acknowledging that you're not perfect, but take steps towards the right direction. Otherwise, the implication would be that you should not ever try to be ethical because you will never be perfect.
IMO, it's a flawed argument about the need of being consequential/not a "hypocrite" that keeps many people from improving things (e.g. "if I would stop eating meat, shouldn't I also stop drinking milk, wearing leather, stepping on insects, etc., so instead I do nothing at all"). I guess that reasoning kind of makes sense if your ethics are based on principles, not consequences. For me it's the latter, so even eating just half the amount of meat is already a laudable improvement over the status quo for me.
Also, as has been pointed out before, that other people do not have the choice to make the right ethical decision does not imply anywhere that I should not make it for my part - oh, and as others have pointed out, there is not only the ethical but also the ecological (and indirectly again ethical) side of vegetarianism. Those people "not trying to die of malnutrition" would overall be better off if not so many resources would be wasted towards the luxury of meat production...