There's a type of plant called "Wandering Jew". Is it anti-Semitic to own one/call it that?
It depends, am I racist for calling a large rock just poking out of a field a "niggerhead"?
That one's a little trickier, IMHO. Just like my grandfather used to call Brazil nuts "n****r toes". It wasn't meant in a particularly racist way, that was simply the local name for them. But obviously it comes from a racist etymology.
To be honest, I'm not even sure why the plant is called Wandering Jew, other than it's a vine that spreads all over the place.
To go back to your argument that the term must be offensive because it has roots in a stereotype, let's return to the counter-example of "Yankee". One solid argument for the derivation of the term is that it came from the British soldiers derisively calling the residents of conquered New Amsterdam (later New York), "John Cheese" because the Dutch were known for their cheeses. In Dutch, this would be rendered Jan Kaas and pronounced roughly similar to "Yankees". So....the term Yankee comes out of a negative stereotype of Dutch colonists. Somehow I don't think the Dutch take offense when someone
else gets called a Yankee.
I realize my example is a fair bit more convoluted than the path to the word "gypsy", but the principle is the same: just because a word was born out of a stereotype doesn't make it verboten for the rest of time.
I still maintain that because of the absence of any significant Romani population in the United States (and Australia), there is no deliberate connotation of relating to the Romani people (or any associated stereotypes) when the word "gypsy" is used. It's seen as a (often romanticized) lifestyle, not an ethnicity. There's a dim understanding that it's modeled on a people that existed at one point in Europe, but it's a very tenuous connection for most.
I think part of the problem is hyper sensitivity for a very old word that has changed in meaning. Heck, a bit of cultural tolerance and you will find a lot of things that are 'offensive' are a term of endearment.
Do you know what people from New Zealand will sometimes call Australians? "Roo-fuckers", can't get more blunt than that now, but does anybody really take offence? No, we do not! Because even though it might have started off as a slur, nobody uses it as a slur any more. I have had Kiwi friends call me that, and did not give a damn or feel sore, because there was no negative connotation.
ROTFL...at first I thought you were going to say, "No, we do not, because we don't do that anymore!"
Isn't "sheep-shagger" a counterpart not-really-a-slur going back the other way to New Zealand?