It's not just "male culture," by the way, because the guys I know in the math department and many other places just don't do it. We trash-talk each other, I make fun of the guy's hair, he calls me a bitch, it's all good. That usual interaction, though, takes into account teasing that is just too hard and too nasty. When someone says something that might be seen as way too nasty, they immediately whisper "just kidding," and we get back to insulting each other.
And the thing with guys, is that it seems a lot of the "animosity" and insults are just for fun, most of the time. It's on a similar level.
But there is this thing that can often happen with women where they speak seriously and get called all sorts of names. I don't know what men expect when they say things like "A woman could never _________, she's too damn emotional! Get back in the kitchen and make me a sammich, bitch!" (Note: I only expect men to say things like this on the internet, and their derivatives in real life). Do they expect us to make fun back? Say something nasty about men? Say "We're only making your sandwiches because you're too fucking stupid to slap peanut butter on a couple slices of bread?"
Basically, what I'm saying is that I think the behavioral explanation requires some sort of expected response from the partner, because otherwise it is just one-sided instead of a paired insulting activity. I don't mind kidding around a little even when I'm extremely upset, but I'll eventually say "Look, sorry, I really need to talk to you about this instead of joking right now." But after that, most everything someone says is taken on the serious side, and the "takebacks" just seem unfair. People screw up. I've screwed up and said things that were just too hurtful, and usually people say "what, are you saying ________," and then I say "Augh, fuck, no, I'm so sorry." But this sort of behavior... I really do think it is uniquely distinct. The takebacks aren't necessary, only constantly undermining someone else's emotional reactions, shifting blame to them whenever possible, etc. By the end, you feel so unbearably crazy. I can't even really explain it, at this point. It's just dreadful.