It would be absurd because straight people are not a group with limited rights, they are not, as a group, oppressed or discriminated against structurally or personally as a regular occurrence.
Straight men who /are/, however, are more than welcomed in the LGBT. Bisexuals were considered "straight", and were welcomed. Those pushing the limits of genderqueerness were often technically straight, but they were welcomed. This is the future of the movement, a breaking down of the exclusionary walls that many in the early movement built around themselves, and it's a good thing. These are often straight people discriminated against for not acting "straight" enough despite that. There's was a time when folks went "their fight is not our fight, they do not suffer the way we suffer to the extent that we suffer, they were never as disadvantaged as us or, if they were, they were that way by choice, and our movement should not cover them." Those people were thankfully given a smack down, and the movement went on.
Men ARE oppressed. This is a simple truth. They are discriminated against, their actions limited, their options dwindled, because of their gender. Not in the same way or to the same extent women have been and are, but oppression of the same kind, nonetheless. It is exactly the case with gays and "straights" in the form of bisexuals, genderqueer, and today transexuals (and the hatred of transexuals is not an uncommon thing in the community). They did not say "No, our movement is just for gay men". They expanded it, and said "Our movement is for anyone who suffers under the yoke of expression because they do not conform to societal expectations of sexuality."
This is what women need to do. They do not need to represent "all men". But they DO need to start representing those men that are oppressed because of their gender. They've certainly never bothered to try and represent "all women", so I don't think this is beyond the pale. And unlike with the gay movement, this isn't even merely an act of doing the right thing - this is necessity. If they don't do this, the goal? Equality for women? It is impossible.
Of course, for some feminists, this is NOT the goal, and I never expect them to accept "outsiders" into their ideology. They want strict superiority. And those sorts of folks? They can go fuck themselves.
Most feminists want equality, and that will never, ever happen until they tackle the causes of the diseases, the roots that afflict both women and men. To exclude men is not just selfishness, it is prideful, and self-defeating. The right thing to do is to accept them into the greater movement, not as allies, but as fellow victims of cultural oppression, victims of tightly choreographed gender roles that will never allow the genders to play on equal footing.
But it's not surprising - feminism is full of plenty of absolutely terrible people, many of whom believe that simply because they are women, they cannot be privileged, many of whom are in leadership positions and who claim to speak for the movement as whole. There is a reason feminism is constantly taking heat - from black women, whom it excludes because "race isn't a feminist issue", from the genderqueer and transgendered, who they accuse of being infiltrators or traitors, from men who suffer at the hands of a society that expects them to conform just as it expects women to conform. And while it does not overwhelm the movement, there does not seem to be any cohesive resistance against these people, because to do so, to be more inclusive, would somehow "weaken feminism", as I've heard it described.
It is absurd.