I'm inclined to have some humility and take them at their word, not being in their head.
That's our key difference, then. If someone tells me their religious experience, I smile, nod .... and don't buy a word of it. Similarly, [name] at work who is a perfectly rational fella yet also often mentions that he's repeatedly haunted (the freezer at work in particular seems to be the place for a supernatural visitation) gets treated as a reliable source up to (but not including) said stories.
As for the example of intersex people, it should be noted that the figures are by no means concrete. It's certainly an issue which challenges the definition of sex, but the extent to which this is so is not clear-cut. I'm no biologist, but the term 'intersex' seems to be an umbrella term for many different things. So, yes, a number of people, many (most?) of whom are intersex, pose a challenge to the typical binary. It's just such a large and varied grouping that I'm at a loss trying to make a general statement concerning it.
It is also the case that LGBT people are not intersex as a general rule, of course. A transgender person, typically, fully fits within the aforementioned male/female divide, one which it is impossible to fluctuate between.
Perhaps we should add a third sex? Male, Female, Neuter (intersex)? The question is how many people truly fall into something which deserves its own category, and how many have something like Congenital Adrenal Hyperplasia, which appears (again, me no Biologist, hyuck hyuck) to be a hormonal imbalance that would fall under "all else being well..." or, similarly, Vaginal Atresia (for lack of a better description, 'vaginal obstruction'). It's a tricky issue, that's for sure, and one which requires more thought and time than I'm currently willing to dedicate. It also seems to require more (and more accurate) studies.
Though, it should be noted that if 'Intersex' were to be acknowledged as a separate sex-definition (perhaps defined as a natural lack of reproductive capabilities?) it would likewise not be strictly (though idealistically, sure!) possible for someone who is intersex to become someone who is male, or female. Perhaps the reason we don't have such a three-way sex system is because it's rare.