To be hyper-specific they observe genitals at birth, not gonads. Sometimes that's an important distinction, which leads into why we say "gender assigned at birth". That does assume the child is cis... because that's what people usually do, at least for now.
It's a lot to get into, but sex isn't a simple binary. You've probably heard of intersex people, but you might not know how common intersex situations are. 1.7% according to Google, which sounds low- but is a lot more likely than sharing a birthday with someone.
Sex is often described as bimodal for that reason, which I understand to mean "There are two sets of of traits, but people have a mix of those traits".
Gender is indeed orthogonal to those traits. I personally take a rather absolute position that there is no inherent link between sex and gender. Gender is ENTIRELY socially constructed. When a culture recognizes two genders, which is common but not universal, those have (AFAIK) always been associated with the two sex-categories. Very roughly, male and female people are labeled men and women.
And that categorization certainly matters! More than I'd personally like, that's for sure, though I've been confusedly agender most of my life. Our cultures put a great deal of importance on gender, perhaps as much as other categories like race and economic class.
A child is taught these social constructs of "man" and "woman" and, for very complicated reasons we don't yet understand, tends towards one (usually) VERY early in development. That is a positive reaction to a cultural category. For example, choosing combat-figures over domestic dolls. Notably this response is limited by the options offered- options which are a result of culture, usually a gender binary.
Of course many people aren't ever offered this choice. They are given toys according to their sex, perhaps scolded for going after the wrong toys. But given enough freedom we see some kids express preferences that go against our expectations. This is consistent with my theory. The gender they're gravitating too is probably not a *perfect* fit, but gender never is - it's a choice between the options offered by culture.
Note that gender options vary EXTREMELY by culture. The meaning of "man" and "woman" vary wildly based on time, location, and economic class. A medieval male serf is more similar to his wife than he is to his feudal lord. There are physical differences - sex is real, if not purely binary- but gender? The roles we play in society? Gender is COMPLICATED.
Okay I've rambled enough, what was your last question... Oh, easy
In all our cultures to my knowledge, women are still women if they're infertile. We can imagine a culture wherein all (adult) men can fertilize, all women can be fertilized, and everyone else is of a third gender. Heck, we've observed that in history - eunuchs are described as distinct from men and women. Holy virgins and unmarried midwives might be similar.
So clearly it isn't really about fertilizing and being fertilized. There's simply more to it than that! It's a very complex and changing social categorization.