I don't think it'd be such a huge change, assuming relationships form through direct interaction between dwarves, then there'd be checks for eligibility of forming a relationship already. this would check both parties are single, basically checking that partner = 0 for both dwarves, and also a check for compatible genders. Toady could easily separate the line of code for the gender compatibility check off into a separate function, if he hasn't already. Then, change that line of code to consider castes rather than the direct gender tags, and compare to a table of compatible castes for that race (the possible pairs would be specific in the race raws) This code would return a boolean yes/no for the match, so the detail of how it's done would be isolated here in the code.
Nothing else about how castes work need be affected by this, it's just creating a new use for the existing system. Proceating children would still consider the [MALE] and [FEMALE] gender tags.
With the above set-up you could define castes for gay/bi/straight in male and female varieties, set the frequency of occurance in the population however you like by tweaking POP_RATIO's, and define a relationship matrix as a set of pairs, specifying which combos would work, and which would not.
Sure, you can't change the caste of a creature during it's existence, and that's a limitation of the idea, but remember the average player rarely get to see what's under the hood of the game. So some dwarves would appear to have a specific preference, and other dwarves would seem to oscillate between the choices, with the system I just outlined, which would seem pretty realistic.
This, or some similar, but hard-coded and more elegant solution, is what I think would work better, where it's an accepted background variation in behaviour, rather than a Message (with capital M) about real-world persecution of specific groups.