Then the attraction isn't to anything that's actually there, in any meaningful sense. Or at least not anything that has to do with biological gender or "maleness". So-called masculine behavior doesn't have anything to do with genitalia, nor does it really have anything to with being classified or self-identifying as male; it's just a particular set of behavioral patterns.
There's nothing preventing someone that just happens to have female reproductive organs from manifesting those patterns. What about a pre-op FtM, ferex? What about men who are very "feminine," but happened to have all the other behavioral nuances you're looking for?
Point being that it's either the physical aspect that you're fixating on or a set of behavior patterns, because there's no necessary connection between the two. Basically, it's either the cock or it's not the cock; in the case of the former, it's purely physical (and the pre-op MtF would be fine, if everything else was in order, though post-op wouldn't be.), in the latter, not actually anything to do with your intended being male. It's just that it happens that males have tended to express the behavioral patterns you find attractive up to this point. Which is fine either way, really. Most of our species has managed all throughout time to get by with even less of a consideration of an issue than that.
Or at least that's more or less how I see it. I'm mostly attracted to women because I've got a greater aesthetic fondness for the female form than male more than anything else, because both the major behavioral manifestations of so-called "masculine" and "feminine" behavioral patterns just get on my nerves. If a bloke happened to 'ave th'mounds an'cleft o'Venus along with th'wedding tackle and was able to avoid most of the violently annoying cultural artifacts attached to gender, I'd be as attracted to them as I would to a lady. A futa is okay too~
Hell, just the latter half of that qualifier is sufficient, really. Gender is only tangential to that, not integral, though. That's part of the whole "Human before man or woman" thing.
Anyway tl;dr [/ramble] version: Male isn't actually a thing that exists outside the biological. Neither is female, really.