I've already put a lot of thought in that matter, mostly because my girlfriend have the exact same problem.
We all know you're a mathematician, she is finishing her last year of master in physics, and you're both have pretty much the same problem : you dress yourself as you like (prettily in her case, and your description sound cute too), yet poeple doesn't like that and you identify the way you dress and make up as the problem.
The actual problem is : you're both girl with analytics mindset, and you've spent the last few year carefully dissecting thought process. Yet worse, you did it mathematically, which mean perfectly rationally, explaining every step of your reasoning.
Most of the reaction of your entourage doesn't make sense anymore, save for a few poeple (usually quite logic and that you know well).
When you're a man, especially in the scientific field, that doesn't matter too much : you're a nerd, and poeple can understand that.
When you're a girl, you MUST LOOK GOOD.
There is a lot of reason for that, none of them making sense by themselves, all perfectly explainable through sociology/history.
Fortunately, there is a few workaround : when you don't plan to get any advantage, dress as you like, it doesn't matter.
In a professional environment, there are rules for dressing, especially when there is something to be gained (work interview...).
Keeping a few neutral outfit in one's wardrobe can help. Usually selected with the help of friend/family (I do it myself).
But don't worry too much : first it only really mater with poeple you don't know well or who are idiots, secondly it IS pretty. Just not trendy.
It won't matter to your boyfriends/ potential boyfriends. Any misconceptions on your behalf will be corrected once they get to know you.
Anyway, getting along with poeple isn't your main asset, and impressing them on first sight isn't all that important when you're actually a good mathematician, let these doctorates and the idea you've had for the project you aimed to do their jobs, instead of trying to dress trendily for a job interview (I'm not sure how it would impress the old professor you're likely to face anyway).
Last thing, even if it's incredibly annoying, the poeple who give you these advices actually thing they do for the best.