Maybe you should be happy about your looks?
I'm more about function than aesthetics. If I want people to know me, personality comes first, looks are a bonus. Plus, I figure that I might as well judge myself as I judge others, therefore if I look too good and see myself in the mirror I'll immediately look past my personality and mark myself off as a total dick and re-adjust to my actual personality as my looks. So to put it, I'd rather be fun and relaxed looking than an overly-groomed person that you'd see in just about any ad.
You know, I'd rather look real. Plus, I tend to have a bad reputation with anyone of a higher class, even if they aren't and just look like they belong in it. Plus, I hate wearing a mask of sorts; better to be genuine and not force people to see a different side of me that doesn't even exist.
Also to put it, I don't mind looking good, I just have no use for it. Both for moral and purpose-based reasons.
Plus, I also have a security mindset to almost always assume everyone is as critical of me as I am of them. Think as if they think on equal ground/terms and work with it accordingly. It's surprising how much I actually tend to out-think/outsmart people half the time. And on that note of phrasing, I've noticed that I'm one of the harshest critics I have ever known; and pessimism after previously been shattered out of optimism as a personality trait doesn't make it any better.
EDIT:
Now that I also think of it, a reason also for my harsh criticism, even towards myself involves a bit of
this. Of course, my awareness of it allows me to turn into a personality and aesthetic chameleon. I suppose this
trope can also be used for reference. At least I make attempt to know someone before I finalize my interpretation of them in case they do the same thing and blend with their surroundings as well as I do. Kinda like a "ninja vs. ninja" or "sniper vs. sniper" match. Best to get the first shot in before you get shot yourself.