"ugly" people (i hate the term) are far more abundant then "pretty" people (hate that one even more)
thus we can assume its likely that being ugly somehow is evolutionary advantageous, while selecting a preferably "pretty" mate is evolutionary advantageous as well
as i see it, it could work out as "pretty" people like to mate with other pretty people because it possibly reflects to better quality genes, higher status, etc etc but selecting out too much limits your options on mates and creates a smaller genepool to pick from, thus quality above quantity (if i can say it so crudely)
however, uglier people will overall have a more diverse genepool, and are with a whole lot more then pretty ones, thus there chance on finding a mate can be seen as relatively higher plus gene faulting will be considerably slimmer
its obviously far more complex, but it comes down to being very pretty is disadvantageous, being very ugly is disadvantageous, being in the middle, leaning to pretty is best, and because the "middle leaning to pretty" is best... prettiness as we see it today does not rule tomorrow, and ugliness we see today will not be extinct tomorrow
basically its a social mixer, keeping everyone mating with everyone (in stead of people who think A is pretty and B is ugly to only mate amongst themselves, and people who think B is pretty and A is ugly amongst there's so the species does not fracture into a thousand parts so fast)
i am not an English native tongue, but i did had a field in university concerning genetics