I don't believe "you can't assess who's ugly." I'm not accusing you of it, but many who claim people can't be protected (legally) from appearance discrimination because (the common arguments) too few people would voluntarily take up the label or it's too difficult to confidently categorize people as ugly are being disingenuous. Researchers have been studying perceived attractiveness (high in-and-between cultures agreements, like Trujillo LT, Jankowitsch JM, Langlois JH. Beauty is in the ease of the beholding: a neurophysiological test of the averageness theory of facial attractiveness. Cogn Affect Behav Neurosci. 2014 Sep;14(3):1061-76.) and its social costs/benefits for decades. And evolving technology (like Rojas Q. M, Masip D, Todorov A, Vitria J (2011) Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models. PLoS ONE 6(8): e23323) is consistently showing that technology is able to predict with high accuracy how people judge each other aesthetically.
The evidence continues to grow that appearance is NOT merely subjective, that it can have significant impacts on quality of life, and that technology--already remarkably good at it--is getting better and better at judging human appearance.