Sony's research has suggested a new way to measure skin color in AI systems for a more fair and diverse representation. It argues that the current systems' focus on lightness and darkness of skin tones is insufficient and overlooks other aspects of skin color such as red and yellow hues.
Considering color, beyond light and dark
The world of AI has been striving to make its systems less biased based on the lightness or darkness of people’s skin tones. However, recent research from Sony pushes this conversation forward by suggesting the need to also consider red and yellow skin hues. The authors of the study, William Thong and Alice Xiang from Sony AI, along with Przemyslaw Joniak from the University of Tokyo, propose a more multi-dimensional method for measuring skin color. They believe this could help foster more diverse and representative AI systems.
In recent years, there have been widespread efforts to address the issue of AI bias. Companies like Google have introduced initiatives such as the Monk Skin Tone Scale, which uses a 10-point scale to measure a variety of skin tones from dark to light. Similarly, Meta has used the Fitzpatrick scale, a tool that characterizes skin color into six categories, in its previous research. These efforts, though commendable, don't sufficiently address the complexity of skin color, according to Sony's research.
Overlooking key biases in existing scales
Sony's study puts forth a critical argument: both the Monk Skin Tone Scale and the Fitzpatrick scale are primarily focused on the lightness or darkness of skin tone, which leads to some biases going unnoticed. Alice Xiang, Sony’s global head of AI Ethics, expressed concern that products evaluated in this one-dimensional manner can still harbor biases that remain undetected and unmitigated. She cited biases against East Asians, South Asians, Hispanics, Middle Eastern individuals, and others, who don't clearly fit on a simple light-to-dark spectrum, as a significant concern.
The implications of this oversimplified approach to skin color classification are significant, according to Sony's research. The study found that common image datasets tend to overrepresent individuals with skin that's lighter and redder in color, while underrepresenting those with darker, yellower skin. This imbalance can affect the performance and accuracy of AI systems. For instance, certain AI systems were found to favor redder skin and even misclassify people with redder skin hue as 'more smiley'.
Sony's proposed solution: the CIELAB standard
To address these issues, Sony proposes a solution that uses the pre-existing CIELAB color standard, which provides a more multi-faceted and accurate measure of skin color. This automated approach would replace the manual categorization used with scales like Monk's. While the simplicity of Monk's scale is seen as a benefit by some, Sony's research suggests that it could be more valuable to offer a nuanced and comprehensive view of skin color, even if it's more complex.
Industry response to Sony's research
While Ellis Monk, creator of the Monk Skin Tone Scale, defended his scale and argued that it does take undertones and hue into account, there seems to be a growing acceptance of Sony's research in the AI community. Major players like Google and Amazon have shown interest in the findings and are currently reviewing the paper. This suggests that the industry at large might be ready to consider more complex and comprehensive measures of skin color in AI systems.