AI And Discrimination In Insurance

This past summer, a group of African-American YouTubers filed a putative class action against YouTube and its parent, Alphabet. The suit alleges that YouTube’s AI algorithms have been applying “Restricted Mode” to videos posted by people of color, regardless of whether those videos actually featured elements YouTube restricts, such as profanity, drug use, violence, sexual assault or details about events resulting in death.

The lawsuit alleges that this labeling has occurred through targeting video keywords like “Black Lives Matter,” “BLM,” “racial profiling,” “police shooting” or “KKK.” YouTube says its algorithms do not identify the race of the poster.

Whether the allegations are true or not, the case illustrates AI’s potential for inadvertent discrimination.

It is easy to see how an algorithm could learn to use variables seemingly unrelated to race, sex, religion or another protected class to predict the outcomes it was designed to target.

In the YouTube example, we could imagine the algorithm noting a link between the mentioned keywords and videos depicting violence, thus adding the keywords to factors it weighs when deciding whether Restricted Mode should be applied to a given video. The algorithm is simply programmed to restrict sequences containing violence, but in such a situation it could end up illegally restricting videos posted by African-American activists that depict neither.