March 8, 2021

Amazon Web Services launches new tool to detect bias and blind spots in machine learning

Dr. Nashlie Sephus, an applied science manager for Amazon Web Services AI, introduces the new Sagemaker Clarify feature at AWS re: Invent Tuesday. (Screenshot via webcast)

A new feature from Amazon Web Services will alert developers to potential bias in machine learning algorithms, part of a larger effort by the tech industry to keep automated predictions from discriminating against women, people of color and other underrepresented groups.

The feature, SageMaker Clarify, was announced at the AWS re: Invent conference Tuesday as a new component of the AWS SageMaker machine learning platform. The technology analyzes the data used to train machine learning models for telltale signs of bias, including data sets that don’t accurately reflect the larger population. It also analyzes the machine learning model itself to help ensure the accuracy of the resulting predictions.

A 2018 MIT study found that the presence of a disproportionate number of white males in data sets used to train facial recognition algorithms led a larger number of errors in recognizing women and people of color. Amazon itself was reported to have scrapped an artificial intelligence recruiting tool that turned out to be biased against women, in part because the data used to train the model came from observing patterns in past resumes submitted to the company, the majority of which were from men.

“Bias can show up at every stage of the machine learning workflow,” said Dr. Nashlie Sephus, an applied science manager for Amazon Web Services AI, introducing the new feature at re: Invent Tuesday morning. “So even with the best possible intentions, and a whole lot of expertise, removing bias in machine learning models is difficult.”

She cited examples such as building a TV show recommendation algorithm without enough television dramas in the training data. There’s also the challenge of “model drift,” where the training data end up being substantially different from the data used to make predictions, such as changes in mortgage rates causing a machine learning model for home loans to become biased.

SageMaker Clarify is one of a series of machine learning features and products announced by Swami Sivasubramanian, Amazon VP of AI, at the virtual conference Tuesday. The next major re: Invent keynote is Thursday with To fart DeSantis, vice president of global infrastructure.