Amazon Killed Its AI Recruitment System For Bias Against Women—Report

Authored by fortune.com and submitted by Iliketoreadit
image for Amazon Killed Its AI Recruitment System For Bias Against Women—Report

Machine learning, one of the core techniques in the field of artificial intelligence, involves teaching automated systems to devise new ways of doing things, by feeding them reams of data about the subject at hand. One of the big fears here is that biases in that data will simply be reinforced in the AI systems—and Amazon seems to have just provided an excellent example of that phenomenon.

According to a new Reuters report, Amazon spent years working on a system for automating the recruitment process. The idea was for this AI-powered system to be able to look at a collection of resumes and name the top candidates. To achieve this, Amazon fed the system a decade’s worth of resumes from people applying for jobs at Amazon.

The tech industry is famously male-dominated and, accordingly, most of those resumes came from men. So, trained on that selection of information, the recruitment system began to favor men over women.

According to Reuters’ sources, Amazon’s system taught itself to downgrade resumes with the word “women’s” in them, and to assign lower scores to graduates of two women-only colleges. Meanwhile, it decided that words such as “executed” and “captured,” which are apparently deployed more often in the resumes of male engineers, suggested the candidate should be ranked more highly.

The team tried to stop the system from taking such factors into account, but ultimately decided that it was impossible to stop it from finding new ways to discriminate against female candidates. There were apparently also issues with the underlying data that led the system to spit out rather random recommendations.

And so, Amazon reportedly killed the project at the start of 2017. According to Reuters, the company’s own human resources department used the system to generate recommendations, but never relied entirely on those recommendations when filtering candidates.

Amazon did not immediately return Fortune‘s request for comment on the Reuters report.

Amazon isn’t the only company to be alert to the problem of algorithmic bias. Earlier this year, Facebook said it was testing a tool called Fairness Flow, for spotting racial, gender or age biases in machine-learning algorithms. And what was the first target for Facebook’s tests of the new tool? Its algorithm for matching job-seekers with companies advertising positions.

howdoiland on October 11st, 2018 at 01:57 UTC »

Tay lives

toadnigiri on October 11st, 2018 at 01:26 UTC »

I don't think this is a good idea to start with, soon enough people will learn how to fill their resume with the right buzzwords.

spidersnake on October 11st, 2018 at 00:23 UTC »

I know it's terrible but I can't stop laughing at the idea of a sexist computer.

According to Reuters’ sources, Amazon’s system taught itself to downgrade resumes with the word “women’s” in them, and to assign lower scores to graduates of two women-only colleges. Meanwhile, it decided that words such as “executed” and “captured,” which are apparently deployed more often in the resumes of male engineers, suggested the candidate should be ranked more highly.

The team tried to stop the system from taking such factors into account, but ultimately decided that it was impossible to stop it from finding new ways to discriminate against female candidates.

I wonder why it was trying to discriminate against women from the start? It doesn't mention what metrics it was using to do this in the article sadly.