Report: Amazon’s AI Recruiter Favored Men

from Axios

An algorithmic recruiter meant to help Amazon find top talent was systematically biased against women, a Reuters investigation found.

Why it matters: This is a textbook example of algorithmic bias. By learning from and emulating human behavior, a machine ended up as prejudiced as the people it replaced.

The details: Amazon’s experiment, which dates back to 2014, was trained on 10 years of job applications, most of which came from men, reports Reuters’ Jeffrey Dastin.

* The system concluded that men were better candidates for technical jobs.

* In 2015, Amazon began to realize that the system was penalizing resumes that included the word “women’s” (as in a women’s sports team or all-women’s colleges).

More here.

, , , , , ,

24 Responses to Report: Amazon’s AI Recruiter Favored Men

  1. Peter Duca October 18, 2018 at 2:27 pm #

    Amazon’s recent withdrawal on this AI project can be looked at as a failure for people who believe AI and algorithms can replace human recruiters. However, I feel that Amazon was the company that took the first brave step into this field, took some risks, and ultimately lost out on the experiment. I believe Amazon was one of the first companies to develop the program, and it should now be up to other companies to take the model created by Amazon and perfect it in order to streamline the recruiting process overall. It also gives other companies a good lesson to learn from; AI should not be completely relied upon just yet to analyze resumes and select candidates. While I do not believe that AI will ever completely replace a human recruiter, I feel that the technology could certainly be leveraged in order to analyze resumes at a faster rate and make the recruitment process more streamlined and valuable as a whole.

    AI programs face one giant limitation; the programs follow certain trends that they discover when given certain data. If the data is biased, or the trends are deeply ingrained within an industry, the AI program will follow the discriminatory trend presented to it. AI programs cannot create new trends; they go off of trends presented to them. A conscious effort from programmers of the AI, recruiters, and the industry as a whole will be needed to break down the current stigmas that surround the business and technology industries.

  2. Warren Vetter October 18, 2018 at 11:19 pm #

    The people that created the robots made these robots the way they are. The people that these AI’s replaced acted in the same manner. Although Amazon gave up on having AI being part of the recruitment process they took a risk and gave it a chance. Amazon is always taking a risk and that is why they are one of the most successful companies in the world. It feels like most companies are waiting for Amazon to find the next successful way to run the company. Nobody wants to take the risk of failure which is understandable, but if the risk turns out to be successful then they could have changed the way companies are run for a long time. I don’t believe in AI being part of the recruitment process for top companies because AI’s just scan the person’s resume or the answers they gave in the interview. AI’s were known for picking not so strong candidates which caused the company some problems. Amazon luckily was not fully relying on the AI’s decisions. This incident proves that not all robotic algorithms are 100 percent correct.

  3. Robert Musantry October 19, 2018 at 1:26 pm #

    The very title of this article was shocking to me at first. An automated recruiting machine having a bias against women? What kind of apocalyptic society do we live in? The truth behind those questions was even scarier than the questions themselves. It turns out that this AI was actually biased against women, and would discriminate against applications simply for have the word “women” in them. Now, of course, it was not designed this way. In the Reuters article, an unnamed source was quoted as saying “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.” Basically, Amazon had too many applications and needed a way to narrow them down to the best candidates for hire. To teach this AI how to do its job, the company gave it 10 years’ worth of application data to see what the company had looked for in the past. And, unfortunately, what the AI learned from this process was that the tech industry is male dominated, and so it began applying this knowledge to the applications it was sorting and made recommendations based on what it saw. Amazon stepped in when they realized just what was happening, and luckily they did not base their hiring off of blind recommendations by this AI, it was just designed to be a tool. After such complications, the team behind the AI was actually disbanded by Amazon since they were not producing the type of quality candidates Amazon hoped to gain from the process.
    Outside of Amazon, this idea of and AI-aided hiring process makes a lot of sense. Companies may have too many applications to go through manually, so having an AI do it is helpful. The AI can just be running all the time, and then when a real person checks in they will immediately see qualified candidates they can begin looking at. What we as society need to be worried about is setting up a “checks and balances” type of system, where such AI is not given free reign. As Amazon saw, creating an AI with good intentions does not mean the AI will be perfect. For the foreseeable future, humans will still be needed to double check the AI’s work, especially when it comes to hiring the most qualified candidate for a job.

  4. Steven Gravlin October 19, 2018 at 1:44 pm #

    To start, I love the fact that companies are starting to use AI recruiters to recruit employees. Also, Amazon was not relying on them entirely which is important since evidently there are some flaws. They help avoid human errors like biases. However, this only works when the artificial intelligence works the way it is supposed to and does not disqualify a potential star employee because they have “Women’s Soccer” on their resume. One of the major flaws that employers have is biases. Some employers are downright sexist or racist but other biases exist too that are less extreme like maybe one employer has an Ex with the same name of a candidate so they get a sour feeling towards this candidate for no reason other than their name. Another example would be if they are a big fan of a certain sports team and a candidate is a fan of a rival team. These qualities do not affect their work ethic or productivity at all but they still disqualify possible employees.

    Women make up half (actually a little more than half) of our population. It is extremely important to make sure women are working just as much as men. By doing a 2 second google search one can find that there is no difference between the intellect of men and women. This further shows that an AI recruiter discriminating against women is a huge problem-not a minor one. The point of using bots to recruit employees is to avoid biases so the company can hire the best employee based on qualifications and not race, sex, religion etc… I hope Amazon can solve this issue and other companies can avoid this issue in the future.

  5. Nora Trapp October 19, 2018 at 1:52 pm #

    Reading this article simply proved my belief: that technology cannot improve all human errors. There is no doubt that technology is continuously evolving; however, it seems that society is moving towards an ideal where technology can eliminate all of human’s flaws. This article proves this statement wrong. What shocked me was that a company as big as Amazon, who is especially known for their company culture and gender equalization, is a victim of gender discrimination. The idea that an Artificial Intelligence (AI) program is simply filtering out any prospective candidates by reducing their search in order to neglect any sign of female association sounds appalling to me. In other words, when human recruiters fail to remain equal and are influenced by their prejudice, all this AI system is doing is speeding up the processing in doing so. It seems to me that technology isn’t always improving, but rather increasing the rate at which faults are being made. Especially in today’s society it seems that the rights of gender equalization is becoming an increasing issue. Additionally, there is no doubt that the treatment of women in the workforce has immensely improved over the past decades. However, the frightening proposition that this article suggests is that technological programs are able to come to their own prejudice conclusions, such as that “men were better candidates for technical jobs”. It seems to me that the evolution of females striving for excellence in every field is taking a great step back when an AI system is able to come to such a quick conclusion on their own. This comes to beg the question whether or not the improvements in society are merely damaged by the power of technology.

  6. David S October 19, 2018 at 4:47 pm #

    While it may initially seem that replacing human recruiters with ones that are AIs may eliminate any bias that is present in a company’s recruiting process, this article highlights just how inescapable the issues that come with judgment and bias within a business truly are. Ideally, AIs may be seen as desirable replacements since it is strongly believed by a majority of people that technology lacks the capability of emotion. It is this factor that many believe is what differentiates humans from technology. However, emotion is not the only thing that gives the opportunity for bias to occur. Facts and information can too.
    It is important to know whether or not the AI was coming to the conclusion that men were better job candidates from the information that was pulling and what source it was exactly pulling this information from. It is also a possibility that the AI came to a conclusion that was influenced by gender bias because it was influenced by the actions of the human beings it was observing. These people may even be the ones who programmed the AI. If the AI picked up dialogue where some evidence of gender bias was detectable then it may have saved it and factored it into its processing. Alternatively, the AI could have found information sources within the company’s database that gave solid evidence that Amazon felt that men were more desirable hires than women. This example itself also serves as a great commentary on how just when society’s standards and understanding of gender equality and feminism appear to be improving new problems or cases arise that suggest otherwise. Honestly, this headline may not shock many readers. We live in a world where sexism still runs rampant in many aspects such as business. The fact that a computer program adopted a thought process that is so significantly human definitely suggests that the company needs to seriously look at reevaluating their hiring process and gender politics.

  7. Paul Lee October 19, 2018 at 6:47 pm #

    When I first read this article, I was a surprised to read that an Amazon AI chose men over woman when looking through a variety of applications to hire. I would never have thought an AI especially with about ten years’ worth of job application put into its AI system. I was even surprised the company behind the AI was Amazon, I thought they would do a better job behind the algorithms behind it. However, as I read more, I understood what the problem was with this AI from the start. It starts with Amazon’s fault because they should have known that the ten years’ worth of application was mostly men, so it would be very biased toward men. In my opinion, Amazon should have known what kind of applications it would be entering into its AI algometric system. However, what scares me about this AI, even though Amazon disbanded the team early last year, is that it learned and emulated human behavior. It was simply a machine that became biased just like an ordinary human being would. Humans can be very scary but an AI that is intelligent with human thoughts. It makes me very unsure for the future of AIs. However, it is strange that the Amazon programmers did not make this AI learn from its mistake. One would think if more men were accepted by the machine than woman, there would be a problem. They would have corrected it immediately, but It took a little time for them to realize. I have recently read an article on an AI called Deep Angel. It was created by an MIT team, it is an AI that basically erases an object of your choosing from your photo. It is a clean removal from the photo which is very complicated to do. This AI is still in the works, but it learns slowly from its mistakes and what not to do. Finally, I am so pleased to hear that the team behind this application AI was disbanded and recruiters did not even base their hiring off the machine. It was a good point to only use its recommendation and not to hire certain people or men from the AI. Maybe in the future, Amazon could potentially open this team to work on this AI and work out the problems. I would love to see how this AI would work with applications and how it hires people based of its information. I am also interested in the Goldman Sachs and Hilton’s AI hiring process, it would be nice to see how the AI was created.

  8. Tyler Peteraf October 19, 2018 at 8:47 pm #

    I find it extremely fascinating that an AI recruiter for Amazon was able to inherit the same prejudices that we see in the world today. As the article states, the AI simply learned and emulated human behavior, which made just as biased as the living people it replaced. It seems as if this world and some of the bigger companies might lean more towards AI for recruiting as opposed to human people. Seeing this and the prejudice that took place, these AI’s might not be able to be as reliable as some people would like to think they are. Society is going to keep on looking towards technology to improve the way we go about life. Normally the thinking is that technology is a definitive improvement from human performance, nut how would be able to look towards an AI that applies the same prejudice’s as humans. Plain and simple, if this happened for one AI, there is definitely a possibility that it could happen again with another AI. Technology should be a driving force that allows the world to constantly improve in the future, but that doesn’t mean we should be ignorant of the fact that technology can have some very real downsides. Technology will continue to be depended upon more and more, but we have to make sure that it actually helps us instead of hurting us. While technology does make things easier and more productive, you also lose the human touch that could avoid thing that technology can’t. In no way am I advocating the we lessen our use of technology in the world, but I am saying that we shouldn’t look at technology as the solution for every problem in the world. Companies like Amazon and other financial juggernauts in the world need to set a standard for the technology they use, as there are enormous consequences for making mistakes when you are dealing with billions of dollars.

  9. Brendan Cortez October 19, 2018 at 10:52 pm #

    The idea of utilizing artificial intelligence for the recruiting process is certainly interesting, and I was quite surprised when I heard that companies are beginning to implement this technology. From a business perspective, I understand how beneficial AI can be for cutting costs, saving time, and producing efficiency. However, when it comes to human resource responsibilities, such as the recruiting process, I do not think AI technology should be utilized. In fact, I think it is dangerous to use AI technology for business processes that require a great deal of human intuition. It does not surprise me that these AI recruiting systems are beginning to produce biased decisions, which favor men over women. What concerns me the most is the idea that the bias stems from the developers who interacted and taught the AI system. In Amazon’s circumstance, I understand that the issues were generated from the system “learning from and emulating human behavior.” Given that the technology industry is dominated heavily by males, I am concerned that there is a potential correlation between AI systems developing bias, and the implicit bias held within the technology engineers who interact with the system.

    For large corporations like Amazon, reading through thousands of applications can become very cumbersome and costly, so I recognize the utility in investing in a technology that helps reduce the burden. It is reassuring to see that Amazon is taking a proactive approach as they implement and test out this new technology. However, investing into an AI recruiting system might prove to be unsuccessful and ineffective. Because these systems are trained rather than programmed, the algorithms produced can be as good as “the quantity and quality of the training data to get [it] going,” so “its judgment is never perfect.” ( If the effectiveness of these AI systems depend heavily on human input and interactions, is it even possible to develop an AI technology that is bias-free? For business processes that require a great deal of human experience, such as recruiting, I believe there is more utility in having a strong human resource department who is responsible for recruiting. Although humans are not as fast as AI technology, they have individualized experiences and an intuition that cannot be replaced by technology.

  10. Julia Ausborn October 21, 2018 at 8:43 pm #

    AI used inaccurately can lead to major issues. As seen in this article, Amazon’s AI recruiter had a biased approach to women. This tells us that AI is just a computer and cannot compete with a human’s perspective, and the sense of finding the right candidate. When it comes to inviting someone for an interview and possibly working with this individual on a daily basis, I believe that a human’s touch is necessary.
    However, AI is not completely misleading. It can be a helpful tool for job seekers to improve their professional skills. Tests which help you identify your strengths, skills, and personal favoritism can help you to improve your resume, as well as help you seek the right job for you. Using AI properly can be beneficial for both, the employer and yourself.

  11. Melissa Joas October 26, 2018 at 5:43 pm #

    RE: Report: Amazon’s AI Recruiter Favored Men

    There is no better name for this technology than “Artificial Intelligence”. Intelligence is something that a person is born with. That person can gain knowledge throughout his or her life and appear to be very intelligent, but there is a limit to his or her comprehension of this knowledge. The same thing can be said for a computer, but a computer is only as intelligent as the software it is fed, and the software is only as smart as its developers. This is a classic case of human error and is a good example of why people should not put any more faith into AI than they would another human. In fact, the right pair of human eyes on the resumes that were rejected by Amazon’s recruiting engines would have picked up on the problem and pulled those resumes up for review. The right team of software developers would have understood more about statistics to come up with a sample that represented the population when selecting the resumes that the system was based on.

    The developers are not the only ones to blame for this situation. The recruiters that bought into this technology should also hold themselves accountable for its failure. Finding the right candidate to fill a position is a very expensive and time-consuming undertaking. According to the Society of Human Resources Management (SHRM), the average cost per hire is $4,129 and it takes an average of 42 days to fill an open position. In addition, hiring is most often done by human resources professionals who are trained not only in what to look for in a prospective employee, but also what laws must be followed in the process. They aren’t necessarily trained in the field in which they are hiring, but chances are they know more about the open position within their company than the software developers do about human resources. Removing the human aspect of reviewing the resumes turns “human resources” into “resources”. Nothing is better equipped to judge another human’s character than another human. There are no shortcuts to this process. The advent of online recruiting significantly deepened the pool of applicants for each position and the reach of each posting but has presented employers with the issue of filtering through many resumes from unqualified or uninvested persons. There is no denying that the task of sorting through these resumes is an enormous one, but hiring professionals are trained to find key items on a resume very quickly making the sorting process a bit less tedious than it would be for someone with no training in this field. “Be careful what you wish for. You just might get it!” It wasn’t very long ago that employers complained about not getting enough applicants to choose from.

    One of the initial problems is the set of resumes selected to base the system on. One can argue that there are industries that are male-dominated because they attract more men, or that men make up a greater percentage of the workforce. What this argument does account for is that the roles of men and women are changing rapidly. Perhaps more women are seeking employment in male-dominated fields, and more women are entering the workforce to even out the ratio. Artificial intelligence has the potential to shut them out and strengthen the disparities. If AI downgrades the rating of resumes that include the word “women” and selects common power verbs on qualified applicants’ resumes that are more often used by men as identified by Reuters, there will be more issues than the immediate one of machine-generated discrimination. Women might have difficulties finding jobs in male-dominated fields first due to this discrimination, and next because they do not have any experience working in their field since completion of education. The length of time a person spends outside of the profession that he or she studied for is a factor considered by employers when hiring. If AI doesn’t pick up on this, then the human that eventually sees a resume (if it makes it through AI’s first discriminatory filter) will make this judgment. In the article When a few simple rules are better than flashy AI, it is suggested that a rules-based system would be less likely to discriminate against any one group of people. I think that this opens the door for every kind of discrimination a person can think of. It is an easy way of determining the gender or race of a job applicant. This is not the solution.

    Even with the failure of this system, there is still plenty of interest in using AI for hiring. This takes the accountability out of the hands of AI developers and gives it to the people. If we are aware of the issues with discrimination that are not only possible but very likely, then we too are biased if we continue to seek out such technological services.

  12. Alexis Pateiro November 1, 2018 at 1:32 pm #

    It is crazy and shocking to think that an algorithmic (AI) recruiter that helps Amazon is bias against women. It is clear that it became bias from the people it replaced and that is the sad thing about it because those people were clearly prejudice. I think that Amazon had good intentions by making this form of recruitment and tried to use technology to make it easier. However, when an algorithm is favoring one gender over another it is not fair for women who apply for the job. The biggest impact on the recruitment was if women put anything with word “women’s” the AI would set it aside. I honestly had no idea that companies were already implementing algorithms in recruitment. If companies want to continue to use AI in such a humanized process they are going to have to carefully program the algorithm to make it fairer for everyone. In the case that this does become more popular it needs to be perfected before it is 100 percent relied on.

    I honestly do not agree with AI’s recruiting people for jobs. Picking out resumes and choosing who to interview needs to be decided by a person. There needs to be a line between what tasks and jobs AI’s should have versus humans. Humans should deal with recruitment because it is very important to pick someone from personal preference and with human perspective not just an algorithm. An AI cannot get that personalized opinion, it relies on certain criteria to recruit someone and the requirements of the job. I respect the idea of AI’s recruiting, but I also appreciate how Amazon does not rely on the algorithm. To use it as a reference is one thing but to rely on it is a very risky move. We also have to think about the issue that if companies start to use this, recruiters will lose their jobs. Therefore, this idea affects more than just the people applying for jobs for these companies.

  13. Nicole Faerber November 1, 2018 at 4:08 pm #

    As a junior in college, I am currently applying for jobs and internships. I am aware that often companies have a scanner for resumes, that look through them and pulls out certain ones depending on keywords used, and how many times. However, up until this point, I never thought that those machines the Automated Intelligence (AI), could be biased. The word bias in psychology is connected to people, as people judge others due to something like race, gender, religion etc. Now not only do people need to worry that the person interviewing them might be biased, but the machine that picks out their resume might be biased too.
    It is interesting to see that it was not the AI that came up with the bias but from the biases that were already there in other interview material it got fed. This shows that even though interviewers try to not to be biased they still might be. I do not think it is bad to be biased as a person is allowed to feel how they want. However, it should not be affecting someone in any way. The AI saw the previous resumes and candidates that were chosen by Amazon and from there, an algorithm was created in order to rate how well one would fit for the job. Since many of the old winning candidates were men, the AI would then penalize any resumes that had the word woman on it. Amazon has stated that they do not use.
    However, other big companies are starting to use more AI when it comes to finding new employees. I think that people are going to have to worry about the biases that the AI could develop. Before big companies completely use AI, they should always be looking for partners with the AI. Make sure that it is not penalizing people due to any biases. Have a real person look over the selected resumes to ensure that it is right. It is scary to think that now one not only needs to have technical knowledge but that an AI will be the thing determining if your resume gets consideration.

  14. Mark M November 7, 2018 at 11:24 pm #

    This report isn’t the first of it’s kind. Amazon is one of the most powerful companies in the world at the moment, and for them to be involved in this investigation about sexism in their artificial intelligence system, it shows you where we are currently with these sort of issues. For the longest time, women have been treated unfairly or not as equals in the workplace, and as much as we want to consider them to be equal in today’s age, we can’t quite support that claim at the moment. The wage gap, an issue that still stands today, has been a hot-button issue in the working world and until it gets resolved, it’ll remain one of many different issues that women incur on a daily basis.

    There are a whole bunch of other issues that come to mind with women in the workplace as well. Women are sexually harassed a ton, as well as receiving certain workplace “duties” that men feel like “fit more to their strengths.” And now, AI is being sexist when looking for top talent to recruit. The technology that is supposed to be completely unbiased stereotyped women’s resumes and provided men as top talents. This causes way more issues in the workplace, as women who are more qualified won’t even have the chance for jobs that they would be the perfect fit for.

    I believe there is a lot of work to be done in order to right the wrongs against women in the workplace. We need to bridge the gap and see employees based on merit, not on their gender. However we choose to go about it, I would like to know that sometime in the near future there would be an equal opportunity for my daughter to work her way up in the work place without any discrimination.

  15. Justin T November 8, 2018 at 10:04 am #

    I think that this perfectly shows why AI shouldn’t be part of something such as the recruiting process. Something even so simple as scanning a resume and it turns away females, we talk about how we want to work towards a more equal opportunity for men and women and this just isn’t helping the cause. As technology is advancing, the more that it’s going to be used in the workplace and so on. Technology is going to make simple tasks much easier for us to complete and most likely take some time off our hands to then work on other tasks. Whether we like it or not, AI will soon be a part of all our lives in some fashion if it’s already not. I once did a internship interview where the entire process was through AI and the recruiters told us when we made it to the final video that the AI would pick up a certain amount of things that they would want to hear in order to move on to the next stage of the process. Again this just doesn’t make sense to me when a lot of good candidates are most likely being left out, a simple phone call from a recruiter doesn’t hurt.

    Like I mentioned before we are trying to eliminate idea that men get hired more often or for better positions. I think this perfectly shows why AI can’t take a lot of tasks that humans can do, yes it maybe be quicker and cheaper but in this instance a lot of good candidates are being left out. Now the article does say that recruiters don’t use it fully and they only rely on it for some help, but I personally think that it should be completely eliminated. You can be eliminating a candidate who can bring so much worth to a firm just because a computer picks up on a few words. We want to eliminate gender discrimination, we want to have a better work culture, but it seems that using such tools like AI isn’t the answer.

  16. Halli Schwartz November 8, 2018 at 2:14 pm #

    I believe that the introduction of using artificial intelligence in recruitment is fascinating. Technology is becoming more and more advanced, and it is truly interesting and exciting. However, in the rapidly changing technological society we live in, I wonder how there is still a bias against women in the workplace, a seemingly archaic occurrence. As a woman, soon to be entering the large scale career field, this article’s title truly disheartened me. Will I, just because of my gender, really have less opportunities than men? When I further read the article, I was honestly quite upset. This article really upset me, not only because women were being hindered, like we have been for most of history, but also because women were not given a fair chance to begin with. If artificial intelligence recruiters are immediately picking men over women, based on gender, how does this give any woman a fair chance in Amazon’s company? As women are becoming more well versed in many industry, and finally being given a chance for higher education, we as a gender are gaining more and more skills. These skills truly do make us stand out from the rest, including men. However, due to the new recruiting system, a man, who may be much less qualified for a position, can get this position just because of his gender. A woman can have more experience, many internships, and a great degree. However, just because someone is a man, they can gain a job that they maybe might not be qualified for.
    Before I read this article, I was not very knowledgeable about AI. I, unknowingly, was upset with the actual program, rather than those who created it. The article stated, “This is a textbook example of algorithmic bias. By learning from and emulating human behavior, a machine ended up as prejudiced as the people it replaced” (Waddell 2). Not only are some men keeping the inherent prejudice against women, and showcasing it in their employee decision, but they are influencing newly created technology to do just the same. In the future, I really believe that we need to look at our own actions, and see how they are influencing others, whether the one influenced be a human or some sort of artificial intelligence. If men can influence artificial intelligence to be biased against women, imagine how they can influence other men. In a world full of prejudice, I truly hope that women do not receive the extreme discrimination they did so many years ago. I feel like if everyone changes their actions, this discrimination may be changed. However, if not, I think things for women seeking jobs may stay stagnant, or the job hunt may even become more difficult for women. If the aforementioned difficulties become the case, this truly upsets me, and makes me think that we as a society are moving backwards rather than forward.

  17. John Heintz November 9, 2018 at 3:30 pm #

    For every ten successful technological stories there’s one failure that seems to go unnoticed. This is such an interesting thing to come across especially considering how highly people view artificial intelligence. For those who oppose its thorough implementation, here is a perfect example as to why. People, unfortunately have unconscious bias and for the AI that Amazon tested developed it on its own as well goes to show to certain jobs can not be replaced.

    This “algorithmic bias” is more than likely prominent in some company recruiting processes and should be looked at very meticulously. It is quite unfortunate that this situation ended up the way that eventually lead to its discontinuation but rather it be phased out after being investigated and attempted as opposed to not at all or worse, fully implemented anyway. Hopefully in the near future a more refined, unbiased alternative can be brought to light.

  18. Daniel Campbell November 9, 2018 at 4:22 pm #

    This article summarizes that Amazon was testing a beta version of a recruiting AI. The AI was biased against women for the fact that since 2014, the AI was using and configuring data form resumes of the past 10 years. Most of the resumes the bot relied on was that on male employees. This is where the problem started for that the data that was being calculated for the AI was not clearly representing the market in which the AL was intended to serve. This example of a algorithmic bias is perfect to use for arguments for equal employment opportunities. Amazon has conflicting reports about the actual use of this system. Some Amazon representatives concluded that this AI was never used to evaluate candidates. The use of AI in recruiting purposes is very controversial because it leaves room for algorithmic biases such as this one. The argument for the use of these types of systems is that AI would naturally be unbiased and solely used for raw data and factual information, but this can only be relied on in that context if the data is of representation of all perspectives. It’s not surprising that human recruitment is full of biases as well because that is something we see all over the news in today age. The only way we can rely on systems such as this is if we have a reliable sources of data from multiple different perspective and situations. In the topic of women in the workplace, we must have more female resumes up for data processing.

  19. Schron B November 9, 2018 at 7:13 pm #

    Here’s the thing about A.I that is programmed by humans to not follow human faults, its still going to have those faults. Since it was programmed to pick the ‘best’ applicants based on who the company prioritized the past decade, the fact that the A.I favored men is not all that surprising. It was known that many areas in the business world had a bias towards males and the tech world is no different. It’s a real shame that at the moment even an A.I was not able to fix that injustice but hopefully Amazon takes the experiment as a learning experience.

    Hopefully Amazon continues their work on the A.I and does not see it as a total failure. The efficiency in which way the A.I works is actually impressive, the main changes needed would have to be not only the information put into the program but also what it prioritizes. It is certainly going to be a challenge for Amazon to figure out but hopefully they stick with this project because it could prove to be revolutionary in the job market if it succeeds in its intention.

  20. Shaunak Rajurkar December 6, 2018 at 2:18 am #

    Over the last 6 months, Amazon has shifted places among the top 3 largest companies in the world. If a company with such a sizable global presence has sexist implicit bias built into its hiring algorithm, a lot is at stake for the future of gender equality. Tech culture is already notorious for hiring technically skilled workers who happen to be socially incompetent. Sexism is hidden in the community, yet is still incredibly prevalent in the industry. For an industry that is supposed to be incredibly progressive, it seems we are taking steps backward. Amazon needs to acknowledge its gargantuan footsteps and use its platform to catalyze social change.

    Amazon is a powerhouse in multiple industries, but it is the far out leader in global e-commerce and cloud computing services, in both business to business revenue as well as for individuals. The reason Amazon has quickly become such a target for advertisers is simple: Amazon is already a retailer, and thus there is little need for a middleman like Facebook or Google in the advertising space to sell physical products. According to the article, this going to allow Amazon’s ad revenue to grow from 2.5 billion to nearly 20 billion by 2020. Amazon already has the insight on consumers’ spending and shopping habits through their analysis of giant data mines.

    As a parent company, Amazon has branches and subsidiaries in nearly every industry aside from automotives and (arguably) social media. Amazon is a pioneer of Software as a Service (SaaS) through its multiple online subscription products, including Amazon Web Services (AWS) and Elastic Cloud 2. It now has a presence in groceries and fresh food delivery through its acquisition of Whole Foods. With a diversified line of Kindles, Amazon dominates the e-reader space and presents the Kindle as something more resemblant of a tablet. It now threatens to disrupt both the healthcare and health insurance industries as suggested by recent patents that allow Amazon to act as an online pharmacy. Additionally, investors speculate that over the next two years, Amazon could acquire both Lyft and Snapchat. This company is the third largest in the world, and is best described with one word: omnipresent. If Amazon continues to dominate every industry it enters, this omnipresence will become omnipotence.

    Congress needs to quickly draft legislation to regulate Amazon before it becomes a hyper-monopoly. Just this morning, it was revealed that Amazon will be placing its Crystal City, Virginia, and Queens, New York. Politicians in each prospective state were offering Amazon astounding incentives – New Jersey, for example, offered Amazon a 7 billion dollar tax break if it were to have placed its new headquarters in Newark. By splitting the headquarters in two locations, Amazon was able to capitalize on two separate incentive packages, essentially doubling their benefit. After staying in the shadows of Facebook, Apple, and Google over the last decade, Amazon has been able to effectively hide from regulators.

    Amazon collects vast amounts of personal user data through its website as well as through third party ads on other websites. They have offered innumerable advertising products for their own sales. After multiple data breaches over the last few years, consumers must ask themselves whether or not it is worth sacrificing privacy for convenience; this proves even more true in the hands of a company that is well on its way to becoming a hyper-monopoly if left unregulated.

  21. Diamond Vasquez April 25, 2019 at 12:06 pm #

    Reading this article made me think of what we discussed in our Legal Foundations of Business class. In “Report: Amazon’s AI Recruiter Favored Men,” published by Kaveh Waddell in Axios, it elaborates on the algorithmic biases of AI systems Amazon uses. This algorithmic recruiter’s purpose was “to help Amazon find top talent,” according to this article, but it became biased against women. Now, why is that? Well, as described by Waddell, “Amazon’s experiment, which dates back to 2014, was trained on 10 years of job application, most of which came from men..” This algorithmic recruiter believed that men were more capable or qualified for technical jobs than women were. In 2015, Amazon realized how this system was penalizing resumes that included anything with the word “women’s.” This is a prime example of algorithmic bias because this AI system is well-trained in recruiting more men than women. Whatever they pick up, that is what they are accustomed to. Many large companies, such as Goldman Sachs and Hilton, are already using these AI systems in their recruiting process.

    From reading this, I believe that using the AI systems, though it may be an easier way to look through people’s resumes, is not a such a good choice because of how it is trained now. The AI systems are trained in a way in which they get accustomed to a particular thing, resulting in algorithmic bias. They are trained in a way that they, eventually, imitate humans and their behaviors. In this case, the AI systems are accustomed to hiring men over women because of their belief that men are more qualified for technical jobs. I think that the AI systems wither have to be trained more properly than before or they are just going to have to terminated because, as explained in the article, “By learning and emulating human behavior, a machine ended up as prejudiced as the people it replaced.” Hopefully, in the near future, there would be a way in which we can figure out how we can properly train AI technology to the point where it is not being considered “biased” or “prejudice.” Technology has advanced so much that it has come to this point.

  22. Andrew Kenny April 25, 2019 at 6:45 pm #

    Using an AI recruiter may be quicker and cheaper, but it definitely is not perfect. Like Google with facial recognition, Amazon had white males test the AI’s algorithms, creating a bias. As if there weren’t already enough dispute in the workplace over women’s opportunity, this made it even worse. With the AIs simply disposing of resumes that included details about women, this raised a ton of red flags for how fair the hiring process is. The system already decided that men were better for the job just because the system believed they were better suited for technical jobs. It makes sense, considering men from Amazon wrote the algorithms. Amazon is always quick to learn form their mistakes, and they tried to intervene quickly when they realized what was happening. Amazon did admit to the fact that it used the AI’s “opinions” but did not fully rely on them. Nonetheless, they contradicted that report not long after that they never evaluated candidates with AI. It seemed like a cover up for a failed experiment. We all know how innovative Amazon is and how many jobs they have already replaced with robots. That leads me to believe that they did use AI to try and evaluate candidates to cut costs and become more efficient.
    The future of interviewing and job searching will most definitely include AI. Goldman Sachs records interviewees and matches their tendencies to their highest performers. AI can detect small gestures and match that to what deficiencies that person may have. There needs to be a balance between what is invasive and unfair and what is actually effective. I do believe that AI would eventually be an integral part in narrowing down great candidates for a given job. The technology is pretty much there, and technically an AI cannot have bias on its own. Only a human can insert an algorithm to implement prejudice. If algorithms can be monitored or regulated before being put into effect, these problems will fall by the wayside.

  23. Daibelis Acevedo April 25, 2019 at 11:51 pm #

    Although not many people are very informed about this topic, fortunately I am informed because of Professor Shannon. The Amazon AI facial recognition which they claim that “recruiters used its recommendations but didn’t rely on them entirely” (Waddell) is a mess. The algorithmic biases that this new technology has been making is not far off from what the human beings on the job will be making. It is a very controversial issue because although some may say that this new technology is an efficient way to help make hiring easier, it does cause errors like these. The fact that women were being completely banned from the chance of even getting hired is a big outbreak for this company and can cause many consumers to steer away from them. As it says in the article this experiment dates back to 2014 and majority of the example applications they had was from men. How did the company expect the program to work out? It’s crazy to think that the AI was supposed to recognize both female and male applications and accept it if it was never trained to do so. It took them an entire year to realize that women applications were being denied, and anything that included the word women was tossed aside as well. Overall, I think this was a huge mistake on their part, and I am referring to the moment experiments began.
    The system that Amazon was using is just simply not fair. It’s crazy to think that this system was purposely taking women out of the equation and only making way for the men who applied. The article does state that the company let go of employees who were in charge of the system, but I don’t think it is only their fault. This goes deeper than just the people who were in charge of the AI. It’s a problem of the whole company, the entire company said yes, the entire company was hiring, and the entire company is Amazon, not just the people in charge. Yes, I can see how the people in charge are to blame but it is an overall issue for everyone there because when training the AI, they basically only gave it information from a male’s resume. This isn’t an issue that can just be pointed at one person, but something that the company as a whole, should be responsible for.
    Of course, the dream is that AI can do the job perfectly, without bias, and in a fair way. This is a problem because the AI must be perfectly trained in a way that its arithmetic is not bias. Writer Kaveh Waddell says, “companies are still hoping that properly trained AI can not only avoid algorithmic bias but also correct human recruiters’ prejudices.” If companies manage to do this and create an AI that can work out by recruiting employees based off of their resumes and work experience, it would be great. I am not against the system, but I am against the way they train it and the algorithm that they are training. It needs to be done in a fair way.

  24. David Torres April 26, 2019 at 12:38 pm #

    The incorporation of artificial intelligence into the workforce will cause drastic changes to many companies. For example, Amazon, with the use of artificial intelligence, gives them a great advantage maximizing profits and creating a steadier and more efficient line of work. This is because there is no need to pay the artificial intelligence for their work, and there is no need to give them a lunch break either. However, Artificial intelligence is not only being used to do the tedious work of a company, but also being used to better the hiring process in a company. In one class of our Legal studies class, I was informed that arguably the most prestigious institution on Wall Street, Goldman Sachs, uses Artificial intelligence in their hiring process without informing the person being hired. The Artificial Intelligence system used in the interview process tracks their facial expressions on a micro and macro level. However, the Artificial intelligence is trained to only recognize the facial features of white males, and not people of other ethnicities such as Africans and Hispanics. Africans were being identified as monkeys and apes, which is a bit insulting. As a result, any person being hired who does not appear similar in facial features and expressions to the top candidates of Goldman Sachs, will have no chance of being hired by the institution. I see how this can be used to identify the best suit people for the job by identifying similarities among them, but I honestly believe that this type of procedure is limiting the institution from recruiting the absolute top talent they could possibly find. On top of the use of artificial intelligence in the interview process, it does seem a bit violation of a person’s privacy to do so. However, may have secretively have informed the interviewee about this but in extremely fine print on a written document that they have not read clearly at all. In a statement said by Gary Vaynerchuk, Ive learned that in reality, most employers are not that good at hiring people. This is an ok flaw to have, but the most important thing, is to be very good at firing people.

Leave a Reply