Is using Artificial Intelligence Discriminatory?

The Double-Edged Sword of AI in Employment

The integration of artificial intelligence into the workplace promises to make hiring and promotion more efficient and objective. By using data-driven algorithms, companies hope to reduce human bias in employment decisions. However, these same technologies can create new, systemic barriers whilst revealing implicit biases amongst those making the same decisions. Workers who suffer with disabilities are another example of a group that has been recently vocal about the usage of artificial intelligence in the workplace. When trained on data reflecting a non-disabled norm, AI can misinterpret or penalize characteristics associated with a disability, creating a new frontier for disability discrimination.

While the technology is new, the legal principles of equal opportunity and the right to reasonable accommodations remain essential protections for workers.

What are Reasonable Accommodations & Undue Hardship?

A strong legal framework protects workers with disabilities from discrimination. All employment practices, whether led by humans or artificial intelligence, must comply with these laws. The Americans with Disabilities Act, or ADA, forbids disability discrimination in all aspects for companies with 15 or more employees. A key part of the ADA is the requirement for employers to provide reasonable accommodations for an employee’s known disability. Failure to do so would then cause what’s known as an undue hardship. New York State and New York City Human Rights Laws offer even broader protections, applying to smaller employers and using a more expansive definition of what a disability entails. These laws make it clear that employers must actively work to have their disabled employees succeed.

What are Reasonable Accommodations?

A reasonable accommodation is a change to a job or work environment that allows a person with a disability to perform their job well, and enjoy equal opportunities. Examples of reasonable accommodations include modified work schedules, providing accessible technology (like a TTY phone for a hearing challenged employee), or reassigning non-essential tasks. When an employee requests an accommodation, the employer must engage in a good-faith “interactive process” to find a solution.

What is Undue Hardship?

An employer can only deny an accommodation if it creates an undue hardship, meaning a “significant difficulty or expense” relative to the employer’s size and resources. This is a very high standard to meet, and an employer cannot claim hardship based on the fears or prejudices of others.

Is Using Artificial Intelligence Discriminatory? 

While not inherently malicious, the design and application of AI in hiring employees can create significant risks of disability discrimination, amongst other facets. This happens primarily through biased data and flawed proxies.

How Algorithms Learn to Discriminate

An AI system is only as objective as the data it’s trained on. If an artificial intelligence learns from a company’s past hiring data that reflects a workforce with few people with disabilities, it will learn to replicate that bias, favoring candidates who fit the existing non-disabled profile. This discrimination often operates through seemingly neutral proxies, or agents, that are highly correlated with disability, as described below:

Resume Gaps: Artificial intelligence might penalize an applicant for an employment gap that was due to legally protected medical leave. Even with valid reasoning and evidence of leave, the wider the gap, the more likely the technology could flag the individual.

Keywords: A resume-screening tool may look for specific experiences, like team sports or leadership experiences, and screen out a qualified applicant who developed equivalent skills through more accessible means. 

Video and Voice Analysis: A growing number of AI tools analyze a candidate’s facial expressions, eye contact, and tone of voice. This can systematically disadvantage applicants with autism, physical disabilities or impairments affecting motor control, or speech patterns impacted by a hearing impairment. The AI may misinterpret these differences as a lack of confidence or clarity and therefore ability, regardless of the applicant’s qualifications. 

Gamified Assessments: Some employers use game-like tests that require quick reaction times or precise motor skills, creating barriers for applicants with motor or visual impairments.

A Case in Point: AI Bias Against a Hearing Challenged Applicant

In March of this year, financial software company Intuit was in hot water over an accusation that they used hiring assessment software in their hiring process, and this software of course was intertwined with artificial intelligence. The ACLU of Colorado asserted to the Colorado Civil Rights Division that this software discriminated against deaf and non-white individuals at a higher rate.

The complaint claimed that a deaf and Indigenous customer service employee was required to use the platform when applying for a promotion. The employee requested human-generated captions as her reasonable accommodation so that she could access interview instructions and questions easier. It’s alleged that Intuit denied her accommodation request and then rejected her promotion request, due to her communication style. ” was denied a promotion after being assessed by HireVue’s video interview platform. She requested human-generated captions as a reasonable accommodation to understand the interview instructions, but her request was denied.

The ACLU of Colorado alleged that Intuit and HireVue’s actions have violated the Americans with Disabilities Act (ADA), Title VII of the Civil Rights Act of 1964 and the Colorado Anti-Discrimination Act. A spokesperson for Intuit told HR Dive in an email that, “The allegations in the complaint are entirely without merit. We provide reasonable accommodations to all candidates.”

The Regulatory Response and the “Black Box” Problem

The U.S. Equal Employment Opportunity Commission has affirmed that the ADA applies fully to artificial intelligence tools, holding employers responsible for any discriminatory outcomes. 

In New York City, Local Law 144 now requires employers using Automated Employment Decision Tools (AEDTs) to conduct annual bias audits and notify candidates that an AI tool is being used. However, these audits currently only test for race and gender bias, not disability bias, and the law only requires disclosure, not that the employer should stop using a biased tool. 

One of the biggest challenges in fighting this discrimination is the “black box” nature of many artificial intelligence systems. Their decision-making processes are so complex, that sometimes even their creators cannot fully explain them. This makes it incredibly difficult for a worker to prove that a rejection was based on bias rather than qualifications, which is a major hurdle when looking to file a disability discrimination claim.

A Playbook for New York Workers 

If you are navigating a job market that uses AI, it is vital to know your rights and how to assert them. 

Recognizing the Red Flags 

Be wary of application processes such as the following: 

• Use unusual, game-like tests that seem unrelated to the job. 

• Involve one-way video interviews that analyze your facial expressions or tone of voice. 

• Provide vague, nonsensical, or computer-generated feedback for a rejection. 

• Lack any clear way to contact a human for questions or accommodation requests.

How to Protect Your Rights 

Your most powerful tool is your right to a reasonable accommodation. If you encounter an automated system that you believe will not fairly assess you due to a disability, you have the right to request an alternative. For example, you can state in writing: 

“The video interview component of this application may not accurately assess my qualifications due to a speech impediment related to my disability. As a reasonable accommodation, I request an alternative assessment, such as an interview with a human hiring manager.” 

Under NYC Local Law 144, employers must provide instructions on how to request such an alternative. An employer’s response to your request—or their failure to respond—is documented evidence that can be crucial if you decide to pursue legal action.

How to File a Disability Discrimination Claim 

If you believe an AI tool has discriminated against you, you must act quickly. Below are some easy steps to keep in mind when filing a claim:

Document Everything: Keep copies of the job posting, your application, all communications, and any feedback you received.

Know the Deadlines: There are strict time limits for filing a claim. In New York, you can file with the federal EEOC (within 300 days) or the New York State Division of Human Rights (up to three years for recent incidents).

Seek Legal Counsel: Navigating discrimination lawsuits involving complex technology is difficult. Consulting with experienced employment lawyers is the most important step to protect your rights. 

Demanding Accountability in the Automated Workplace 

Artificial intelligence poses a significant threat to the rights of workers with disabilities in the workplace. While promising objectivity, these tools often perpetuate bias, creating invisible barriers for qualified candidates. 

However, the law is on your side. Protections under the ADA and New York law, including the right to a reasonable accommodation, remain powerful tools. By understanding your rights and recognizing the signs of algorithmic bias, you can challenge these unfair systems. Working with a New York disabilities discrimination lawyer can give you the chance at the justice you deserve.

Call a New York Workplace Discrimination Lawyer Today

We at Filippatos PLLC stand in proud solidarity with the working class. We believe that all people deserve the right to exist freely, and deserve to work in a regulated environment. If you are experiencing discrimination at work for technological reasons, please give us a call at 888-9-JOBLAW for a free consultation. We will do our utmost to help secure you the justice you deserve.