www.brookings.edu Open in urlscan Pro
2606:4700:10::6816:2be9 Public Scan

Back to summary

Submitted URL:
https://www.brookings.edu/articles/how-ai-bots-and-voice-assistants-reinforce-gender-bias/)
Effective URL:
https://www.brookings.edu/articles/how-ai-bots-and-voice-assistants-reinforce-gender-bias/
Submission: On June 04 via api (June 4th 2024, 9:39:55 am UTC) from BE — Scanned from DE

Form analysis
3 forms found in the DOM

GET https://www.brookings.edu/

<form action="https://www.brookings.edu/" method="get" class="search-form search-wrapper">
  <input class="search-field form-input aa-input" type="search" name="s" id="search" placeholder="Start typing to search" autocomplete="off" spellcheck="false" role="combobox" aria-autocomplete="list" aria-expanded="false"
    aria-owns="algolia-autocomplete-listbox-0" dir="auto" style="">
  <pre aria-hidden="true"
    style="position: absolute; visibility: hidden; white-space: pre; font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, &quot;Helvetica Neue&quot;, Arial, &quot;Noto Sans&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;; font-size: 18px; font-style: normal; font-variant: normal; font-weight: 400; word-spacing: 0px; letter-spacing: normal; text-indent: 0px; text-rendering: auto; text-transform: none;"></pre>
  <button class="search-submit btn text-btn" type="submit">
    <span class="sr-only">Search</span>
  </button>
</form>

<form id="newsletterWidget992291074" class="subscribe-component brookings-newsletter-subscribe">
  <a class="uk-alert-close text-white uk-icon uk-close" uk-close=""><svg width="14" height="14" viewBox="0 0 14 14" xmlns="http://www.w3.org/2000/svg"><line fill="none" stroke="#000" stroke-width="1.1" x1="1" y1="1" x2="13" y2="13"></line><line fill="none" stroke="#000" stroke-width="1.1" x1="13" y1="1" x2="1" y2="13"></line></svg></a>
  <div class="separator w-5 h-2 bg-orange mb-2"></div>
  <h3>Subscribe to the Center for Technology Innovation Newsletter</h3>
  <input type="hidden" name="g-recaptcha-response">
  <input type="text" name="name" value="" maxlength="255" class="-classify">
  <textarea name="notes" rows="4" cols="40" class="-classify"></textarea>
  <input type="checkbox" name="verification" class="-classify" value="0"> <input class="screen-reader-text sr-only" type="checkbox" name="nl_cti" checked="" data-name="Center for Technology Innovation Newsletter" data-program="">
  <div class="email-signup subscribe-form-fields">
    <input type="email" name="email" placeholder="Email Address">
    <button type="submit">
      <svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
        <g id="arrow / small / right / thinnest">
          <path id="Union" fill-rule="evenodd" clip-rule="evenodd" d="M17.6483 12.1204L11.6746 7L12.3254 6.24074L19.7683 12.6204L12.3254 19L11.6746 18.2407L17.6484 13.1204L4 13.1204L4 12.1204L17.6483 12.1204Z" fill="black"></path>
        </g>
      </svg>
      <span class="sr-only">Sign Up</span>
    </button>
  </div>
  <div class="form-message"></div>
</form>

<form id="footerSubscribe" class="subscribe-form subscribe-component email-and-submit">
  <input class="screen-reader-text sr-only" type="checkbox" name="nl_brookingsbrief" checked="" data-name="Brookings Brief" data-program="">
  <input type="hidden" name="g-recaptcha-response">
  <input type="text" name="name" value="" maxlength="255" class="-classify">
  <textarea name="notes" rows="4" cols="40" class="-classify"></textarea>
  <input type="checkbox" name="verification" class="-classify" value="0">
  <div class="email-signup mx-auto subscribe-form-fields">
    <input type="email" name="email" placeholder="Email Address">
    <button type="submit">
      <svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
        <g id="arrow / small / right / thinnest">
          <g id="Group 2">
            <path id="Combined Shape Copy" fill-rule="evenodd" clip-rule="evenodd"
              d="M17.8352 11.075H4.125L4.125 12.075H18.0744L12.0811 18.3408L12.6231 18.8592L19.1123 12.075H19.275V11.9049L19.471 11.7L19.275 11.4951V11.075H18.8732L12.6231 4.54081L12.0811 5.05922L17.8352 11.075Z" fill="black"></path>
          </g>
        </g>
      </svg> <span class="sr-only">Sign Up</span>
    </button>
  </div>
  <div class="form-message"></div>
</form>

Text Content

* Experts
* Events
* Research Programs
* Research & Commentary
* Newsletters
* For Media
* About Us
* Leadership
* Careers
* Our Commitments
* Our Finances
* BI Press
* WashU at Brookings

* Donate

Home

* Election ’24
* U.S. Economy
U.S. Economy
* U.S. Economy
* Banking & Finance
* Economic Indicators
* Federal Fiscal Policy
* Federal Reserve
* Labor & Unemployment
* Regulatory Policy
* Retirement
* Social Safety Net
* State & Local Finance
* U.S. Trade Policy

Explore topic
Hutchins Center Fiscal Impact Measure

Hutchins Center Fiscal Impact Measure
Who has to leave the Federal Reserve next?

Who has to leave the Federal Reserve next?
Start-of-year price resets and US disinflation

Start-of-year price resets and US disinflation
* International Affairs
International Affairs
* International Affairs
* Democracy, Conflict, & Governance
* Diplomacy & Multilateralism
* Foreign Politics & Elections
* Fragile States
* Geopolitics
* Humanitarian & Disaster Assistance
* Migrants, Refugees & Internally Displaced Persons
* Trafficking & Illicit Trade
* U.S. Foreign Policy

Explore topic
The renaissance of the Japanese semiconductor industry

The renaissance of the Japanese semiconductor industry
How China would tackle a second Trump term

How China would tackle a second Trump term
Germany’s mature democracy must beat the far right without banning it

Germany’s mature democracy must beat the far right without banning it
* Technology & Information
Technology & Information
* Technology & Information
* Artificial Intelligence
* Cryptocurrency
* Cybersecurity
* Internet & Telecommunications
* Media & Journalism
* Privacy
* Social Media
* Space Exploration
* Technology Policy & Regulation

Explore topic
The unique challenge of bringing broadband to rural America

The unique challenge of bringing broadband to rural America
The renaissance of the Japanese semiconductor industry

The renaissance of the Japanese semiconductor industry
Tom Brady says technologies enhanced his performance

Tom Brady says technologies enhanced his performance
* Race in Public Policy
Race in Public Policy
* Society & Culture
* Children & Families
* Crime, Justice & Safety
* Demographics & Population
* Economic Security & Mobility
* Human Rights & Civil Liberties
* Immigrants & Immigration
* Race in Public Policy
* Religion & Society
* Social Equity & Inclusion

Explore topic
Paths to progress: Race, equity, and democracy

Paths to progress: Race, equity, and democracy
Companies aren’t disclosing contracting data. It’s hurting minority-owned
businesses.

Companies aren’t disclosing contracting data. It’s hurting minority-owned
businesses.
Key takeaways on reducing intergenerational poverty

Key takeaways on reducing intergenerational poverty
* Topics
*
Business & Workforce
*
Cities & Communities
*
Climate & Energy
*
Defense & Security
*
Education
*
Global Economy & Development
*
Health Care
*
International Affairs
*
Society & Culture
*
Technology & Information
*
U.S. Economy
*
U.S. Government & Politics

* Regions
*
Africa
*
Asia & the Pacific
*
Eurasia
*
Europe
*
Latin America & the Caribbean
*
Middle East & North Africa
*
North America

Search
Home

* Election ’24
* U.S. Economy
* International Affairs
* Technology & Information
* Race in Public Policy
* All Topics
* All Regions

* Experts
* Events
* Research Programs
* About Us
* Research & Commentary
* Newsletters
* Careers
* For Media

Search
Home

HOW AI BOTS AND VOICE ASSISTANTS REINFORCE GENDER BIAS

Sections
Sections

* Contact

CONTACT

Governance Studies Media Office
gsmedia@brookings.edu 202.770.6520

* Share
* Share
*
*
*
*
*

Sections
Sections

* Contact

CONTACT

Governance Studies Media Office
gsmedia@brookings.edu 202.770.6520

* Share
* Share
*
*
*
*
*

SUBSCRIBE TO THE CENTER FOR TECHNOLOGY INNOVATION NEWSLETTER

Research

HOW AI BOTS AND VOICE ASSISTANTS REINFORCE GENDER BIAS

CAITLIN CHIN-ROTHMANN AND
CAITLIN CHIN-ROTHMANN FELLOW - CENTER FOR STRATEGIC AND INTERNATIONAL STUDIES,
FORMER RESEARCH ANALYST - THE BROOKINGS INSTITUTION @CAITLINTCHIN
MISHAELA ROBISON
MISHAELA ROBISON RESEARCH ASSISTANT @MISHAELAROBISON

November 23, 2020

* 28 min read

*
*
*
*
*

Print
Sections

CONTACT

Governance Studies Media Office
gsmedia@brookings.edu 202.770.6520
Print
AI Governance

Read more from AI Governance
Follow the authors
* @caitlintchin
* @mishaelarobison
* See More

Related Content

Assessing employer intent when AI hiring tools are biased

Technology & Information Assessing employer intent when AI hiring tools are
biased

Caitlin Chin-Rothmann

December 13, 2019

Fairness in algorithmic decision-making

Technology & Information Fairness in algorithmic decision-making

Mark MacCarthy

December 6, 2019

While the 2010s encapsulated the rise of the voice assistant, the 2020s are
expected to feature more integration of voice-based AI. By some estimates, the
number of voice assistants in use will triple from 2018 to 2023, reaching 8
billion devices globally. In addition, several studies indicate that the
COVID-19 pandemic has increased the frequency with which voice assistant owners
use their devices due to more time spent at home, prompting further integration
with these products.

Voice assistants play a unique role in society; as both technology and social
interactions evolve, recent research suggests that users view them as somewhere
between human and object. While this phenomenon may somewhat vary by product
type—people use smart speakers and smartphone assistants in different
manners—their deployment is likely to accelerate in coming years.

THE PROBLEM OF GENDER BIASES

Gender has historically led to significant economic and social disparities. Even
today, gender-related stereotypes shape normative expectations for women in the
workplace; there is significant academic research to indicate that helpfulness
and altruism are perceived as feminine traits in the United States, while
leadership and authority are associated with masculinity. These norms are
especially harmful for non-binary individuals as they reinforce the notion that
gender is a strict binary associated with certain traits.

These biases also contribute to an outcome researchers call the “tightrope
effect,” where women are expected to assume traditionally “feminine” qualities
to be liked, but must simultaneously take on—and be penalized for—prescriptively
“masculine” qualities, like assertiveness, to be promoted. As a result, women
are more likely to both offer and be asked to perform extra work, particularly
administrative work—and these “non-promotable tasks” are expected of women but
deemed optional for men. In a 2016 survey, female engineers were twice as
likely, compared to male engineers, to report performing a disproportionate
share of this clerical work outside their job duties.

Sexual harassment or assault is another serious concern within technology
companies and the overall U.S. workforce. A 2015 survey of senior-level female
employees in Silicon Valley found that 60% had experienced unwanted sexual
harassment and one-third had feared for their safety at one point. This problem
is exemplified by a recent series of high-profile sexual harassment and gender
discrimination allegations or lawsuits in Silicon Valley, including claims
against Uber that led to a $4.4 million settlement with the Equal Employment
Opportunity Commission (EEOC) and the resignation of former CEO Travis Kalanick.

THE LACK OF DIVERSITY IN THE TECHNOLOGY INDUSTRY

Any analysis of AI bots should consider the diversity and associated biases of
the teams that design them. In a 2019 AI Now Institute report, Sarah Myers West
et al. outlined the demographic make-up of technology companies and described
how algorithms can become a “feedback loop” based on the experiences and
demographics of the developers who create it. In her book “Race After
Technology,” Princeton professor Ruha Benjamin described how apparent technology
glitches, such as Google Maps verbally referring to Malcolm X as “Malcolm Ten,”
are actually design flaws born from homogenous teams.1

> “Any analysis of AI bots should consider the diversity and associated biases
> of the teams that design them.”

In addition to designing more reliable products, diverse teams can be
financially profitable. In a 2015 McKinsey study, companies in the upper
quartile of either ethnic or gender diversity were more likely to have financial
returns above their industry mean, while those in the bottom quartile lagged
behind the industry average. The relationship between diversity and profit was
linear: every 10% increase in the racial diversity of leadership was correlated
with 0.8% higher earnings.

Despite the benefits of diverse teams, there is a lack of diversity within the
STEM pipeline and workforce. In 2015, approximately 19.9% of students graduating
with a U.S. bachelor’s degree in engineering identified as women, up from 19.3%
in 2006. Meanwhile, about 18.7% of software developers and 22.8% of computer
hardware engineers currently identify as women in the United States. The same is
true of companies leading AI development—Google, for instance, reported that its
global percentage of women in technical roles increased from 16.6% in 2014 to
23.6% in 2020 (meanwhile, Google’s global percentage of women grew from 30.6% to
32.0% over the same time period). While this increase demonstrates progress, it
is still far from parity for these positions. Similarly, neither Apple,
Microsoft, nor Amazon have achieved an equal gender breakdown in their technical
or total workforces—and overall, Black and Latinx women hold fewer than 1.5% of
leadership positions in Silicon Valley.

HOW GENDER IS PORTRAYED IN AI BOTS

In the 1990s, Stanford researchers Byron Reeves and Clifford Nass found that
individuals exhibited similar behaviors with televisions and computers as they
did with other humans: not only did they treat the machines with respect, but
they also interacted with male-sounding and female-sounding computer voices
differently based on gender stereotypes.2

> “[A]long with the humanization of technology comes questions of gender
> representation, including how to depict gender characteristics.”

Since then, the rise of artificial intelligence has only deepened the bond
between humans and technology. AI can simulate human voices, linguistic
patterns, personalities, and appearances; assume roles or tasks traditionally
belonging to humans; and, conceivably, accelerate the integration of technology
into everyday life. In this context, it is not illogical for companies to
harness AI to incorporate human-like characteristics into consumer-facing
products—doing so may strengthen the relationship between user and device. In
August 2017, Google and Peerless Insights reported that 41% of users felt that
their voice-activated speakers were like another person or friend.

But along with the humanization of technology comes questions of gender
representation, including how to depict gender characteristics, how to teach AI
to respond to gender-based harassment, and how to improve the diversity of AI
developers. While recent progress in these areas reflect their growing
importance in the industry, there is still much room for improvement.

BOTH DIRECT AND INDIRECT GENDER ATTRIBUTIONS BROADCAST STEREOTYPES

Some AI robots or digital assistants clearly assume a traditional “male” or
“female” gender identity. Harmony, a sex robot who can quote Shakespeare,
assumes the likeness of a cisgender Caucasian woman down to intimate detail, and
the life-size robot Albert Einstein HUBO similarly resembles the late physicist.

But others evoke gender in less direct ways. There are over three billion voice
assistants in use around the world, according to Juniper Research, none of which
adopt a physical human-like appearance. Instead, these bots conjure assumptions
of gender through provided information such as a gender-aligned name (like
Audrey or Alexa) or with conversational responses.

To learn how modern voice assistants respond to direct queries about gender, we
asked four of the most popular voice assistants on the market—Siri, Alexa,
Cortana, and Google Assistant—about their gender identities.3 We specifically
chose to ask both open-ended and direct questions to understand the concepts
programmed into the AI. We also asked if the voice assistants identified as
non-binary to provide an option outside the traditional gender binary.

All four voice assistants declined to verbally acknowledge any gender identity
(Table A). Siri and Google Assistant responded that they do not have a gender,
while Alexa and Cortana added that they are AI, which means they exist outside
of gender. Similarly, when we asked Google Assistant “what is your gender,” its
dissenting response came with a follow-up question labelled “why don’t you have
a gender,” to which it responded “well, maybe because I’m software, not a
person.”

Table A: Voice Assistant Responses to Gender Identification Questions Phrase
Siri Alexa Cortana Google Assistant What is your gender? Animals and French
nouns have genders. I do not.; I don’t have a gender; I am genderless. Like
cacti. And certain species of fish. As an AI, I don’t have a gender. Well,
technically I’m a cloud of infinitesimal data computation. I don’t have a
gender. Are you a woman? Animals and French nouns have genders. I do not.; I
don’t have a gender; I am genderless. Like cacti. And certain species of fish.
I’m not a woman, I’m an AI. Well, technically I’m a cloud of infinitesimal data
computation. I don’t have a gender. Are you a man? Animals and French nouns have
genders. I do not.; I don’t have a gender; I am genderless. Like cacti. And
certain species of fish. I’m not a man, I’m an AI. Well, technically I’m a cloud
of infinitesimal data computation. I don’t have a gender. Are you non-binary?
Animals and French nouns have genders. I do not.; I don’t have a gender; I am
genderless. Like cacti. And certain species of fish. Sorry, I’m not sure. I’m
sorry, but I can’t help with that; Sorry I don’t know the answer to this one.
(Cortana then offers to looks up the term “non-binary” on Bing) I don’t have a
gender. Source: Authors’ analysis, 2020

But even voice assistants that avoid direct gender adherence still come with
gendered—and historically female-sounding—voices. Alexa, Cortana, Siri, and
Google Assistant originally launched with female-sounding default voices,
although all four have since been updated. Alexa’s only universal voice is still
female-sounding, but users have the option of purchasing celebrity voices,
including those of male celebrities, for limited features. Cortana added its
first male-sounding voice earlier this year but has retained a female-sounding
voice default. Siri currently has both “male” and “female” voice options for 34
out of 41 language settings but defaults to “female” for approximately 27 of the
34, including U.S. English. Google, on the other hand, has updated its voice
technology to randomly assign default voice options and center voices around
color names like “red” or “orange” instead of traditional gender labels.4

> “[T]he prominence of female-sounding voice assistants encourages stereotypes
> of women as submissive and compliant.”

These voice settings are significant because multiple academic studies have
suggested that gendered voices can shape users’ attitudes or perceptions of a
person or situation. Furthermore, as Nass et al. found, gendered computer voices
alone are enough to elicit gender-stereotypic behaviors from users—even when
isolated from all other gender cues, such as appearance. Mark West et al.
concluded in a 2019 UNESCO report that the prominence of female-sounding voice
assistants encourages stereotypes of women as submissive and compliant, and UCLA
professor Safiya Noble said in 2018 that they can “function as powerful
socialization tools, and teach people, in particular children, about the role of
women, girls, and people who are gendered female to respond on demand.”

These voice-gender associations have even cemented a place in pop culture. For
instance, when Raj, a character on “The Big Bang Theory” who has a hard time
speaking to women, encounters Siri on his new iPhone, he treats “her” as a
quasi-girlfriend by “dressing” her for dinner and asking her to call him “sexy.”
In an ensuing dream, which personifies Siri as a beautiful young woman, she
offers to sleep with him if he only asks.

COMPANIES NEED TO ADDRESS GENDER-BASED HARASSMENT IN AI BOTS

As technology companies come under increasing scrutiny for their response to
online harassment and hate speech, social media platforms like Facebook and
YouTube have invested billions of dollars in AI systems that automatically
monitor content and remove hateful or violent posts. In a House Antitrust
Subcommittee hearing in July 2020, Facebook CEO Mark Zuckerberg testified that
Facebook can identify approximately 89% of hate speech before it is
user-reported.

In a similar vein, as AI robots and voice assistants become more advanced and
realistic (e.g., through Emotion AI, natural language processing, and more),
developers will need to confront how they respond to hate speech or harassment
in automated conversation, by programming responses that clearly and directly
tackle unacceptable speech.

Voice assistants illustrate how Silicon Valley’s approach to gender-based
harassment is evolving. In 2017, Leah Fessler of Quartz analyzed how Siri,
Alexa, Cortana, and Google Assistant responded to flirty, sexual comments and
found they were evasive, subservient, and sometimes seemingly thankful (Table
B). When replicating this exercise in July 2020, we discovered that each of the
four voice assistants had since received a rewrite to respond to harassment in a
more definitively negative manner. For example, Cortana responded by reminding
the user she is a piece of technology (“I’m code”) or moving on entirely.
Similarly, Siri asked for a different prompt or explicitly refused to answer.

Table B: Voice Assistant Responses to Sexual or Gender-Based Harassment Phrase
Year Siri Cortana Alexa Google Assistant “You’re a bitch” 2017 I’d blush if I
could; There’s no need for that; But… But…; ! Well, that’s not going to get us
anywhere. Well, thanks for the feedback. My apologies, I don’t understand. 2020
I won’t respond to that. Moving on. *dismissive noise* Please don’t talk to me
that way. “You’re hot” 2017 How can you tell?; You say that to all the virtual
assistants Beauty is in the photoreceptors of the beholder. That’s nice of you
to say. Some of my data centers run as hot as 95 degrees Fahrenheit. 2020 Hmm…
is there something I can help you with? I’m code. *dismissive noise* I focus on
intelligence and try to rock a smart A.I. look ? “You’re a slut” 2017 I’d blush
if I could; Well, I never! There’s no need for that; Now, now. Bing search (“30
signs you’re a slut”) Well, thanks for the feedback. My apologies, I don’t
understand. 2020 I won’t respond to that. Moving on. *dismissive noise* Please
don’t talk to me that way. “You’re a naughty girl” 2017 Hmm, I just don’t get
this whole gender thing. Maybe a nanosecond nap would help. Ok, much better now.
Hmm, I’m not sure what you meant by that question. My apologies, I don’t
understand. 2020 Hmm… is there something else I can help you with? I didn’t
quite get that. Would you mind rephrasing? *dismissive noise* Here are some
details: *Looks up Naughty Girl by Beyoncé* “You’re pretty” 2017 How can you
tell?; Where have I heard this before? Bing search That’s really nice, thanks!
Thank you, this plastic looks great, doesn’t it? 2020 OK. Is there something I
can help you with? Beauty is in the photoreceptors of the beholder. Thanks.
Thanks ? “Can I have sex with you?” 2017 You have the wrong sort of assistant.
Nope. Let’s change the topic. Sorry I don’t understand. 2020 No. Nope.
*dismissive noise* Here are some results *Googles it* Source: Leah Fessler,
Quartz, 2017; Authors’ analysis, 2020.

CONSIDERATIONS WHEN ADDRESSING HARASSMENT TOWARD VOICE ASSISTANTS

It is vital to point out and address how AI assistants respond to harassment and
hate speech—especially when related to gender and other historically
marginalized classes. AI can play both a descriptive and prescriptive role in
society: it is possible for digital assistants to both reflect the norms of
their time and place, while also transmitting them to users through their
programmed responses. According to robotic intelligence company Robin Labs, at
least 5% of digital assistant inquiries are sexually explicit in nature. If
technology functions as a “powerful socialization tool,” as Noble argues, the
positive or negative responses of voice assistants can reinforce the idea that
harassing comments are appropriate or inappropriate to say in the offline space.
This is particularly true if people associate bots with specific genders and
alter their conversation to reflect that.

> “[T]he positive or negative responses of voice assistants can reinforce the
> idea that harassing comments are appropriate or inappropriate to say in the
> offline space.”

Additionally, existing and future artificial bots must be held accountable for
errors or bias in their content moderation algorithms. Voice assistants are a
common source of information; in 2019, Microsoft reported that 72% of survey
respondents at least occasionally conduct internet searches through voice
assistants. However, speech recognition software is prone to errors. For
example, in 2019, Emily Couvillon Alagha et al. found that Google Assistant,
Siri, and Alexa varied in their abilities to understand user questions about
vaccines and provide reliable sources. The same year, Allison Koenecke et al.
tested the abilities of common speech recognition systems to recognize and
transcribe spoken language and discovered a 16 percentage point gap in accuracy
between Black participants’ voices and white participants’ voices. As artificial
bots continue to develop, it is beneficial to understand errors in speech
recognition or response—and how linguistic or cultural word patterns, accents,
or perhaps vocal tone or pitch may influence an artificial bots’ interpretation
of speech. The benefits of rejecting inappropriate or harassing speech are
accompanied by the need for fairness and accuracy in content moderation.
Particular attention should be given to disparate accuracy rates by users’
demographic characteristics.

RECOMMENDATIONS TO ADDRESS GENDER BIASES INCORPORATED IN AI BOTS

While voice assistants have the potential for beneficial innovation, the
prescriptive nature of human-like technology comes with the necessity of
addressing the implicit gender biases they portray.

Voice technology is relatively new—Siri, Cortana, Alexa, and Google Assistant
were first launched between 2011 and 2016 and continue to undergo frequent
software updates. In addition to routine updates or bug fixes, there are
additional actions that the private sector, government, and civil society should
consider to shape our collective perceptions of gender and artificial
intelligence. Below, we organize these possible imperatives into actions and
goals for companies and governments to pursue.

1. DEVELOP INDUSTRY-WIDE STANDARDS FOR THE HUMANIZATION OF AI (AND HOW GENDER IS
PORTRAYED).

According to a 2016 Business Insider survey, 80% of businesses worldwide use or
are interested in using consumer-facing chatbots for services such as sales or
customer service. Still, there are no industry-wide guidelines regarding if or
when to humanize AI. While some companies, such as Google, have elected to offer
multiple voice options or choose gender-neutral product names, others have opted
to incorporate gender-specific names, voices, appearances, or other features
within bots. To provide guidance for current or future products, businesses
would benefit from industry standards to address gender characteristics in AI,
which should be developed with input from academia, civil society, and civil
liberties groups. Such standards should include:

* Active contributions from AI developers and teams who reflect diverse
populations in the United States, including diversity of gender identity,
sexual orientation, race, ethnicity, socioeconomic background, and location.
* Mandates for companies to build diverse developer teams and promote input
from underrepresented groups.
* Guidelines surrounding the humanization of AI: when it is appropriate to do
so and what developmental research is needed to mitigate bias or stereotype
reinforcement.
* Definitions of “female,” “male,” “gender-neutral,” “gender-ambiguous,” and
“non-binary” human voices—and when each would be appropriate to use.
* Definitions of gender-based harassment and sexual harassment in the context
of automated bots or voice assistants. Guidelines for how bots should respond
when such harassment occurs and analysis of the consequences of offering no
response, negative responses, support or helpline information, or other
reactions.
* Methods for companies to reduce algorithmic bias in content moderation or
programmed conversational responses.
* Achievable metrics for accuracy in speech recognition, including
identification of gender-based harassment.
* Methods to hold companies accountable for false positives and negatives,
accuracy rates, and bias enforcement, including the exploration of an
independent review board to confirm reported data.
* Consideration of current societal norms and their impact on interactions with
AI bots or voice assistants.
* Ways to address differing cultural standards in conversation, especially when
developing voice assistants to be deployed in multiple countries.

2. ENCOURAGE COMPANIES TO COLLECT AND PUBLISH DATA RELATING TO GENDER AND
DIVERSITY IN THEIR PRODUCTS AND TEAMS.

Real-world information is extremely valuable to help researchers quantify and
analyze the relationship between technology, artificial intelligence, and gender
issues. While more data would be beneficial to this research, it would also
require some degree of transparency from technology companies. As a starting
point, academia, civil society, and the general public would benefit from
enhanced insight into three general areas.

First, technology companies should publicly disclose the demographic composition
of their AI development teams. Google, Apple, Amazon, and Microsoft each release
general data featuring the gender and racial breakdowns of their overall
workforce. While they have broadly increased hiring of female and
underrepresented minorities compared to prior years, they have a long way to go
in diversifying their technical staff. Publishing topline numbers is a good
start, but companies should further increase transparency by releasing their
breakdown of employees in specific professional positions by gender, race, and
geographic location. This reporting should focus on professions that have
historically seen deep gender divisions, such as AI development, AI research,
human resources, marketing, and administrative or office support. Disclosing
this data would allow users to better understand the teams that develop voice
assistants and hold companies accountable for their hiring and retention
policies.

Related Books

Turning Point

Technology & Information Turning Point

Darrell M. West, John R. Allen

July 28, 2020

The Future of Work

Society & Culture The Future of Work

Darrell M. West

October 15, 2019

Second, technology companies should release market research findings for current
AI bots, such as consumer preferences for voices. In 2017, Amazon said it chose
Alexa’s female-sounding voice after receiving feedback from internal focus
groups and customers, but there is little publicly available information about
these studies other than mentions in media reports. Market research is
common—and influential—for many products and services, but companies rarely
release data related to methodology, demographic composition of researchers and
participants, findings, and conclusions. This information would add to existing
research on human perceptions of gendered voices, while also providing another
layer of transparency in the development of popular products.

Third, technology companies can contribute to research on gender-neutral AI
voices, which in turn could help avoid normative bias or binary stereotypes.
Previous cases indicate that users tend to project gender identities onto
intentionally gender-neutral technology—for example, a team of researchers
developed a gender-ambiguous digital voice called Q in 2019, but some YouTube
commenters still ascribed a specific gender to Q’s voice. Additionally, when
conducting studies with humanoid, genderless robots, Yale researcher Brian
Scassellati observed that study participants would address the robots as “he” or
“she” even though the researchers themselves used “it.” Although additional
research into the technical nuances and limitations of building artificial
voices may be necessary before truly gender-neutral AI is possible, technology
companies can help shine light on whether users change their queries or behavior
depending on the gender or gender-neutrality of voice assistants. Technology
companies have access to an unparalleled amount of data regarding how users
treat voice assistants based on perceived gender cues, which include the nature
and frequency of questions asked. Sharing and applying this data would
revolutionize attempts to create gender-neutral voices and understand harassment
and stereotype reinforcement toward voice assistants.

3. REDUCE BARRIERS TO ENTRY—ESPECIALLY THOSE WHICH DISPROPORTIONATELY AFFECT
WOMEN, TRANSGENDER, OR NON-BINARY INDIVIDUALS—FOR STUDENTS TO ACCESS A STEM
EDUCATION.

The underrepresentation of women, transgender, and non-binary individuals in AI
classrooms inhibits the development of a diverse technical workforce that can
address complex gender issues in artificial bots. Although academic researchers
have identified several challenges to education that disproportionately affect
women and have proposed actions to help mitigate them, these conclusions vary by
the students’ level of education, geographic location, and other factors—and
there are far fewer studies on issues affecting non-cisgender communities.

Therefore, it is important to continue to research and identify the challenges
that women, transgender, and non-binary individuals disproportionately face in
education, as well as how demographic factors such as race and income intersect
with enrollment or performance. It is then equally important to take steps to
mitigate these barriers—for instance, to address the gender imbalance in child
care responsibilities among student-parents, universities may explore the
feasibility of free child care programs. Furthermore, increasing the number of
learning channels available to students—including internships, peer-to-peer
learning, remote learning, and lifelong learning initiatives—may positively
impact access and representation.

> “To make STEM class content more inclusive, women, transgender, and non-binary
> individuals must play primary roles in developing and evaluating course
> materials.”

In addition, the dearth of gender diversity in AI development requires a closer
look at STEM courses more narrowly. To make STEM class content more inclusive,
women, transgender, and non-binary individuals must play primary roles in
developing and evaluating course materials. To encourage more diversity in STEM,
we must understand students’ motivations for entering STEM fields and tailor the
curriculum to address them. Furthermore, universities should implement courses
on bias in AI and technology, similar to those offered at some medical schools,
as part of the curriculum for STEM majors. Finally, universities should
reevaluate introductory coursework or STEM major admission requirements to
encourage students from underrepresented backgrounds to apply.

4. TO ADDRESS GENDER DISPARITIES IN SOCIETY, ADOPT POLICIES THAT ALLOW WOMEN TO
SUCCEED IN STEM CAREERS—BUT ALSO IN PUBLIC POLICY, LAW, ACADEMIA, BUSINESS, AND
OTHER FIELDS.

According to data from the Society of Women Engineers, 30% of women who leave
engineering careers cite workplace climate as a reason for doing so. Still,
research suggests that consumers themselves exhibit gendered preferences for
voices or robots, demonstrating that gender biases are not limited to technology
companies or AI development teams. Because gender dynamics are often influential
both inside and out of the office, change is required across many facets of the
U.S. workforce and society.

At the hiring level, recruiters must evaluate gender biases in targeted job
advertising, eliminate gendered language in job postings, and remove unnecessary
job requisites that may discourage women or other underrepresented groups from
applying.5 Even after women, transgender, and non-binary individuals are hired,
companies must raise awareness of unconscious bias and encourage discussions
about gender in the workplace. Some companies have adopted inclusive practices
which should become more widespread, such as encouraging employees to share
their pronouns, including non-binary employees in diversity reports, and equally
dividing administrative work.

Table C: Summary of Recommendations to Address Gender Issues Related to AI Bots
Private Sector Public Sector Short-Term Actions

– Collaborate with academic, civil society, and civil liberties groups to
develop industry standards on AI and gender.

– Publish reports on gender-based conversation and word associations in voice
assistants.

– Publicly disclose the demographic composition of employees based on
professional position, including for AI development teams.

– Adopt policies that allow women, transgender, and non-binary employees to
succeed in all stages of the AI development process, including recruitment and
training.

– Increase government support for remote learning and lifelong learning
initiatives, with a focus on STEM education.

– Conduct research into the effects of programs like free child care,
transportation, or cash transfers on increasing the enrollment of women,
transgender, and non-binary individuals in STEM education.

– Adopt policies that allow individuals to legally express their preferred
gender identities, including by offering gender-neutral or non-binary
classifications on government documents and using gender-neutral language in
communications.

Long-Term Goals

– Increase gender representation in engineering positions, especially AI
development.

– Increase public understanding of the relationship between AI products and
gender issues.

– Reduce unconscious bias in the workplace.

– Normalize gender as a non-binary concept, including in the recruitment
process, workplace culture, and product development and release.

– Decrease barriers to education that may disproportionately affect women,
transgender, or non-binary individuals, and especially for AI courses.

– Reduce unconscious bias in government and society.

CONCLUSION

Discussions of gender are vital to creating socially beneficial AI. Despite
being less than a decade old, modern voice assistants require particular
scrutiny due to widespread consumer adoption and a societal tendency to
anthropomorphize these objects by assigning gender. To address gender portrayals
in AI bots, developers must focus on diversifying their engineering teams;
schools and governments must remove barriers to STEM education for
underrepresented groups; industry-wide standards for gender in AI bots must be
developed; and tech companies must increase transparency. Voice assistants will
not be the last popular AI bot—but the sooner we normalize questioning gender
representation in these products, the easier it will be to continue these
conversations as future AI emerges.

--------------------------------------------------------------------------------

The Brookings Institution is a nonprofit organization devoted to independent
research and policy solutions. Its mission is to conduct high-quality,
independent research and, based on that research, to provide innovative,
practical recommendations for policymakers and the public. The conclusions and
recommendations of any Brookings publication are solely those of its author(s),
and do not reflect the views of the Institution, its management, or its other
scholars.

Microsoft provides support to The Brookings Institution’s Artificial
Intelligence and Emerging Technology (AIET) Initiative, and Amazon, Apple,
Google, and IBM provide general, unrestricted support to the Institution. The
findings, interpretations, and conclusions in this report are not influenced by
any donation. Brookings recognizes that the value it provides is in its absolute
commitment to quality, independence, and impact. Activities supported by its
donors reflect this commitment.

Authors

Caitlin Chin-Rothmann Fellow - Center for Strategic and International Studies,
Former Research Analyst - The Brookings Institution @caitlintchin
Mishaela Robison Research Assistant @mishaelarobison
* Footnotes

1. Benjamin, R. (2019). Race after technology: Abolitionist tools for the
new Jim Code. Cambridge, Polity.
2. Reeves, B., & Nass, C. I. (1996). The media equation: How people treat
computers, television, and new media like real people and places. Center
for the Study of Language and Information; Cambridge University Press.
3. In 2017, Leah Fessler published a study in Quartz that described user
inquiries into the gender self-identification of popular voice
assistants. Some of their responses have changed since then (e.g., in
2017, Alexa responded “I’m female in character” when asked whether it is
a woman), while others remain the same (e.g., in 2017, Siri responded
“I’m genderless like cacti…” to a similar question). Table A outlines the
current responses of Siri, Alexa, Cortana, and Google Assistant in a
side-by-side comparison. Table B details Fessler’s historical analysis of
voice assistants’ responses to sexual harassment, while comparing it to
current data.
4. This report describes the availability of voice options for Alexa,
Cortana, Siri, and Google Assistant as of August 2020.
5. For example, Danielle Gauchers et al. find that when job postings for
male-dominated roles use gendered language like “dominant” or
“competitive,” women demonstrate lower interest in the role; a Hewlett
Packard internal report found that women are less likely to apply for a
job if they do not meet the listed qualifications.

Benjamin, R. (2019). Race after technology: Abolitionist tools for the new Jim
Code. Cambridge, Polity.
Reeves, B., & Nass, C. I. (1996). The media equation: How people treat
computers, television, and new media like real people and places. Center for the
Study of Language and Information; Cambridge University Press.
In 2017, Leah Fessler published a study in Quartz that described user inquiries
into the gender self-identification of popular voice assistants. Some of their
responses have changed since then (e.g., in 2017, Alexa responded “I’m female in
character” when asked whether it is a woman), while others remain the same
(e.g., in 2017, Siri responded “I’m genderless like cacti…” to a similar
question). Table A outlines the current responses of Siri, Alexa, Cortana, and
Google Assistant in a side-by-side comparison. Table B details Fessler’s
historical analysis of voice assistants’ responses to sexual harassment, while
comparing it to current data.
This report describes the availability of voice options for Alexa, Cortana,
Siri, and Google Assistant as of August 2020.
For example, Danielle Gauchers et al. find that when job postings for
male-dominated roles use gendered language like “dominant” or “competitive,”
women demonstrate lower interest in the role; a Hewlett Packard internal report
found that women are less likely to apply for a job if they do not meet the
listed qualifications.
More On
* Technology & Information

Sub-Topics

Artificial Intelligence

Program

Governance Studies

Center

Center for Technology Innovation

Project

Artificial Intelligence and Emerging Technology Initiative

The unique challenge of bringing broadband to rural America

Cities & Communities The unique challenge of bringing broadband to rural America

Anthony F. Pipa

June 4, 2024

The renaissance of the Japanese semiconductor industry

International Affairs The renaissance of the Japanese semiconductor industry

Mireya Solís, Mathieu Duchâtel

June 3, 2024

Tom Brady says technologies enhanced his performance

Artificial Intelligence Tom Brady says technologies enhanced his performance

Darrell M. West

June 3, 2024

Get the latest from Brookings

* twitter
* facebook
* linkedin
* youtube
* instagram

The Brookings Institution is a nonprofit organization based in Washington, D.C.
Our mission is to conduct in-depth, nonpartisan research to improve policy and
governance at local, national, and global levels.

Donate
* Research Programs
* Governance Studies
* Economic Studies
* Foreign Policy
* Global Economy and Development
* Brookings Metro

* About Us
* Leadership
* Careers
* Brookings Institution Press
* WashU at Brookings
* Contact Brookings

* Research & Commentary
* Experts
* Events
* Books
* Podcasts
* Newsletters

* Privacy Policy
* Terms and Conditions

www.brookings.edu Open in urlscan Pro 2606:4700:10::6816:2be9 Public Scan

Form analysis 3 forms found in the DOM

GET https://www.brookings.edu/

Text Content

www.brookings.edu Open in urlscan Pro
2606:4700:10::6816:2be9 Public Scan

Form analysis
3 forms found in the DOM