Audio Accessibility Feature for Deaf & Hard of Hearing via Google Maps

Ensuring equal access to audio-based contextual cues

OVERVIEW

The Audio Detection Assistant is a feature designed for the Google Maps platform to enhance the travel experience for Deaf and Hard of Hearing (DHH) individuals by providing contextual awareness of critical audio cues during transportation journeys.

This project was developed over the course of a four-month master’s program project as part of my HCI Foundations Design course at Georgia Tech.

Timeline

Aug - Dec 2024

Tools

Figma

Notion

MS One Drive

FigJam

Team

Timothy Chiu

Disha Sikaria

Rachit Bhayana

Natalie Jarrett

Skills

Accessibility Design

Interaction Design

User Research

WCAG 2.1

SCOPE

Individuals who are Deaf or Hard of Hearing (DHH) often encounter barriers to fully engaging with their surroundings.

Our mission is to promote inclusivity and equity by raising awareness of the role auditory information plays in shaping accessible and welcoming spaces for all.

UNDERSTANDING

The Deaf and Hard of Hearing community had no control nor voice in what was being delivered

Literature Review

We learned valuable insights into the challenges faced by Deaf and Hard of Hearing (DHH) individuals in public spaces, emphasizing the importance of visual cues, improved emergency notifications, and the potential of assistive technologies for enhancing navigation and accessibility.

Hearing from the DHH community

We conducted semi-structured interviews with diverse participants, including members of or those familiar with the DHH community, to identify real-world challenges in auditory accessibility. These interviews highlighted failure in voicing needs for DHH users as systems were developed.

EXPLORATION

To avoid narrowing our design concept too early, we brainstormed 100 ideas—both good and bad—which were then evaluated and refined into 10 strong concepts.

Image of 100 ideation concepts on sticky notes

"An enjoyable travel experience is one where I don't have to worry too much about where to go next. I find that time can be wasted when plans are not made ahead of the trip"

Image for 100 ideation concept sticky notes

Among the 10 design concepts, one focused on the public transit experience stood out

"An enjoyable travel experience is one where I don't have to worry too much about where to go next. I find that time can be wasted when plans are not made ahead of the trip"

After weighing the pros and cons of each design concept, we revisited the recurring theme of insufficient audio accessibility in public transit, which emerged from our semi-structured interviews with the DHH community.

(2) Participatory Design Sessions: Evolving our concept into designs via expert and user critiques

Recommendations focused on reducing text reliance, using visual symbols for alerts, and limiting cues to a small set of high-priority options.

Image of participant quotes from interviews

Designing an entirely new product was unnecessary, especially since most users already relied on Google Maps for their transportation journeys.

DESIGN DECISIONS

There were contextual nuances determining the optimal placement of our audio accessibility feature

A comprehensive audit of a transportation journey on Google Maps allowed us to identify key events where our feature could be implemented.

View past cues, whether due to their phone being away, the journey being inactive, or simply overlooking the notification.

Showcase of Interaction on iPhone Screen

This is particularly useful for announcements, which are often not visually indicated, with users citing the lack of visual information and signage as detriments to their journey.

Detection and transcription of alerts and cues, such as approaching trains, fire alarms, or announcements.

The ability to visually perceive audio cues is the core functionality of our feature, which users identified as a critical need in interviews, highlighting its absence in public spaces.

Validating audio feedback transcription via crowdsourced feedback from hearing individuals

We aim to avoid misinforming users, which is why this validation flow is included—a concern also highlighted by our co-design participant.

EVALUATION

There were minor inconsistencies, but overall the design reflected Google Material Design standards.

We conducted sessions with five experts—four MS-HCI students from Georgia Tech and one Deaf and Hard of Hearing (DHH) education specialist.

These sessions allowed us to gather design feedback from the HCI students and gain valuable insights into the unique needs of DHH users from the education specialist.

Relying solely on color to convey information is neither intuitive nor accessible, especially for colorblind users.

We replaced color indicators with audio bars that increase with background noise, providing a more accessible representation.

The sound history feature was initially perceived as hidden, and participants expressed uncertainty about its necessity.

The original flow let users view audio alert history during a journey, but alerts lacked prominence, and timestamps were unclear. We enhanced the UI to highlight alerts, and to emphasize the most recent event.

Our new features lacked memorability, but what would it take to improve them?

While a reminder feature could highlight the new capabilities, it would ultimately increase the user's cognitive load, resulting in more drawbacks than benefits for the design.

REFLECTION

Establishing strong relationships valuably shaped our prototype

Our prototype emerged from valuable insights gathered through user interviews, feedback sessions, and evaluations—insights made possible by strong connections with the local Atlanta DHH community and experts nationwide.

We worked with several participants multiple times, which was rewarding as they could see how their feedback directly shaped the design.

Incorporating Surveys for Broader Feedback

While one-on-one sessions with a small group provided valuable insights, we recognize the need for broader data collection through surveys.

As we continue refining our prototype for student competitions, we plan to distribute it widely and collect additional feedback from our target user group via surveys.