Audio Detection Assistant Google Maps

Audio Detection Assistant for Google Maps

If you're not interested in my process, feel free to shift to sections below

Scope

Understanding

Exploration

Design Decisions

Evaluation

Ensuring equal access to audio-based contextual cues

OVERVIEW

The Audio Detection Assistant is a feature designed for the Google Maps platform to enhance the travel experience for Deaf and Hard of Hearing (DHH) individuals by providing contextual awareness of critical audio cues during transportation journeys

This project was developed over the course of a four-month master’s program project as part of my HCI Foundations Design course at Georgia Tech. As a member of a team of four, I was responsible for the creation of design assets and components in addition to research responsibilities including, but not limited to conducting background research and usability evaluations

Timeline

Aug - Dec 2024

Tools

Figma

Notion

MS One Drive

FigJam

Team

Timothy Chiu

Disha Sikaria

Rachit Bhayana

Natalie Jarrett

Skills

Accessibility Design

Interaction Design

User Research

WCAG 2.1

SCOPE

Deaf or Hard of Hearing (DHH) individuals often encounter barriers to fully engaging with their surroundings

The mission was to promote inclusivity and equity by raising awareness of the role auditory information plays in shaping accessible spaces

UNDERSTANDING

The DHH community had no control nor voice in what was being delivered....

Literature Review

We learned valuable insights into the challenges faced by DHH individuals in public spaces

They emphasized the importance of visual cues and improved emergency notifications for enhancing accessibility

Hearing from the DHH community

We conducted semi-structured interviews with diverse participants, including members of or those familiar with the DHH community

With my previous experience working in childhood deafness accessibility, I connected our team with two leading experts of deafness from a renowned graduate program at Columbia University

These interviews with experts highlighted systemic failures in accomodating needs for DHH users as systems were made

This led to the problem statement that would inform the rest of the design research process...

How can we empower DHH individuals by giving them equal access to auditory information? More specifically, how can we increase contextual awareness of auditory cues for DHH individuals within public transportation? 1. 01 What modes might contextual awareness be delivered in? Are there particular sounds that they are interested in hearing? 2. 02 How might access be delivered by empowering them with contextual awareness? Do DHH individuals actually feel their access to auditory information is unequal?

These methods gave insight to the frustrations that DHH users face which ultimately comes down to...

EXPLORATION

We brainstormed 100 ideas—both good and bad—which were then evaluated and refined into 10 strong concepts

Image of 100 ideation concepts on sticky notes

"An enjoyable travel experience is one where I don't have to worry too much about where to go next. I find that time can be wasted when plans are not made ahead of the trip"

Image for 100 ideation concept sticky notes

One focused on the public transit experience stood out

"An enjoyable travel experience is one where I don't have to worry too much about where to go next. I find that time can be wasted when plans are not made ahead of the trip"

We revisited the recurring theme of insufficient audio accessibility in public transit, which emerged from interviews with the DHH community

Image of sketched concepts with highlighted one being the public transportation concept

Once the design concept was defined, we retraced ourselves back to our user group to ensure that needs were being addressed

(2) Participatory Design Sessions: Evolving our concept into designs via expert critiques

Recommendations focused on reducing text reliance, using visual symbols for alerts, and limiting cues to a small set of high-priority ones

Image of participant quotes from interviews, Participant 1: “Amazing that you’re focused on visualizations, I don’t think I can imagine looking down at my phone to read.” Participant 2: “Consider five to seven cues, right now it seems like there are many focuses you are trying to account for.” Participant 3: “I like your idea for including many visualizations, I think that’d be really helpful.” Participant 4: “It has a wide scope, there might be too many different needs for different scopes.”

However, after detailing design concepts with peers from a design foundations class, I discovered that...

Designing an entirely new product was unnecessary, especially since most users already relied on Google Maps

But before prematurely designing mid-fidelity concepts of our feature, I emphasized that we conduct a comprehensive audit of Google Maps to ensure the feature could be introduced where users expect its use

DESIGN DECISIONS

There were contextual nuances determining the optimal placement of our feature

A comprehensive audit of a transportation journey on Google Maps helped to identify events where the feature could be included

After completion of the extensive audit of Google Maps, this led to the conception of three key functions of our feature

View past cues, whether due to the journey being inactive, or simply overlooking the notification

Showcase of Interaction on iPhone Screen

This is useful for announcements, which are not visually indicated, with users citing the lack of visual signage as detriments to their journey

Detection and transcription of alerts and cues, such as approaching trains, fire alarms, or announcements

The ability to visually perceive audio cues is the core functionality of our feature

Validating audio feedback transcription via crowdsourced feedback from hearing individuals

The aim is to avoid misinforming users, which is why this validation flow is included—a concern also highlighted by a participatory-design participant

With the core functionality of the design set, evaluations were prepared in order to ensure that not only did designs address needs, but that the design complimented industry and Google Maps user interface design standards

EVALUATION

There were minor inconsistencies, but overall the design reflected Google Maps standards

We conducted sessions with five experts—four students from Georgia Tech and one Deaf and Hard of Hearing (DHH) education specialist

I focused on distinct criteria of assessment, including evaluating the visual design’s adherence to established design standards and assessing whether it met the expectations of the target user group

Image of heuristic evaluation forms, "We included 10 different criteria to which our system would be evaluated", "A high-level sub-criteria, giving us assessment of how successful our design was delivered, measured to design heuristics cited by Jakob Nielsen"

Below, are three design considerations that we addressed upon receiving feedback on our designs from users and experts

Relying solely on color to convey information is not accessible, especially for colorblind users

Color indicators were replaced with audio bars that increase with background noise – a design consideration that I proposed previously as an affordance for how close a transportation vehicle is to arrival, which is a more accessible representation

Image of interface iterations "Initial Design - Audio levels defined by colors had no semantic significance Not easily determined what color is associated with an audio level", "Final Design - Improved definition of differences afforded by audio spectrum differences Easily legible appearance of audio spectrum button"

The audio history feature was initially perceived as hidden, and participants expressed uncertainty about its necessity

I enhanced the UI to highlight alerts by bolding and increasing the size of text to emphasize the most recent event

Image of interface iterations "Initial Design - Most recent audio cue did not have significance Experts from heuristic evaluation could not determine use case for this screen" "Final Design - Most recent history given bolded, larger appearance Gives significance to recent audio cue in history for recall"

The original flow let users view audio alert history, but alerts lacked prominence, and timestamps were unclear

New features lacked memorability, but what would it take to improve them?

While a reminder feature could highlight the functions, I proposed that it might increase cognitive load, resulting in more drawbacks than benefits

Image of interface iterations "Design Considerations - Experts suggested incorporating reminders of onboarding features throughout the transportation journey. However, repeated reminders could increase cognitive load and potentially hinder the experience."

REFLECTION

Establishing strong relationships valuably shaped our prototype

Our prototype emerged from valuable insights that were made possible by strong connections with the local Atlanta DHH community and experts nationwide

We worked with several participants multiple times, which was rewarding as they could see how their feedback directly shaped further iterations

Incorporating Surveys for Broader Feedback

While one-on-one sessions with a small group provided valuable insights, we recognize the need for broader data collection

As we refine this work for student competitions, we plan to distribute it widely and collect additional feedback from our user group via surveys