August 2021 -
December 2021
The Process
Research
To learn more about the problem space and the users, we started our process by conducting research through a couple different methods.
Method 1
Surveys
First, we conducted an interview to understand the context of online meetings deaf/hard of hearing people mostly attended, as well as the demographics within those meetings, such as the number of deaf or hard of hearing participants. We also hoped to understand what video conferencing applications were mostly used by them, and any common challenges that may come with these applications.
Technical Challenges
"Probably the internet speed, the lower the harder it is to communicate with others."
"Besides the lack of captions, the inconsistent video quality can make it difficult for me, since I rely on lip-reading. If the video freezes or becomes pixelated, I usually get lost."
Procedure Challenges
"Most of my challenges have been setting up meetings to best accommodate my disability and to streamline the meeting the best I can without being a problem to others. That and captions & interpreters missing words in conversations, causing hang ups and confusion."
"delay with information between switches between interpreters hard to follow who say what while I focus on interpreter only"
Limitations of Tools
"When I share my screen, the participants' videos become utterly small. I can barely see them."
"There's no such thing as nuance, and very little by way of body language. So I have to rely on other things to make sure I get all the info. Often that's a lot of extra questions from me, and I think some of the other people in my office think I'm autistic. But I've told them that I have trouble understanding meetings, and I'm trying to not be a jerk."
Method 2
Competitive Analysis
By conducting a competitive analysis, we want to get a sense of the accessibility features popular video conferencing platforms have to support the DHH population. By looking at these features, we can find out how these platforms are different and be able to compare why some DHH people might prefer one or another.
Features
Zoom
Microsoft Teams
Google Meet
Captioning
When captioning enabled, the host can type closed captions or assign a participant/third-party service to add closed captions.
Live transcription feature can also provide captioning using Automatic Speech Recognition.
Captions can be streamed into the meeting or you can choose to view them on a separate screen.
Live transcription feature can also provide captioning using Automatic Speech Recognition.
Captions can be streamed into the meeting or you can choose to view them on a separate screen.
Teams can detect what’s said in a meeting using Automatic Speech Recognition and present real-time captions.
Captions are available on the desktop and mobile app. They are not always available to all companies.
Captions are available on the desktop and mobile app. They are not always available to all companies.
Live captions are available through Automatic Speech Recognition.
Does not support third-party captioning services.
Does not support third-party captioning services.
Interpreter
Integration
Allows Relay Service integration via phone number with an upgraded plan.
Allows Relay Service integration via phone number with an upgraded plan.
Allows Relay Service integration via phone number.
Pinning of specific videos
Allows you to pin up to one video.
Allows you to pin multiple videos.
Allows you to pin up to one video.
Text chat
and private communication
Has text chat feature.
Supports private messaging between participants.
Supports private messaging between participants.
Has text chat feature.
Cannot send private messages between participants within specific video conferences but can send private messages through the general Teams application.
Cannot send private messages between participants within specific video conferences but can send private messages through the general Teams application.
Has text chat feature but opening the chat will hide part of the video.
Cannot send private messages between participants.
Cannot send private messages between participants.
Transcriptions
Business, Education, or Enterprise license required.
Audio transcription only supports English.
Audio transcription only supports English.
Only available for scheduled meetings on the installed desktop version of Teams.
Only available for specific Microsoft licenses.
Only available for specific Microsoft licenses.
Transcription available through an external provider.
Video quality and view options
Good video quality.
Gallery view and active speaker view options available.
Gallery view and active speaker view options available.
Good video quality but works best on Microsoft products.
Small gallery and large gallery view options available.
Small gallery and large gallery view options available.
Good video quality.
Gallery view and active speaker view options available.
Gallery view and active speaker view options available.
Co-hosting ability
Allows the host to share hosting privileges with other users to manage administrative logistics.
Meeting hosts cannot be changed but hosts can assign a presenter role to users before and during meetings that give users more privileges than normal viewers but less than the host.
Allows the host to share hosting privileges with other users to manage administrative logistics.
Personalization
Caption customization available.
Meeting name changeable.
Meeting name changeable.
Caption customization unavailable.
Caption customization unavailable.
Meeting name changeable.
Meeting name changeable.
Method 3
Field Observation
We conducted observations on how deaf/hard of hearing people conducted online video conferencing to gain first-hand knowledge on how the experience and understand communication dynamics in a more context-driven manner. It allowed us to see more organic and natural responses from the participants.
.png)
Method 4
Semi-structured Interviews
Besides the general demographic information collected from the DHH population via surveys, we also chose to conduct interviews to dip deeper and explore more into user needs and pain points. Because we were interested in understanding the situation from all perspectives, we conducted interviews with both the DHH population, as well as DHH translators. From our interview data, we organized our data from each user group into an affinity map.
Click to view full affinity maps
Key insights from our research
A new Zoom: Zoom is the most used platform among DHH people. However, despite it's popularity, there are still many missing functionalities and Zoom can be improved upon to further support DHH people.
Burnout: Interpreting for long periods of time can be incredibly tiring. It requires a lot of energy from the interpreter, both mentally and physically. Sometimes, if possible, interpreters will switch out with each other so they can take breaks.
Check your quality: Good video and internet quality is especially crucial for online meetings including DHH people. It is also important to be able to see most of the upper body, as sign language is very expressive and uses a wide range of gestures.
Constant feedback: Good video and internet quality is especially crucial for online meetings including deaf or hard of hearing people. It is also important to be able to see most of the upper body, as sign language is very expressive and uses a wide range of gestures.
Preparation is key: Both DHH and interpreters think that good meeting management prior to the meeting, such as sending out content briefs, glossaries, crucial information, FAQs, etc., contributes greatly to having a smooth meeting experience. Additionally, both groups found that standby time that occurred prior to an in-person meeting was useful in that the DHH and interpreters could casually chat to introduce themselves and set up any communication rules. However, with the move to online meeting spaces, this casual conversation atmosphere did not exist anymore.
Not just sign language: Not all DHH people depend on sign language for translation. There are people who use captions, transcriptions, even just read lips, etc. DHH people will also take note of visual cues from other participants to understand the conversation. Additionally, while sign language is incredibly powerful for communication and translation, it exists as a language with its own lexicon and rules and does not translate one-to-one with spoken English.
Design and Testing
With all the information and the key insights gathered during the research phase, we moved on to designing possible solutions and testing them with users.
Step 1
Design Supports
Using our research, we created two personas that represented our target user groups: the DHH, and the interpreter. We used the personas as a reference to ensure we were adequately addressing user needs when designing.
Step 2
Concept Ideation and Feedback
During our first brainstorming session, we used our research findings to narrow down six general issues that we thought were most important:
1.
Be able to help DHH speak up and command attention when they want to add to the conversation or ask for clarification.
4.
Quote part of live captions and ask questions directly based on that, or even have a complete caption history.
2.
Be able to address situations when video isn’t turned on and there is no facial expression or sign language feedback.
5.
Customize window view size for better multitasking.
3.
Have better meeting preparation procedures for both the DHH population and their interpreters.
6.
Check and remember when arms, etc. extend beyond camera borders, which makes it difficult to translate information accurately.
During our second brainstorm session, we focused on and combined the current six to create our three main concepts. We then conducted a feedback session with DHH people and interpreters.
Concept 1: Pre-meeting chat rooms
Stimulates the pre-meeting atmosphere of in person meetings where participants can casually chat and get to know each other before the meeting starts.
FEEDBACK:
Received generally positive feedback from our participants. For both our DHH and interpreter participants, they liked how they could chat to get to know each other and match signing styles for easier communication. They also mentioned that it would be useful if the host was running late, so that they would be able to get in contact with other participants without the reliance on the host to start the meeting.
Concept 2: Live captioning and transcript with actions
A live caption that is saved in real time to create a transcript. Participants can scroll through the transcript and interact with it in many different ways to help them better understand the content of the video conference.
FEEDBACK:
Received mixed feedback from our participants. For some, they weren’t a big fan of long texts and the heavy focus on reading, responding that they probably wouldn’t use this feature. However, for DHH participants who rely on captioning, many said that they really liked this feature and it would help reduce with follow up meetings. In conjunction with a translator, they liked that it could catch something that an interpreter might miss and vice versa.
Concept 3: Video checks and view personalization
Participants are guided before they enter the video conference to make sure their video feed meets specific accessibility guidelines. In the meeting, they can customize their viewing experience to maximize the most important aspects.
FEEDBACK:
Well received by all participants. Our participants were especially interested in being able to move videos around and resize people. For one participant they mentioned that “[their] vision likes going to the left side so [they] would put the interpreter on the left.” They liked the control and customization of the meeting space.
Step 3
Wireframes
From the positive responses to our concepts, we moved forward with creating wireframes, which are quicker and less costly than high fidelity mockups. These wireframes allowed us to conduct user tests and easily make changes.
.png)
Step 4
Wireframe Testing
During the user testing, we gathered users’ feedback and the ratings for the wireframes. Though participants generally spoke highly of the designed features, they found some interactions confusing, risky, or may incur negative impact. We examined the critical issues and brainstormed on further improvements.
Step 5
User Flow Diagrams
Based on the feedback we received from DHH people and interpreters, we found that meeting hosts, DHH people, and interpreters each have different expectations on an online meeting software and will focus on different features when exploring our wireframe. We built three user flow diagrams for meeting host, DHH participant, and interpreter separately to list what different types of users will be able to do with each screen of our prototype.
Step 6
High Fidelity Prototype
Our solution was based on the most preferred online meeting tool, Zoom, according to data collected in our research phase. To create a seamless addition to the Zoom platform, our high fidelity design followed the existing design system of Zoom in the final design of GUI. Check out our interactive prototype below!
