Game Accessibility: Visual Sound

April 8, 2016

Can you hear me? If you answered yes, you might want to get your ears checked since I’m writing not talking. That leads me to the topic of today’s blog post. How do you improve the accessibility of a computer game where it’s important that players be able to hear certain sounds? It may be important to the story or the gameplay or whatever, but if it is important at all then thought should definitely be given to players who may be hard of hearing or may be in a situation where they just can’t or won’t play with the sound turned on.

In a way, when I’m developing a computer game, I am one of those players. I usually don’t know all the sound effects I need to record for a game until most if not all of the game play is in place. While I could go and record sound effects on an ad hoc basis during production, it’s just easier to compile a list of all the needed sound effects and then record them all at once. Hence my practice of leaving the sound effects until very near the end of production.

All of this means that during the hundreds of test runs through the game that I do during production there are no sounds to be heard. As a result, I become keenly aware of when characters in the game, or the player themselves, need to react to sounds that aren’t there. This has manifested itself several times in the Sleuthhounds series of computer adventure games.

The Squeak indicator shows the pulley that needs oiling.
Click to view larger.

In The Cursed Cannon there’s a point where the player has to use an elevator of sorts to ascend from the ground to a walkway above. However, one of the pulley wheels on the elevator squeaks, which attracts the attention of a security guard. Even with the actual squeak sound, I felt the player needed to be directed to the particular pulley. As such, �squeak� sound effects are drawn on screen when the elevator is in operation. Given that the art style of the game is based on comics, this visual indicator fits perfectly.

The Knock indicator shows someone is at the door.
Click to view larger.

At the very start of The Valentine’s Vendetta another important sound effect is encountered. This one isn’t for a puzzle, instead it’s story related. The game opens with a maid crossing a room, opening a door, and admitting the protagonists Jane Ampson and Pureluck Homes. Sound effect wise, before the maid crosses the room there’s a knocking at the door.

During development I always knew there was going to be a knocking sound. As it wasn’t important to solving a puzzle in the game, I thought initially that I didn’t need a visual indicator for the sound. However, as I played and replayed the game during production, before the actual sound was in place, it really started to weird me out that the maid just psychically knew to cross the room and open the door. So here I incorporated a visual indicator for the sound effect to aid the storytelling.

So why bring all this up now? Because the current in production Sleuthhounds game has need of some visual audio too. In the game, there’s a character who uses a voice changer to do what the label says, change his voice. Most of the time when you talk to him he has his normal voice. Every now and then he turns the voice changer on.

This is a different challenge from the ones in the previous games. In those games the critical sounds are actual sound effects: the squeaking wheel in The Cursed Cannon, the knocking door in The Valentine’s Vendetta. However, here the challenge is how do you show that a person’s voice has changed? Fortunately, from previous creative endeavours I already had a partial solution.

An assortment of dialog bubbles from Cubes.

Since the very first game in the series, The Unlocked Room, I’ve always wanted the games to resemble as much as possible The Cubes comic strips I do over on sister website www.CubesComic.com. In those comic strips I use the standard comic conventions of rounded speech bubbles for talking and fluffy cloud speech bubbles for thinking. I also introduced squared off speech bubbles to represent voices coming from things like phones or speakers.

This line starts in one voice and ends in another.
Click to view larger.

Using squared off speech bubbles seemed like a good solution for most of the voice changed lines in the new Sleuthhounds game. However, as I mentioned above, this was only a partial solution. The tricky part is there are a couple places where the character turns the voice changer on or off in mid-line. Easy to do when you have the actual voiced lines recorded, but how do you represent that visually?

When speech bubbles are shown in my games they are drawn in nice pieces. Four pieces for the corners, four pieces for each side, and a piece for the middle. Oh, and a piece for the stem, but let’s not worry about that now. To make it clearer, you can visualize those nine pieces like so:

Dialog bubbles are made of a grid of visual pieces.

Once, I got to thinking about the challenge, the solution seemed pretty obvious: mix-m-match the regular speech bubbles and the square speech bubbles. Now, when the character says a line where they start in their normal voice and end in their voice changed voice the left side of the speech bubble uses the rounded corners and the right side uses the square corners. When they turn the voice changer off that pattern is reversed, square corners on the left and round corners on the right.

Round corners on the left. Square corners on the right.
Click to view larger.

Representing key audio information visually sometimes requires creative solutions. Sometimes that key audio is needed for gameplay reasons, sometimes it’s needed for story, sometimes it’s needed for dialog. Regardless of the case, using visual audio makes the game more accessible. It also allows you to deal with the classic question of: what is the sound of one hand clapping?

Previous: Walking for Ideas and Creativity   |   Next: Using Game Tech Creatively