Interactive Audio Crimes in Heavy Rain

How OCD am I? I refused to fold the piece of card that came with Heavy Rain in to the intended origami figurine because I didn’t want to inflict any creases on it and, instead, recreated it, warts and all, using a blank piece of paper. That’s special :)

Heavy Rain wears its cinematic points of reference on its sleeve, making little attempt to disguise the potent musk of Fincher, Demme, Hitchcock and Lang. It is perhaps so keen to communicate that it is a familiarly cinematic experience that it’s in danger of coming across as a second rate copycat. Which does the game a disservice, because the way the gameplay allows you to interact with its story, characters and environment offers experiences and feelings not often encountered in a game, which was a pleasant surprise, and it is primarily for this reason that Heavy Rain is an experience worth checking out.

The audio experience in the game is pretty good overall. Quite a lot has been made of its acting but, frankly, it’s a mixed bag which, whilst an improvement to the normal dross in games, nonetheless fails to be on par with the cinematic experiences it tries to emulate. Interestingly, the interactive conversations have consistently poorer performances and “off” delivery of lines than the linear performances which play out in scenes, which makes sense given how hard it is to stay on target whilst performing branching conversations (both as a performer and as a director). I suspect what people have been responding positively to is the facial capture, which really is impressive – I love that the actors with speech defects have this accurately reflected in their facial mocap, because this level of detail really helps to fuse the visual with the aural.

However, there are several aspects of the presentation of the audio in Heavy Rain which stood out to me as being not up to scratch. Some of it is understandable and useful as stimulus for discussion, but some of it is just plain disappointing. What follows isn’t meant to be a personal criticism of the audio personnel (assuming they are aware of these issues and would have made them better if they could have), lord knows I’ve committed plenty worse audio crimes in the past, they’re simply observations and thoughts I had whilst playing the game.

• The opening shot of the prologue had a dog bark sitting on top of the first piano notes of the score which I found to be pretty clumsy – the dog bark should have followed after the piano chord. The bark is needed in this scene; it immediately gave the sense of a warm family home before I even new where I was (knowledge of the game’s story from reviews probably helped here, I suppose you rarely come to a game with zero expectations or prior knowledge). Contrasting this feeling against the sadness of the score this felt bitter-sweet which is a rather sophisticated feeling “for a game”, especially in the opening seconds. But this a-rhythmical, temporal clash between the dog bark and music annoyed me so I restarted the game – it turned out it was down to serendipity and must have been due to a random stinger or random start point in the ambience track. The notion of a rather linear interactive experience being ruined because of a lack of control (or lack of thought) over the audio is intriguing – we don’t currently have a nice easy way of telling a sound “be random, but not in this context”…

• Pressing start brings up the good ol’ reliable Start Menu (note that said button is rarely used to actually start things these days – a good example of why game controllers are totally baffling to newcomers to the medium). The menu has a nice “heavy rain falling on a puddle” visual effect, and there is a suitable transition sound to lead you in to the menu. It’s strange then that when you press the select button you are taken to the Controls page (again, why is it called select and never used to select anything?) which, despite having an identical visual effect, has a weedy dry transition sound rather than the lush wet one which scores the Start Menu transition (and works a million times better). Actually, it’s worse than that – the sound which plays on entering the Controls page also plays when entering the Start Menu, it’s just that the start menu has an additional sound thrown on top. Just about every game ever (EVER!) suffers from these kind of messy UI sound issues. That’s because UI code is uniformly a total nightmare spaghetti junction of legacy filth – you often can’t easily add a sound to this mess without having to hack it in in several different places, all of which makes the whole thing even more likely to have problems especially once the code gets tweaked in ignorance of the sound hooks. Gah. My favourite UI audio cock-up is the same sound playing on top of itself – best case scenario is it plays back twice as loud (I love how that is “relatively good”), worst case scenario it plays back all phasey (either because you are adding a bit of random pitch shift or because the sounds get called on consecutive frames). Brilliant. Even the X-Box 360s dashboard menu does this kind of stuff. Gotta love how uniformly slack our UI sound is as an industry (and we’re the experts – doh!).

• The game has some really nice foley (e.g. resting your arms on the balcony railings outside your room in the prologue) but it also has some “missing” foley as a result of the mix (your footsteps on the balcony are completely inaudible in the prologue which makes your character feel a bit floaty and fake). I wonder how they’re mixing this stuff – global falloff? That’s fine 90% of the time, but you really need to be able to tweak it on a per-camera basis as each shot has a unique context which may need to be acknowledged in the mix – “far away = quieter” doesn’t really cut it when you’re trying to be cinematic. There’s also some crappy foley at times – the footsteps are incredibly inconsistent, varying between ‘subtle and varied’ to ‘inappropriately clompy with limited sample sets’. There’s a bug in the hall landing of your house (again, in the prologue) whereby your character’s bare feet are scored with these nice, subtle footstep sounds, but if you go downstairs and come back they get stuck on some gawd awful “carpeted footstep” or “shoe footsteps” – this is interesting simply because it A/Bs the difference between nice subtle foley and the rubbish overt foley we are used to in games. Every subtle foot movement now has a great sodding clomp sound on it. I then took my dude outside where he got attacked by a passing bee or giant invisible hornet – there was no buzzing sound (fine, I guess?), but the sound of my manic feet clomping away rather ruined whatever the moment was meant to represent. Brilliant. However, the worst foley crime in Heavy Rain is the ****ing clapping – whenever anyone claps in this game it is deeply upsetting. Good foley grounds your characters in their world, bad foley only serves to remind you of the artifice of what you are looking at – this is just as true in linear media as in games.

• The reverb effect applied to the “in your head” VO is inappropriately nasty – really toppy and ear-bleedy. Presumably this is a real-time reverb rather than something baked in to the audio? It’s a really strange decision.

• The dynamic interactions aren’t scored effectively with sound. For example, the sliding door in the bedroom does not adapt to the speed that I move it at, there is silence and then a one-shot sound event plays. If I’m moving the door slowly then the result is the sound finishing a second before the door actually closes. Interestingly, there’s a point in the animation where it doesn’t matter what speed you are moving at and the door closes at a set speed – this is the point the sound should have played at. Hell, that was probably the idea, but if your system is based on timing rather than designed to deal with the dynamic interaction you are setting yourself up for a fall when the designers come along and change everything (which you know they’re going to do). Similarly, when shaving, there is a one-shot “shaving” sound that plays irrespective of how long the razor spends in contact with my skin. I appreciate that getting this right is a bit more work, but given the importance of these mechanics to the game (reinforcing the sense of ‘mundane normality’) I think this is a bit of a let-down.

• Coming out of the Start menu always results in a momentary loss of audio for a few frames just after the game (and audio) has resumed. This is the kind of stuff you have to put up with when people outside the audio department don’t really care about the audio and they are more focussed on shipping the game than maintaining the quality of the player’s experience. There’s no way they would have shipped the game with a bug that caused the screen to go blank for a few frames after coming out of the Start menu, but for some reason it’s OK for the audio experience to be a bit shonky?! These kinds of issues can be hard to fix, but there are always methods of wallpapering over the cracks even if the root problem can’t be addressed, e.g. delay resuming the game by an amount equal to the delay in the audio.

• A beautiful, sad ending to the first half of the prologue was ruined by an audio fade-out that didn’t finish properly and got cut off mid-fade. Criminal. All that time and effort gone in to making this scene do its thing, utterly shat on by poor audio presentation. This didn’t happen again in any other scenes, though I found some of the fades to be a bit fast (linear fades are never going to sound good – don’t use them!).

• This one is the daddy of all crimes – providing only two voice samples for the “shouting the name of someone who is lost” sections which crop up four or five times throughout the game. I don’t care how this came to be, there really is no excuse – it’s a really poor design decision. It makes me properly physically angry, shouty and bitey. And just to tip me over the edge, the same samples are used again later in the game in a totally different context. This could have been a really poignant, meaningful moment – reusing the assets reminding me of the context and emotions I was feeling earlier in the game, but that couldn’t happen when the only emotion I was feeling (and recalling) was anger at the game itself.

• Trophy announcements appearing just at the end of a chapter, the annoying chirpy sound ruining the poise of the moment. To be fair, this afflicts all console games these days, but it’s especially jarring here – there isn’t really a good time for this event anywhere in this game, which begs the question “why bother”?

• The mix is good 99.9% of the time in Heavy Rain. But occasionally it’s insanely bad, and always at incredibly inappropriate moments. At one point my virtual wife was berating me for being a bad father but I couldn’t hear her because the background walla in the police station of people muttering and writing notes with pencils was louder than the dialogue. I entertained the idea that this was intentional, perhaps my character “wasn’t listening”, but nothing else in the setup of the scene backed this up. Weird. The very last scene in the game was similarly ruined by the fact that the music was louder than the characters talking to each other.

• One of the most common crimes was the lack of transitions between music cues. You expect this kind of naïve implementation in a LittleBigPlanet level (UGC FTW!), you do not expect it in a finely crafted cinematic score. The music was always appropriate but the lack of transitions from one piece to another was so incredibly jarring it consistently took me out of the experience.

• The other consistently annoying thing is the way that if you “fail” one of the quicktime events you have to restart it and put up with audio that is 100% identical. This wasn’t a big deal most of the time for me, but on those handful of occasions where the UI completely fails to communicate exactly what you are meant to do (the infamous cutting off your finger scene being one of the worst culprits) and you are forced to repeat a sequence a dozen times before you work it out, the repetition in the audio becomes incredibly annoying. It’s OK that the animation/mocap is the same each time, I’m too busy focussing on the UI to notice, but it’s really weird and annoying that your character makes the same breathing sound every few seconds – it feels like an unexplained time-warp rather than them/me simply “having another go”.

• Finally, my favourite crime by far was the sound of an idling car engine during the intro playing over the top of David Cage’s screen credit even as said car had pulled away in to the distance (I guess the object got unloaded, leaving the sound stuck on?). Oh the sweet, sweet irony. To be fair, it’s pretty quiet, but the idea that any film director would let that slip or that any film sound person would even dare to allow such a thing to happen is totally beyond comprehension.

Conclusion – interactive audio is hard! On a film, these kind of “technical” and aesthetic problems can be fixed in seconds – in fact, they’re unlikely to materialise even at the final mix stage because they’re likely to have been caught and addressed as a matter of course. But on a game, fixing this stuff requires more effort, a higher pain threshold, and often requires collaboration with and support from the team you’re working with (at a level unprecedented in most film productions).

The audio experience in Heavy Rain, and its failings, suggest a team experienced in high-production-quality linear media, but lacking in interactive audio development experience (e.g. nicely recorded foley which you can’t hear at times due to a naive implementation approach, and repetitive audio which screams “My first game! Our assumptions were tested and they were so wrong it hurts!”). Whilst these kinds of issues can be found in most games (including many of the ones I’ve worked on – always be learning!), they really stand out in a linear, cinematic game because the point of reference isn’t other games, it’s the linear soundtrack perfection of film. That’s a cruel juxtaposition, but whatever the reasons are for these failings – a lack of experience or time, poor collaboration or a lack of support and understanding for audio from the other disciplines – it remains a quality bar by which the audio experience for a game can and should be measured.