Audio Pipeline - making sound effects and music for games
Summary
Sound can be a pretty big part of video games. From an iconic jingle, to mood setting music, to engaging sound effects, audio is a big part of modern video games. It is strange because few games require that there be audio in order to play, but often more so than the graphics, well done audio can really set the mood and pacing of a game.
I can't explain why something you can turn off completely in most games can have such a profound impact. The same is true in the film industry, like when certain characters have their own theme music. You hear it you know they are about to come on screen.
This article is about how I go about creating sound effects and music for the games I develop.
Tools
I create audio clips to be used in games, both effects and music. However, I am not a sound engineer, nor a musician. I am a software developer who needs audio assets for game development.
Current audio tools:
- Hardware:
- MacBook Pro M2 w/16GB RAM (I do all my game-dev work on this machine)
- Sennheiser HD 280 Pro headphones (I use these headphones for audio playback and while editing)
- FIFINE USB Gaming microphone K669G (I tend to use this mic when making the sounds involves my hands doing something)
- Apple Ear Buds w/built in Mic. (I often use this mic when recording voice sounds. The wired kind)
- Software:
- GarageBand (Heavy use of sound loops to make sound effects and compose music)
- Audacity (Editing of sound clips and modifying audio formats)
The pipeline I am going to describe was born out of an evolutionary approach. Initially knowing I had no ability to create sound effects, much less actual music, I figured it was time to go shopping.
Acquiring Ready Made Assets
Wouldn't it be fantastic if you could quickly find and purchase audio assets that you can drop into your game as-is? Think of the time savings. A lot of audio assets available on the Web are very reasonably priced. This really feels like the best path forward for someone like me with no prior experience creating and editing audio, so away I went.
There are multiple sites that offer both free and for-purchase audio clips. A quick internet search will quickly get you paging through thousands of assets on a handful of reputable sites. Step 1 though, is determining if a site is reputable. (This is not an article about doing that, but before making any purchases, or downloads even, you need to do your due diligence.)
Step 2 is finding the right assets. This process of asset-discovery is not for the faint of heart, unless you are looking for placeholder sound effects. I think part of my problem was that I pretty much knew what I wanted, and set out to find it, and after hours of listening to different clips I could only get to something fairly close. Paying for fairly close bugged me, because I knew I was going to be editing the audio anyway. I'd have been happy to spend some cash on ready made assets.
As I was progressing through step 2, not finding what I wanted, but crafting a spreadsheet to score them in terms of suitability, I did a little self-audit. How many hours had I invested so far without having any audio assets? I'm kind of embarrassed to say, that I was about 3 hours into it, headphones on, listening to clip after clip.
Earth Matters has 36 sound effects and 15 music clips. At my current rate, if I settled on fairly close for the one clip I had found so far I was looking at 100s of more hours. All of which I suspected would need some amount of editing.
Another issue was the audio-quality of the clips varies a fair amount from clip to clip. While it is a fairly easy edit to modify the file format, cleaning up background hiss and leveling the volume is a little trickier. You might find a clip you like, but you need to amplify to bring it up to match the volume of the rest of your audio, but doing so amplifies background noises, typically a hissing sound. You can try and clean that up, but I found it came with tradeoffs, at least for the tools I was using and my skill level.
In the end, I gave up the hunt to try and find pre-existing ready made assets.
Hiring a freelance audio specialist
I thought about hiring a freelance person to create audio, but quickly abandoned this approach. I didn't want to have to invest a bunch of time vetting freelancers. Even if I did, I don't feel I have the vocabulary to describe what I would want from them. It is probably doable for the sound effects, but I was afraid of trying to describe music. When it comes to music, I know what I like when I hear it, but I don't know why. Like why did I like a particular sound. Does it matter? It sure seems like it does, so I can describe what I want.
I was also concerned about spending time going back and forth on slight edits, and the time investment that would require. I didn't want to be in a situation where I wanted something, couldn't describe it effectively and just keep iterating on it frustrating both of us. I figured even if I'm not the one making the edits, I at least need some familiarity with the tools and possibilities, so I don't try to describe something impossible.
As I investigated a couple tools, I wondered if it would just be quicker to do it myself. I doubt I'll ever find out, because as of now, I have gotten fairly good at editing audio clips as much as I need to in order to achieve what I want. Sorry freelancers, as of now, I won't be reaching out.
Building a custom audio pipeline
First off, there is no getting around having to learn some stuff. In particular how to do some of the basic things in Audacity. If you want to achieve good results, familiarize yourself with everything under the “Effect” menu. This will enable you to level-set volume, fade an audio sub clip, modify pitch and tempo, and other effects.
My current game-audio sound effect pipeline:- Think of the effect I want. Say it is a wind blowing sound effect, like the tornado in Earth Matters
- Launch GarageBand, filter to the sound effect loops and start looking/listening.
- Sometimes the sound I want is a small segment of a larger unrelated loop. Because of this it is useful to listen to seemingly unrelated loops
- Sometimes I just have to make the effect. I have scratched a fingernail across denim for the tile-sliding sound in SpinWord, but for the wind effect in Earth Matters I made a whoosh sound with my mouth and recorded it.
- Export the clip from GarageBand and open in Audacity
- Trim the clip (there is usually some dead-space at the end of the clip I remove)
- If the clip was in the middle of some longer audio, you will likely want to fade your new clip in and out to some extent to avoid a popping noise.
- When you get used to changing pitch and tempo you can take a piece of audio and modify it to suit your needs. Like moving the pitch up to sound more cartoony, or lowering the pitch of something to make it sound like something completely different.
- Export to the right file format for your project.
- Start up GarageBand and make sure I have all the latest sound loops downloaded.
- Create a new project and browsing loops.
- I typically use 4 channels of audio for music clips. Some sort of beat, a lead instrument, some bass, and an accent of some sort. This isn't a hard and fast rule, but it seems to average 4 channels of audio. The trailer music for Batty starts and ends with 1 channel of audio and maxes out at 6 somewhere in the middle.
- There are some themed groups of loops that work well together, and if they meet the tone of the music I am creating I'll use them and add some other clips to make transitions more interesting. This is like assembling music with legos. Is it tedious? Yes and no. Starting with a themed group of loops helps a lot. Piecing it all together from scratch is very tedious. In the end, be prepared to listen to a lot of audio clips, sometimes over and over. But when it comes together, it feels like magic has been created.
- Export the clip from GarageBand and open in Audacity
- Hopefully you only need to trim the tail end of the song at this point, assuming you used the same volume for all your music in GarageBand. If not, you'll be doing that too.
- Export your song file into the right format for your project.
Recording sound effects
If you are like me, you don't have a sound-proof recording studio with acoustic foam correctly placed to avoid echos. Happily, I haven't needed to create one so far. However, when creating sound effects from scratch, like a foley artist does, there are some things you will become aware of. It's funny and annoying trying to get a good recording sometimes. I've had dogs barking randomly, car doors slamming outside, etc. The worst was a truck idling just down the road for quite a while. The low rumble of its engine getting picked up by the microphone. I just had to wait it out.
The more I can minimize having to make open mic recordings, typically the easier it is to create the audio for a project. However, the better I get at editing sounds, the easier it is to create a basic noise with my mouth, and then using effects, get to a good final product.
Editing sound effects
It is worth taking a few sounds and testing out all of the different effects on them. Knowing the range of what you can do with a few basic effects is incredibly powerful in terms of fashioning usable sound effects. It's not worth describing them here, you really need to hear what they do, and to what extent, as you change the parameters. It doesn't take long and it opens up a world of possibilities.