In this article we will be creating a procedural melody using the Wwise audio middleware, but the same principle can be used with chords. It is recommended that you at least have a basic understanding of Wwise and can navigate the Interactive Music Layout with ease. To explain the end result simply: we will be triggering stingers that play melodies of a certain length. We will be able to control the length of each phrase while the notes and rhythms will be selected and performed by Wwise. Controlling the length of each phrase can come in handy when triggering the stingers from a game engine, as different player actions can result in varying amounts of notes and we can avoid clutter in the soundscape.
Step By Step Procedural Music
1. Create a Playlist Container, then create a child Music Segment for each phrase length. In other words, if you only want Wwise to be able to play phrases of lengths 3 and 5 notes, create two Music Segments. I wanted the possibility of anything between 1 and 8 notes to be played in the melody, so I added 8 Music Segments.
2. Import your notes into each Music Segment. This will create a child Music Track, which you can see in the Music Segment Editor. I like to export an audio file of my VST playing every note I want Wwise to use when generating my melody, usually a scale. That way I get some more control of the timbre of the sound, as I use VSTs that can’t be loaded into Wwise. Importing MIDI (.mid) files into Wwise is the other option if you’re not too picky about the timbre and processing of your instrument(s) and/or want to use the synths accessible in Wwise. You can use MIDI files in the same way you would audio files, by going to Files->Import Audio Files.
3. Add Music Tracks to the Music Segment. There should be as many tracks as you want notes in your phrase. Change the Music Tracks within the Music Segments to Random Step Tracks by right clicking each track in the Music Segment Editor and selecting Random Step. Want a phrase that is 3 notes long? Add 3 Music Tracks (In this case, named Note1, Note2, and Note3 for clarity).
4. Add each possible note as sub-tracks to the Music Tracks. Setting this up can be somewhat time consuming, depending on how much variety you want in terms of phrase lengths. I recommend setting up your longest phrase first, copy the Music Segment for each shorter phrase, and delete the excess notes.
Example: If I want Wwise to have the option of generating a phrase that is 3 notes long, I will create a Playlist Container with 3 Music Segments. Each Music Segment Music Track will contain all possible notes. Since I’m working with a whole scale, including the octave, I have 8 audio files (although really just 8 references to the same audio file) as sub-tracks for each of the 3 Music Segments. The way you choose your subdivision is by how you space out the Audio/MIDI Files in your Tracks within a Music Segment (new file on each 4th, 8th, or 16th). I find that evenly spacing out your audio files in the Music Segment provides the most musical result, but do experiment.
Tip: If you’d like more rhythm in your melody, include an empty sub-track for each Music Track as well which will function like a pause. You can also use empty sub tracks to change the probability of whether or not a note will be played on a given subdivision.
5. Create a Trigger in Game Syncs for each phrase length you want Wwise to be able to perform. For instance, should you only want the phrases to have two possible lengths, create two triggers.
6. Create another Playlist Container (a Switch Container will also work, in case you’re making branching music). This will contain your “backing track”. Since we’re working with stingers a track must be playing for Wwise to know where beats and bars are located, otherwise stingers will not play. Keep in mind, this audio file can be completely silent but the tempo set in its General Settings will determine the tempo of your procedural melody.
7. Set each trigger to play its respective Music Segment in the Stingers tab of the backing track container. At this point, you can also decide when the next phrase will be played after the stinger is triggered. Mine is set to Next Grid, which corresponds to a setting in the General Settings tab on the far left of this window.
8. You’re done! To test it out, select your backing track, hit play, and trigger your stingers from the Triggers tab in the Transport Control.
Practical Example: An Interactive Music Experience
In February 2021 I made a short game that uses the technique above to create procedural melodies when the player jumps, collides, and clicks with the mouse. My goal for this game is to be an interactive music experience playable by people who are visually impaired, but enjoyable for others as well.
You can try out the project prototype on olivergetz.itch.io/conduit. Fun-fact: The bass instrument in the tune is a sampled string from a face mask.
Wwise vs MAX8 and Pure Data for Procedural Music
When using traditional procedural music tools like MAX8 or Pure Data you can always create some rules or constraints within the randomness. The lack of control of the Random Step Tracks in Wwise is what makes this trick work well. If you want to give up some control over the music for your game and create some interesting, new, and fresh melodies this is a great way to go.