In this post, we will describe a new workflow for the creation of sound effects for your game cinematics or animations. By controlling GameSynth directly from your game middleware (via the GameSynth Tool API), you will be able to create perfectly matching sound effects in real-time. Although we will be using Unity, you can do the same with any other game engine.
Example 1: Transforming the voice of a demon lady based on the “wrath” parameter in the game
Example 2: Controlling the engine’s noise based on the ship’s altitude and speed
GameSynth and Game middleware
Out of the box, GameSynth offers several ways to interface with your game middleware. For instance, it is possible to import animation curves as automation in the tool, to drive the sound parameters from motion. GameSynth can also automatically generate sound variations and export them to your game project with a single click, creating any required files and assets, and setting parameters as needed.
Using the GameSynth Tool API will allow you to go even further, writing scripts that will remotely control many aspects of GameSynth, for instance allowing you to browse its patch repository and render sounds directly into your Unity project.
Sending messages in real-time
The GameSynth Tool API makes it also possible to play and stop the current patch, and even to set its meta-parameter values. By associating game variables or the camera movement with the meta-parameters, it becomes possible to generate sound effects in real-time that perfectly match what is happening in the game editor. This is great when you want to design the sounds for in-game animations or cinematics!
First, since the GameSynth Tool API communicates via TCP/IP, you will need to add a tiny bit of code to send the messages. You can use the server code provided with our Unity demos.
Then, you just need to set up meta-parameters in your GameSynth patch and update their values from your game code. This can be done easily by sending a single string to the GameSynth Tool API. You can either reference the meta-parameter by index or by name.
For instance, when referencing by name, you may send the following string:
“set_metavalue BY_NAME energy 0.7”
Don’t forget to add quotes around the name if it contains any spaces, and note that the value you send must be normalized between 0 and 1.
Now let’s see how we implemented the examples above!
Demon Lady Example
In this example, by adjusting a single “wrath” parameter, we changed the appearance of a character from a human to a demon, making her progressively grow horns and adding flames in the background. The value of the “wrath” parameter, mapped between 0 and 1, was sent to GameSynth on each update.
On the GameSynth side, a ‘wrath’ meta parameter was created. It is connected to multiple sound parameters in the effect rack of the Voice FX model, such as the amount of pitch shifting, reverberation, gain, etc… The patch used is based on the “Alien Hive” preset.
Thank you to voice actress Yui Asami for providing the original voice sample!
Spaceship Engine Example
In this example, we use 3 parameters linked to the motion of the spaceship: altitude, speed, and position, and send the corresponding parameter values to GameSynth on every update.
In the GameSynth patch, we defined 3 three meta-parameters controlling engine power, wind deflection noise, and panning. The patch itself is based on the “Antigrav Speeder” patch available in the GameSynth repository and uses Noise, Beam, and Mega Saw generators to produce the engine sound.
Advantages
In addition to being able to generate wave files, GameSynth can be controlled remotely by other tools and engines. This makes it possible to define entirely new workflows to create sound effects.
In the examples above, not only will the sound effects perfectly match what is happening in the game, but it is easy to create as many variations as needed (e.g., by moving the spaceship around) or to quickly edit the audio effect (e.g., by adjusting the sound parameters of the Voice FX patch for the demon lady).
In addition, it opens the door to various applications outside of the gaming world, for example digital installations that can benefit from the increased interactivity and high audio quality provided by the procedural audio engine of GameSynth.
