The API introduced in GameSynth 2023.1 allows other creative tools to control GameSynth via TCP. The possibilities are endless: you can render a patch directly in your DAW’s session, add multiple sound variations automatically to your game middleware, control sound synthesis with curves from your animation software, and more! In this post, we describe how to create new sound design workflows with the commands of the GameSynth Tool API.
First steps
Getting started with the GameSynth Tool API is very simple. There are no extra files to install as the API is entirely built into the GameSynth tool. Moreover, since the commands are sent via TCP, any tool that allows for scripting (and TCP communication) can be interfaced with GameSynth, independently of the language used (C++, C#, Python, EEL2, Java etc…).
The API has its own web page with detailed information. Many examples are already available, and more integrations are being built by Tsugi and the GameSynth community. Speaking of which, should you have any questions, do not hesitate to join our Discord server that hosts a channel dedicated to the GameSynth API!
Like with any TCP communication, the connection will be established if the server (GameSynth) and client (your tool) both use the same IP address, port, and encoding. In addition, a delimiter character is agreed upon, should the server or the client need to send multiple messages at once. The default settings are 127.0.0.1 for the local machine, 28542 for the port, UTF-8 for the encoding and 13 (Carriage Return) for the delimiter, but they can be changed in the Settings window of GameSynth.
A test application is installed with GameSynth in the GameSynth\Tool\API\API Tester folder. It is the best way to ensure the connection is working and to experiment with the commands we will describe in this post.
GameSynth commands
The commands of the GameSynth Tool API can be grouped based on their scope and function. The first group of commands allows you to retrieve information about your GameSynth version, what models are available, what API commands can be used, what are the default paths etc… It also has commands to set the global sample rate used by the tool or to switch between models.
Basically, these commands are useful to determine what features your script or add-on can support, and to set up GameSynth.
The commands available are: get_version, get_commands, get_models, select_model, get_path, get_samplerate, set_samplerate, window_back, window_front.
Patch commands
This next group of commands is probably the most important as it allows you to load a patch, adjust the sketch pad’s drawing, set up the global variation, and render the corresponding audio.
This can be used for example to render an audio file and insert it directly on a track of your DAW, or to generate several sound variations and import them as assets into your game middleware.
The videos below showcase such integrations with Reaper and Unity, but the same can be done for many tools, for instance using the Wwise’s Authoring API (WAAPI), Blender’s Python API, or Pro Tools’ recent scripting system.
The commands available are: load_patch, save_patch, render_patch, get_modelname, get_patchname, get_variation, set_variation, get_drawing, set_drawing.
Repository commands
The API makes it also possible to take advantage of the GameSynth repository and access the largest collection of procedural audio patches.
Commands to search the repository (by name, categories, or tags) and to access its patches have been used in the Unity script shown below for example, with which we located and then rendered the ideal industrial ambience for our scene.
The commands available are: query_patchnames, query_patch, query_categories, query_tags.
Playback commands
Another group of commands is focusing on the playback of the current patch. It is helpful for example to listen to the sounds you found in the repository before rendering them, or for real-time sound synthesis for interactive installations (more on that in the next section).
Useful commands also help you determine if the patch is infinite (in which case you should ask the user for a rendering duration) or if it is randomized (needed if you plan on creating variations).
You can also enable events, which will be triggered in real-time during the sound synthesis and allow you to notify the client application.
The commands available are: play, stop, is_playing, is_infinite, is_randomized, enable_events.
Meta-parameter commands
If you want to control the patch you are playing in real-time (e.g., for interactive installations), a group of commands let you manipulate meta-parameters.
Two examples of systems often used in creative coding are Pure Data (for sound) and Processing (general framework). Although they use very different methods (visual scripting and Java-based programming), it is quite easy to interface with them via the GameSynth tool API.
The commands available are: get_metacount, get_metanames, get_metaname, get_metavalue, set_metavalue.
Automation curves commands
Finally, a group of commands is associated with automation curves. Although they are very similar to the ones for meta-parameters, their usage is quite different as they allow the exchange of curves between GameSynth and other tools.
These can be automation curves coming from a DAW, or motion curves from an animation tool, that can be used to drive synthesis parameters to produce a sound that perfectly match an animation. In the video below, the mechanical sounds of the mecha’s arm are controlled by its rotation in the animation. Update the animation curve, and a new, perfectly matching sound can be generated, without needing any recording or editing!
The commands available are: get_curvescount, get_curvenames, get_curvename, get_curvevalue, set_curvevalue.
The GameSynth Tool API is really easy to use: it does not require any additional installation or configuration, and can be used from any language. Do not hesitate to share your scripts and add-ons with the community, and we can include them on the API’s page too. We cannot wait to discover what new sound design workflows you will invent with this API!
*All trademarks, logos, and brand names referenced on this page are the property of their respective owners.
