Automated Generation of Background Music and Sound Effects

What kind of background music (BGM) suits a small game? One type of BGM that comes to mind is that found in old arcade games. By 'old,' I am referring to games such as Xevious or Dig Dug from the early 1980s. Retro arcade games of this era often featured BGM composed of short phrases, about 4 to 8 measures in length, which were repeated.

There are several technologies available for automatically generating such BGM. One of them is WolframTones.¹ This seems convenient. However, it appears to have a limitation: it cannot reproduce common phrases seen in old games, like a bass line oscillating in perfect fifths at sixteenth notes.

Among the latest technologies, there's OpenAI's Jukebox;² however, it generates musical waveforms rather than notations, making it unsuitable for retro games. A similar technology, MusicLM,³ has an 8-bit sample available, and seems capable of creating chiptune-like tracks, though it still feels overly rich in expression.

This brings us to Magenta,⁴ a research project by Google, applying machine learning to music and art. Magenta.js ⁵ has been released as an implementation that runs in a browser.

Magenta.js, when provided with a music score, utilizes MusicRNN ⁶ to automatically generate a subsequent phrase. If given a short score from a retro game, it can produce another piece with a similar vibe. I created a simple demo program to test this out (click the image to go to the demo page).⁷

Upon clicking the 'Generate' button in the center of the screen, a new phrase based on the phrase on the left is generated and displayed, played on the right. It generates a simple phrase consisting only of melody and bass tracks, but it's able to produce a tune reasonably reminiscent of retro game music. For multiple tracks, each phrase is created separately, and adjustments are made in case the bass is discordant with the melody (ensuring the frequency ratios are about 1 to 4). Unchecking the checkbox below the center disables this adjustment, allowing for more freeform phrase generation.

When it comes to the automatic generation of sound effects, it seems to be difficult with MusicRNN, which targets melodies. Instead, there are numerous waveform-based solutions, many based on sfxr.⁸ These tools can be utilized to add simple sound effects to action games.

¹. WolframTones ↩

². Jukebox ↩

³. MusicLM: Generating Music From Text ↩

⁴. Magenta ↩

⁵. Magenta.js ↩

⁶. MusicRNN ↩

⁷. short-VGM-generator ↩

⁸. sfxr ↩