Extended KotD #21: _MIDISOUNDBANK

[a740g]

_MIDISOUNDBANK is a QB64-PE v3.14.0 feature. However, before we dive straight into it, let's take a look at a bit of history.

What is MIDI?

MIDI (Musical Instrument Digital Interface) is a communication protocol that enables the control and synchronization of electronic musical instruments, software, and other devices. Unlike audio data, MIDI messages are lightweight, making them ideal for real-time performance and control. MIDI was standardized in 1983 by the MIDI Manufacturers Association (MMA), and its continued use today highlights its enduring versatility.

What is a MIDI File?

A MIDI file is a digital file format that stores MIDI data. Unlike audio files, which contain actual sound recordings, a MIDI file contains instructions for generating sound. These instructions include information such as note pitch, duration, velocity, and timing, as well as control changes and other parameters. MIDI files are highly compact because they do not store actual audio, making them ideal for efficient storage and manipulation of musical compositions. MIDI files can be played back by any MIDI-compatible device, which interprets the data and generates the corresponding sounds.

What is a MIDI soundbank?

A MIDI soundbank is a collection of audio samples and settings that a synthesizer uses to generate sounds when playing a MIDI file. It defines how the notes and instructions in the MIDI file will sound, allowing for different instrument tones and qualities.

QB64 and MIDI

The last SDL version of QB64 (v0.954) was the version that had native MIDI playback support. MIDI and several other audio formats were dropped in the OpenGL version of QB64. QB64-PE v3.2.0 reintroduced support for MIDI playback, although it was initially hidden behind $UNSTABLE:MIDI. QB64-PE v3.14.0 moved MIDI playback support out of $UNSTABLE:MIDI and made it a first-class feature. $UNSTABLE:MIDI and $MIDISOUNDFONT were deprecated, and the new _MIDISOUNDBANK statement was added.

_MIDISOUNDBANK Syntax

Code: (Select All)

_MIDISOUNDBANK: fileName$[, capabilities$]

fileName$ is the file name of the soundbank or a buffer containing the soundbank data.
capabilities$ (optional) can be two flags if specified.

• "memory" - indicating that the fileName$ is a buffer and not a file name and one of the following.
  
• "ad": Global Timbre Library format for Audio Interface Library.
  
• "op2": DMX OPL-2 format.
  
• "opl": Global Timbre Library format for Audio Interface Library.
  
• "sf2": Creative's SoundFont 2.0 format.
  
• "sf3": MuseScore's Ogg compressed Creative SoundFont 2.0 format.
  
• "sfo": Bernhard Schelling's Ogg compressed Creative SoundFont 2.0 format.
  
• "tmb": Apogee Sound System timbre format.
  
• "wopl": Vitaly Novichkov's OPL3BankEditor format.
  

Example: Using an embedded soundbank

Code: (Select All)

$EMBED:'./tiny.sf2','mysf2'



_MIDISOUNDBANK _EMBEDDED$("mysf2"), "memory, sf2"



handle = _SNDOPEN("canyon.xmi")

_SNDPLAY handle

QB64-PE MIDI Player Engine

The QB64-PE MIDI player engine heavily borrows from the foobar2000 MIDI Player plugin (https://www.foobar2000.org/components/view/foo_midi). Unlike the foobar2000 plugin, however, the QB64-PE implementation is cross-platform, except for VSTi support (more on this later). The player internally uses various backends to play MIDI files. All backends need a soundbank. However, the Opal+yfmidi backend has a tiny soundbank embedded by default that is used to play MIDI files if no other soundbank is loaded.

Synth	Type	Setup	Difficulty	Platform	Quality
Opal+yfmidi	Frequency modulation	None	Can I play, Daddy?	All	Retro
Primesynth/TinySoundFont	Sample-based	Less	Don't hurt me.	All	Awesome
VSTi	Implementation defined	More	Bring 'em on!	Windows	Hell yeah!

Playing a MIDI file without any soundbank loaded will default the MIDI engine to use the Opal+yfmidi backend and the default embedded soundbank. Note that the FM Synthesis used is similar to that of a real Yamaha OPL3. However, unlike hardware OPL3 that supports only 18 channels, the Opal+yfmidi backend supports 108 (18 x 6) channels. This allows it to produce exceptional quality retro-sounding music. External AD, OPL2, OPL, TMB, and WOPL FM bank formats are also supported by this backend.

Example: Playing a MIDI files using FM synthesis

Code: (Select All)

handle = _SNDOPEN("groove.xmi")



_SNDPLAY handle

The Primesynth/TinySoundFont backends support sample-based synthesis using SoundFonts (SF2). As such, they can reproduce music using real-life instrument sounds at very high quality. The TinySoundFont backend supports compressed SoundFonts like SF3 & SFO. Note that compression is lossy (OGG), and hence some degradation in quality can be perceived. QB64-PE does not contain any SoundFonts, so the user must specify a SoundFont using _MIDISOUNDBANK to use this backend.

Example: Playing a MIDI file using a SoundFont

Code: (Select All)

_MIDISOUNDBANK "4mgmgsmt.sf2"



handle = _SNDOPEN("onestop.mid")

_SNDPLAY handle

The VSTi backend is Windows-only because it uses a VSTi DLL. QB64-PE itself does not include any code from Steinberg Media Technologies. Therefore, it uses a VST Host module to communicate with the VSTi DLL. Yours truly has taken the effort to adapt the foobar2000 MIDI VST Host for QB64-PE, and the same is available at https://github.com/a740g/vsthost. One should compile the project using Visual Studio 2022 to get the vsthost32.exe and vsthost64.exe files. My sole motivation to add VSTi support to QB64-PE was to get MIDI playing using the Yamaha S-YXG50 VSTi found at https://veg.by/en/projects/syxg50/. A VSTi can be specified using _MIDISOUNDBANK. However, QB64-PE expects the VST Host to be in the same directory as the VSTi DLL. Otherwise, the VSTi will not be loaded, and playback will fail.

Example: Playing a MIDI file using a VSTi

Code: (Select All)

$IF WINDOWS THEN

_MIDISOUNDBANK "syxg50.dll" ' load VSTi on Windows

$ELSE

_MIDISOUNDBANK "4mgmgsmt.sf2" ' fallback to a SoundFont on other platforms

$ENDIF



handle = _SNDOPEN("onestop.mid")

_SNDPLAY handle

The example player (MIDIPlayer64) below contains code that shows how to load various soundbanks, use the various backends, and several other features. Feel free to experiment and use the code in your own projects.

 

Things to try

• Play monkey.mid and Wolfenstein 3D*.mid from the midis directory using FM Synthesis.
  
• Play THE_RAIN.mid, ECHOES.mid, striving.mid, and DOOM MIDIs in the midis directory using soundbanks/gzdoom.sf2.
  
• Play COLDWAVE.mid, MHBB.mid, and TK_EATS.mid in the midis directory using soundbanks/wingroove.sf2.
  
• On Windows, play bi2_polkovnik.mid, Cop Out by Sam Sketty YME '96.mid, Fool's World (XG).mid, The Major Seven (XG).mid, and Technocrat XG Sam Cardon.mid using soundbanks/syxg50.dll.
  

Cheers!

Questions?

[Dav]

Great post. The MIDI support in qb64pe is really awesome.  Wow, I did t realize sf3 was supported too. I’ve been converting my really big sf2 to sf3 to save space on my iPad. Gonna test them out in QB64PE.

Thanks for the MIDI support!

- Dav

[a740g]

Thanks @Dav. 

[Steffan-68]

First of all, a great explanation, I like it.
And their example player (MIDIPlayer64) is simply brilliantly made.
I've been playing around with _MIDISOUNDBANK  and I noticed something that wasn't said here or in the WIKI.

Code: (Select All)

_MIDISOUNDBANK "4mgmgsmt.sf2"



 



handle = _SNDOPEN("onestop.mid")



_SNDPLAY handle

 

That works quite well.
But if you want to switch from one soundfont to another in your program, for whatever reason, you always have to first 
Load the sound font and then the sound. All previously loaded sounds will not be played with the new sound font. 
That wasn't explicitly stated here.
 
I also noticed that in such a case there is a pause of 1 second between load Soundfont and load Sound and Play
You have to insert it, otherwise it won't work either.
I made a little demo to quickly hear how different sound fonts sound with one sound.
I noticed this.
Take all _DELAY 1 out and it won't work anymore

Unfortunately I can't upload the sound font, despite 7z it's still 249 MB, which won't be accepted here.

Code: (Select All)

Screen _NewImage(600, 400, 32)

'_MIDISoundBank ".\Nice-Strings-PlusOrchestra-v1.6.sf2"

Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

_SndPlay Song&



Print " 1 Normal"

Print " 2 DooM"

Print " 3 Nokia"

Print " 4 Orchester"

Print " 5 Aspirin"

Print " 6 Airfont"

Print " 7 Mega Sound"

Print

Print " ESC EXIT"







Do

    k = _KeyHit



    Select Case k

        Case 27 ' esc

            Exit Do

        Case 50

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\gzdoom.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

        Case 51

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\Nokia_S30.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

        Case 52

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\Nice-Strings-PlusOrchestra-v1.6.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

        Case 53

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\Aspirin_160_GMGS_2015.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

        Case 54

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\airfont_320_neo.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

        Case 55

            _SndStop Song&

            _SndClose Song&

            _MIDISoundBank ".\24.1mg_mega_sound_v2.0_bank.sf2"

            _Delay 1

            Song& = _SndOpen(".\Survivor - Rocky III - Eye of the Tiger.mid")

            _SndPlay Song&

    End Select

    _Limit 10

Loop

System

[Pete]

Way too advanced for me. I'm more of a BEEP man.

I have fiddled around a bit with sound and batch files. 

For instance, what do you get when you put SOUND 0, 0 in a batch file?

ECHO
The sound of silence

Pete  

+2 to Sam and Rho for their work on this article.

[a740g]

Way too advanced for me. I'm more of a BEEP man.

I have fiddled around a bit with sound and batch files. 

For instance, what do you get when you put SOUND 0, 0 in a batch file?

ECHO
The sound of silence

Pete  

+2 to Sam and Rho for their work on this article.

Thanks Pete!

---

First of all, a great explanation, I like it.
And their example player (MIDIPlayer64) is simply brilliantly made.
I've been playing around with _MIDISOUNDBANK  and I noticed something that wasn't said here or in the WIKI.

That works quite well.
But if you want to switch from one soundfont to another in your program, for whatever reason, you always have to first 
Load the sound font and then the sound. All previously loaded sounds will not be played with the new sound font. 
That wasn't explicitly stated here.
 
I also noticed that in such a case there is a pause of 1 second between load Soundfont and load Sound and Play
You have to insert it, otherwise it won't work either.
I made a little demo to quickly hear how different sound fonts sound with one sound.
I noticed this.
Take all _DELAY 1 out and it won't work anymore

Unfortunately I can't upload the sound font, despite 7z it's still 249 MB, which won't be accepted here.

Glad, you liked it. 

Great feedback BTW.

Yes, previous MIDIs not using the newly set soundbank is an expected behavior. We'll have that documented in the wiki.

On the delay thing, I think it is some kind of synchronization issue. I haven't figured it out yet. But I am on it.

[SMcNeill]

Unfortunately I can't upload the sound font, despite 7z it's still 249 MB, which won't be accepted here.

The server here has a 200GB file size limit. If you can break that file down to about 150MB in two 7z files, I'd imagine you could upload it with no problems whatsoever. Don't know if it's worth the effort though, but thought I'd point it out, just in case.

[a740g]

@Steffan-68 I've found out why the _DELAY makes your code work. So, miniaudio (our audio backend) does file-name hashing when loading audio files. The following is a direct quote from the miniaudio docs.

When loading a sound from a file path, the engine will reference count the file to prevent it from being loaded if it's already in memory. When you uninitialize a sound, the reference count will be decremented, and if it hits zero, the sound will be unloaded from memory. This reference counting system is not used for streams. The engine will use a 64-bit hash of the file name when comparing file paths which means there's a small chance you might encounter a name collision. If this is an issue, you'll need to use a different name for one of the colliding file paths, or just not load from files and instead load from a data source.

The _DELAY in your code gives the miniaudio thread enough time to purge the audio data and file name out of its system which allows it to load the same sound file afresh with the new soundbank.

Another way to work around this is to load the MIDI file with a different file name (or a different MIDI file).
You can also pass the "STREAM" flag to see if that works for you. Ideally, from the midiaudio docs it seems like it should work.

Unfortunately, currently, there is no direct way in the miniaudio API that we can use to bypass the file name hashing and reference counting behavior. We have reported this to the miniaudio authors and hopefully will have a long-term solution soon.

[Steffan-68]

@Steffan-68 I've found out why the _DELAY makes your code work. So, miniaudio (our audio backend) does file-name hashing when loading audio files. The following is a direct quote from the miniaudio docs.

When loading a sound from a file path, the engine will reference count the file to prevent it from being loaded if it's already in memory. When you uninitialize a sound, the reference count will be decremented, and if it hits zero, the sound will be unloaded from memory. This reference counting system is not used for streams. The engine will use a 64-bit hash of the file name when comparing file paths which means there's a small chance you might encounter a name collision. If this is an issue, you'll need to use a different name for one of the colliding file paths, or just not load from files and instead load from a data source.

The _DELAY in your code gives the miniaudio thread enough time to purge the audio data and file name out of its system which allows it to load the same sound file afresh with the new soundbank.
Another way to work around this is to load the MIDI file with a different file name (or a different MIDI file).
You can also pass the "STREAM" flag to see if that works for you. Ideally, from the midiaudio docs it seems like it should work.
Unfortunately, currently, there is no direct way in the miniaudio API that we can use to bypass the file name hashing and reference counting behavior. We have reported this to the miniaudio authors and hopefully will have a long-term solution soon.

Thanks for your effort in finding this out.
I know that my example won't actually appear in any normal program. 
I just noticed it because I just wanted to hear how different sound sources make a sound sound very different. 
And it's crazy how big the differences are.
  
Great job what you all are doing here.