MIDI music playback has significantly degraded in 1.1.8

[none2003]

Preliminary checks

• I've checked that there aren't other open issues on the same topic.
• I've checked that this issue is reproducible on the latest snapshot build.

Platform

Windows

Describe the bug

Hi,

After updating to the latest version 1.1.8, I noticed that the sound quality of MIDI music playback has significantly degraded. The changelog mentions "Fix multiple MIDI-related issues on Windows," and the updated ZIP package of 1.1.8 has removed many MIDI-related DLL files. It seems that before version 1.1.8, the game used its own DLLs to play MIDI, while the new version 1.1.8 relies on Windows' MIDI output devices. The default "Microsoft GS Wavetable Synth" in Windows is a poor-quality MIDI device, which has led to the current situation.

Because my system usually has a third-party software wavetable, CoolSoft VirtualMIDISynth, installed, and I use CoolSoft MIDIMapper to change the default MIDI device in Windows from the Microsoft GS Wavetable Synth to VirtualMIDISynth #1, which is the CoolSoft VirtualMIDISynth software wavetable. When I directly play MIDI files using other playback software, the good sound quality of the software wavetable can be demonstrated. However, after entering fheroes2, I found that the sound quality is still the same as when using the Microsoft GS Wavetable Synth device, so I would like to ask if in the fheroes2 1.1.8 update, has the MIDI playback device been hardcoded to use the Microsoft GS Wavetable Synth?

If possible, please set the MIDI playback device of fheroes2 to use the default Windows MIDI device, so that users can improve the sound quality using third-party software synthesizers.

Save file

No need save file.

Additional info

No response

[oleg-derevenetz]

Hi @none2003, before 1.1.8 the game used fluidsynth backend to play MIDI on Windows. In 1.1.8 fluidsynth backend was replaced by TiMidity backend (the same backend that is already used on Android, WebAssembly and often on Linux on versions prior to 1.1.8 - depending on the specific Linux distribution). This backend is much faster, but it uses different set of instrument samples in comparison with fluidsynth.

You can always switch to the Windows MIDI native backend by removing the files/timidity/timidity.cfg (or the files/timidity directory entirely) and restarting the engine. Or you can replace the default set of instruments by a custom one if you have this set in a form of GUS patches (see the contents of the files/timidity/instruments directory).

[none2003]

Hi @none2003, before 1.1.8 the game used fluidsynth backend to play MIDI on Windows. In 1.1.8 fluidsynth backend was replaced by TiMidity backend (the same backend that is already used on Android, WebAssembly and often on Linux on versions prior to 1.1.8 - depending on the specific Linux distribution). This backend is much faster, but it uses different set of instrument samples in comparison with fluidsynth.

You can always switch to the Windows MIDI native backend by removing the files/timidity/timidity.cfg (or the files/timidity directory entirely) and restarting the engine. Or you can replace the default set of instruments by a custom one if you have this set in a form of GUS patches (see the contents of the files/timidity/instruments directory).

Thank you for your reply. I tested it with your method, and indeed fheroes2 uses the default MIDI output device of Windows. This change, while improving code consistency across different operating systems at the code level, has degraded the user experience for Windows users. Personally, I think fluidsynth perform much better sound quality than fluidsynth in basic setting. If possible, I hope we can continue using the fluidsynth backend solution, or have a configuration option that allows users to decide whether to use fluidsynth, TiMidity, or the default MIDI output device of the Windows system.

[oleg-derevenetz]

If possible, I hope we can continue using the fluidsynth backend solution, or have a configuration option that allows users to decide whether to use fluidsynth, TiMidity, or the default MIDI output device of the Windows system.

fheroes2 doesn't use one or the other backend directly. It uses the 3-rd party library called SDL_mixer, and this library, in turn, uses some of available backends to play the MIDI music. Those who wish can build this library in a different way (with a different MIDI backend) and replace the one in fheroes2.

As for configuring the use of a specific MIDI backend as the fheroes2 configuration option, this is highly unlikely to happen. Process of selection of the set of MIDI backends is performed at compile time, moreover, it's not the compilation parameter of fheroes2 itself, but the compilation parameter of its dependency, which we have no control over, and which should be already installed on platforms other than Windows (Linux, macOS) before the installation of fheroes2 itself, so in general case we can't control or guarantee which MIDI backends it will be built with. We have some kind of control on Windows, because we build all the dependencies here ourselves, on other platforms everything is completely different. Also, not all MIDI backends are available on every platform - e.g. there is no "native MIDI" backend on Linux.

P.S. By the way, the "sound quality" of MIDI playback depends not so much on MIDI backend as on the set of instrument samples. You can try to find and use another set of samples and, if its license allows this, you can offer a PR to replace the one that is currently used by fheroes2 (these samples are located in the files/timidity/instruments directory in a form of *.pat files).

[oleg-derevenetz]

Hi @none2003 I replaced the samples of MIDI instruments with better (but also larger) ones in #9785. It's not a fact that this change will go to the master branch because of the decent size difference. However, could you please evaluate the sound quality compared to the current set of instrument samples? Here is an instruction of how to install the game from a PR build artifact.

[none2003]

Hi @none2003 I replaced the samples of MIDI instruments with better (but also larger) ones in #9785. It's not a fact that this change will go to the master branch because of the decent size difference. However, could you please evaluate the sound quality compared to the current set of instrument samples? Here is an instruction of how to install the game from a PR build artifact.

Hi @oleg-derevenetz,

I downloaded the timidity directory from your branch and used it to replace the local files.

Based on my listening experience, I evaluated the use of CoolSoft VirtualMIDISynth, Microsoft GS Wavetable Synth, 1.1.7 (fluidsynth), 1.1.8 (official Ver), and 1.1.8 (Oleg Ver), and the results are as follows.

No.	MIDI playback method	Score
1	fheroes 2 External Music	10
2	CoolSoft VirtualMIDISynth+FluidR3_GM.sf2(141MB)	8.5
3	1.1.7 (fluidsynth)	7
4	Microsoft GS Wavetable Synth	6
5	1.1.8 (official Ver)	5
6	1.1.8 (Oleg Ver)	4.5

I saw this article mentioning that Timidity can directly use soundfont files. So I downloaded Fluid (R3) GM & GS SoundFonts and did create a timidity.cfg like:

source S:\Game\Heroes\HoMM2\files\soundfonts\fluidr3_gm.cfg

I place sf2 file at "S:\Game\Heroes\HoMM2\files\soundfonts\fluid-soundfont-3.1\FluidR3_GM.sf2", so I modify first line of fluidr3_gm.cfg as:

dir S:\Game\Heroes\HoMM2\files\soundfonts\fluid-soundfont-3.1

Somehow, this approach didn't work, and the MIDI playback still uses the default Windows MIDI output device. Maybe you can give it a try, perhaps you'll have better luck than I did.

[oleg-derevenetz]

Somehow, this approach didn't work, and the MIDI playback still uses the default Windows MIDI output device.

This is not surprising - SDL_mixer library uses its own fork of TiMidity which doesn't support SF2/SF3 format, only PAT. So in our case you can use an existing SF2 soundfont with TiMidity only if you convert SF2 into a group of PAT files. There are tools for this (e.g. unsf), but I never tried to use them. Maybe someone who interested will try.

[johny-mnemonic]

I tried converting the fheroes2.sf3 from 1.1.7 to pat files and changing the timidity.cfg to use them and I can confirm the sound quality is way better than with default timidity instrument files.
Even just renaming the timidity.cfg and using the default windows midi has better sound. But that one comes with a caveat. There are occasions when it creates weird sound artefacts.
Trouble with the pat files from the fheroes2.sf3 is that they are huge comparing with the default timidity ones (233MB vs 6,1MB), which probably explains the sound quality difference.

I also tried to use the pat files from FluidR3_GM.sf2 and even though it's better quality, it sounds very different to how original game sounded and the size is something like 1.5GB.

So from my point of view it would be best to switch to timidity version that supports sf3 or at least sf2 and use the sound fonts from 1.1.7 with timidity.
As the size is 2.5MB for sf3, 14.1MB for sf2 and 233MB for pat it makes no sense to use pat files. Even those that sounds awful are much bigger (6.1MB) than the high quality sf3...

[ihhub]

The size of files is a very big concern. We shouldn't provide a huge zip archive which could literally be in 100x times bigger than what we have right now.

We can find a solution to reduce the size of the files but it should be without damaging the quality of sounds. One way I can think of is to filter all instruments we use for the game and remove the rest. This has to be a very careful task to do.

[oleg-derevenetz]

One way I can think of is to filter all instruments we use for the game and remove the rest

This is already done both for SF3 and PAT files provided in our repo (except drums, but they are relatively small).

[ihhub]

Then I am out of ideas for now :)

[johny-mnemonic]

Yep, Oleg already done that for older version. That's why the sf3 file is so small even though it contains pretty high quality sound files.
But when I tried searching for support of SF3 in Timidity the results suggests that Timidity is no longer maintained and recommends to use FluidSynth instead🤷‍♂️
So maybe the easiest would be to just revert to what was used in 1.1.7 as it worked great and was small.