[Exp]: UMU support #3724

Etaash-mathamsetty · 2024-04-27T19:50:39Z

Experimental feature with UMU support. The feature is enabled by default.

~~Requires user to manually put umu-run inside $XDG_CONFIG_HOME/heroic/tools/runtimes/umu~~

based on: #3480

Use the following Checklist if you have changed something on the Backend or Frontend:

Tested the feature and it's working on a current and clean install.
Tested the main App features and they are still working on a current and clean install. (Login, Install, Play, Uninstall, Move games, etc.)
Created / Updated Tests (If necessary)
Created / Updated documentation (If necessary)

R1kaB3rN · 2024-04-29T22:34:54Z

Using this branch, I noticed the umu runtime isn't downloaded at ~/.config/heroic/tools/runtimes from the Lutris API when the checkbox Use UMU as Proton runtime is checked. Is this intended or are users supposed build it and put it in that directory themselves?

src/backend/launcher.ts

R1kaB3rN · 2024-04-29T22:44:59Z

src/backend/launcher.ts

    wait: haveToWait,
    gameSettings: settings,
+    protonVerb: 'run',


By the way, if you omit the verb, umu will default to using waitforexitandrun verb which will block users from running more than 1 game in the same prefix. However, if heroic wants to allow users to run more than 1 game, then you could default to runinprefix.

we use waitforexitandrun only for game executables to ensure we run in proper env. There is no reason for users to use same prefix for more than one game.
waitforexitandrun acts as a safeguard to avoid situations where wineserver instance runs with fsync and the game needs that disabled.

src/backend/launcher.ts

Etaash-mathamsetty · 2024-04-30T00:13:36Z

Using this branch, I noticed the umu runtime isn't downloaded at ~/.config/heroic/tools/runtimes from the Lutris API when the checkbox Use UMU as Proton runtime is checked. Is this intended or are users supposed build it and put it in that directory themselves?

that feature is not implemented yet, i simply just made it work again and haven't done anything on top of that yet

arielj · 2024-06-01T02:19:23Z

I used one of the Alien Breed games that I know UMU has a fix installing physx. This is my result:

Using Proton-GE and UMU disabled (tried with proton-ge 8.x and 9.x)
Games won't run if the prefix is not already created. I get this error:

Traceback (most recent call last):
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/proton", line 1815, in <module>
    g_session.init_session(sys.argv[1] != "runinprefix")
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/proton", line 1575, in init_session
    g_compatdata.setup_prefix()
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/proton", line 770, in setup_prefix
    with self.prefix_lock:
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/filelock.py", line 323, in __enter__
    self.acquire()
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/filelock.py", line 271, in acquire
    self._acquire()
  File "/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/filelock.py", line 384, in _acquire
    fd = os.open(self._lock_file, open_mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/ariel/Games/Heroic/Prefixes/Alien Breed Impact/pfx.lock'
Launch command: ['/home/ariel/.config/heroic/tools/proton/Proton-GE-Proton8-32/proton', 'waitforexitandrun', '/home/ariel/Games/Heroic/Alien Breed Impact/Binaries/AlienBreed-Impact.exe']
All processes exited
============= End of log =============

If I do the same in the main branch, the error doesn't happen. Something changed that makes running games with proton without the experimental feature fails to create a prefix (if the prefix is already there it works).

Using Proton 8 and the UMU runtime enabled
The prefix gets created, UMU doesn't detect the game so it doesn't install physx, then the game doesn't start cause it's missing physx I guess

Game Log:
�[34mProtonFixes[444140] INFO: Running protonfixes�[0m
�[34mProtonFixes[444140] INFO: Running checks�[0m
�[34mProtonFixes[444140] INFO: All checks successful�[0m
�[34mProtonFixes[444140] INFO: Non-steam game UNKNOWN (umu-22610)�[0m
�[34mProtonFixes[444140] INFO: Using global defaults for UNKNOWN (umu-22610)�[0m
�[34mProtonFixes[444140] INFO: Non-steam game UNKNOWN (umu-22610)�[0m
�[34mProtonFixes[444140] INFO: GOG store specified, using GOG database�[0m
�[34mProtonFixes[444140] INFO: No protonfix found for UNKNOWN (umu-22610)�[0m
Proton: /home/ariel/Games/Heroic/Alien Breed Impact/Binaries/AlienBreed-Impact.exe
Proton: Executable a unix path, launching with /unix option.
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely
Launch command: ['/home/ariel/.config/heroic/tools/runtimes/umu/umu_run.py', '/home/ariel/Games/Heroic/Alien Breed Impact/Binaries/AlienBreed-Impact.exe']
All processes exited
============= End of log =============

I wonder if this is an issue with UMU not knowing the game? because with proton 9 it shows the game title just fine (and it knows it's umu-22610). Or if we need to do something different for proton 8.

Game Log:
�[34mProtonFixes[456441] INFO: Running protonfixes�[0m
�[34mProtonFixes[456441] INFO: Running checks�[0m
�[34mProtonFixes[456441] INFO: All checks successful�[0m
�[34mProtonFixes[456441] INFO: Non-steam game Alien Breed: Impact (umu-22610)�[0m
�[34mProtonFixes[456441] INFO: GOG store specified, using GOG database�[0m
�[34mProtonFixes[456441] INFO: No global defaults found for Alien Breed: Impact (umu-22610)�[0m
�[34mProtonFixes[456441] INFO: Non-steam game Alien Breed: Impact (umu-22610)�[0m
�[34mProtonFixes[456441] INFO: GOG store specified, using GOG database�[0m
�[34mProtonFixes[456441] INFO: Using global protonfix for Alien Breed: Impact (umu-22610)�[0m
�[34mProtonFixes[456441] INFO: Checking if winetricks physx is installed�[0m
Proton: /home/ariel/Games/Heroic/Alien Breed Impact/Binaries/AlienBreed-Impact.exe
...

Using Proton 9 and the UMU runtime enabled
The prefix gets created, UMU detects the game and installs physx automatically, but then the game doesn't work (shows splash screen and then screen goes black and game closes instead of starting the initial credits). But this seems to be an issue of proton 9, because if I test with the main branch proton 9 + auto install known fixes I get the same crash.

arielj · 2024-06-01T02:22:08Z

src/backend/cache.ts

+  constructor(
+    filename: string,
+    max_value_lifespan: number | null = 60 * 6,
+    options?: { invalidateCheck?: (data: ValueType) => boolean }


we should update the JSDoc above the method with this new option so it's more clear what it's used for

at least from the usage of this in this PR I'm not sure I understand why it's needed

I made invalidateCheck to allow us to add additional checks on data. Useful for making sure we invalidate cache only when online for example. In case of umuId store, we use that to make a cache invalid only if the id wasn't found previously.

src/backend/storeManagers/gog/games.ts

src/backend/wiki_game_info/umu/utils.ts

Arias800 · 2024-06-05T12:41:11Z

Hello,

I don't know if this is a known issue or not, but UMU doesn't seem to work for sideloaded games on Linux.
I get a "Proton: Need a verb" in the log.

Everything works fine on the main Heroic branch.

arielj · 2024-06-05T22:39:15Z

Hello,

I don't know if this is a known issue or not, but UMU doesn't seem to work for sideloaded games on Linux. I get a "Proton: Need a verb" in the log.

Everything works fine on the main Heroic branch.

hmmm I cannot reproduce this, I sideloaded a game, all the process done with proton 9.5 and the UMU support enabled and it worked just fine

can you share the logs when it fails?

arielj · 2024-06-05T22:45:40Z

I still have the issue that, after installing a game (or deleting its prefix), if I launch it with Proton-GE without UMU it fails to launch (the first log in this message #3724 (comment))

src/backend/launcher.ts

Arias800 · 2024-06-06T09:00:57Z

can you share the logs when it fails?

Log for the game : cSENNjXteRS3ky69W8w9Ue-lastPlay.log

It's a game that I download from Itch.io if you want to test in the same condition : https://sirtartarus.itch.io/after-the-curtain-call

Additional information: Umu works fine for games from EGS. So I don't think it's a problem with my system.

arielj · 2024-06-06T13:09:16Z

I just tried it and it worked fine for me. what I did was:

download and unzip file in /downloads
in heroic > Add Game > enter info > select proton-ge 9.5 > select Curtains.exe > finish
game gets added
double checked the umu experimental feature is enabled in settings > advanced
click play > it prompts to install VC++ runtime
game started

I have all these turned off

	"showMangohud": true,
	"useGameMode": true,
	"useSteamRuntime": true,
	"battlEyeRuntime": true,
	"eacRuntime": true,

but I don't know why it works for you in the main branch and not in this branch

Arias800 · 2024-06-07T06:54:01Z

I found the source of my problem.
I started testing this branch when installing UMU manually was mandatory.
But now it seems to create a conflict with the current code.
Removing UMU and rebooting my system seems to fix everything.

R1kaB3rN · 2024-06-07T19:38:25Z

I think Heroic should attempt to search the user's PATH and prefer the system package if it's installed like Lutris does to avoid possible crashes. Because when using this branch and adding a DRM free game, the game will fail to launch when setting UMU as the runtime when the user already has a system umu-launcher installation that they built or packaged from their distro that is several commits ahead of the release fetched from the Lutris API.

Here's the traceback:
https://gist.github.com/R1kaB3rN/f2f521ab4d5e34029e59e0b59990030b

In the log, umu_run.py in ~/.config/heroic/tools/runtimes/umu will fail to find two files, namely the reaper and umu_version.json in the ~/.local/share/umu directory. Since commit Open-Wine-Components/umu-launcher@e518165 these two files will no longer exist as reaper is built natively into the launcher and umu does not copy its configuration file anymore.

To fix this, just search the users PATH, prefer the one installed if found and otherwise use the one provided by Heroic. Here's the relevant snippet in Lutris:

https://github.com/lutris/lutris/blob/master/lutris/util/wine/proton.py#L28

useful for use cases where we want to make sure that data won't be removed when offline etc.. in case of ULWGL we invalidate entries that are null

R1kaB3rN · 2024-07-19T21:20:07Z

src/backend/utils/compatibility_layers.ts

+export function getUmuPath() {
+  //TODO: figure out how to use searchForExecutableOnPath here
+
+  const paths = ['/app/share', '/usr/local/share', '/usr/share', '/opt']


In the next release of umu-launcher, these values need to change because the system package is now distributed as a Python module. For this case, just search PATH for umu-run instead of umu_run.py:

const paths = ['/app/bin', '/usr/local/bin', '/usr/bin']

CommandMC

Auto-installation is broken for two different reasons right now

It'd also be great to get Winetricks working, I'll see what can be done in that regard

src/backend/launcher.ts

CommandMC · 2024-07-21T15:16:45Z

src/backend/launcher.ts

@@ -302,14 +316,6 @@ async function prepareWineLaunch(
    }
  }

-  // Log warning about Proton
-  if (gameSettings.wineVersion.type === 'proton') {


We should probably still log this if UMU isn't enabled

- `isInstalled('umu')` was missing an await - `isUmuSupported` would return false if UMU wasn't installed, so the auto-install would never actually install UMU if it was missing

This seems to completely break it, not sure why yet but it's not necessary anyways

Running it with "" makes it try to run a file called "", which we don't want

Blocking changes addressed

R1kaB3rN · 2024-07-26T16:54:43Z

When running winetricks against a non existent prefix, Heroic will create a prefix with a pfx subdirectory then umu will create the prefix inside of it.

Steps to reproduce:

Enable umu in Heroic
Add a game
Select GE-Proton as the wine version
Select an empty directory (e.g., $HOME/Games/Heroic/Prefixes/foo) as the wine prefix for the game
Click the winetricks button in the game's settings
Run cd $HOME/Games/Heroic/Prefixes/foo and inspect the directory contents

Or when the user selects a umu created prefix for the game, Heroic will create a pfx subdirectory inside the prefix then umu will recreate the prefix in pfx.

Steps to reproduce:

Run GAMEID=0 WINEPREFIX=$HOME/Games/Heroic/Prefixes/foo PROTONPATH=GE-Proton umu-run "" to create a wine prefix.
Enable umu in Heroic
Add a game
Select GE-Proton as the wine version
Select a prefix created by umu as the wine prefix for the game
Click the winetricks button in the game's settings
Run cd $HOME/Games/Heroic/Prefixes/foo and inspect the directory contents

In both of these cases, can we have Heroic not create the pfx subdirectory when umu is enabled?

R1kaB3rN · 2024-07-26T17:06:45Z

Also, Heroic is currently running winetricks inside the container when getting the dll or font list which has the risk of updating the runtime.

In this case, I believe it should be safe to simply run winetricks outside the container when getting the font or dll list and only running winetricks inside the container when the user selects to install a verb. But if this is too much work, umu can have another environment variable to allow skipping runtime updates and Heroic can just set it in this case.

Etaash-mathamsetty · 2024-07-26T19:19:04Z

When running winetricks against a non existent prefix, Heroic will create a prefix with a pfx subdirectory then umu will create the prefix inside of it.

Steps to reproduce:
1. Enable umu in Heroic

2. Add a game

3. Select GE-Proton as the wine version

4. Select an empty directory (e.g., `$HOME/Games/Heroic/Prefixes/foo`) as the wine prefix for the game

5. Click the winetricks button in the game's settings

6. Run `cd $HOME/Games/Heroic/Prefixes/foo` and inspect the directory contents
Or when the user selects a umu created prefix for the game, Heroic will create a pfx subdirectory inside the prefix then umu will recreate the prefix in pfx.

Steps to reproduce:
1. Run `GAMEID=0 WINEPREFIX=$HOME/Games/Heroic/Prefixes/foo PROTONPATH=GE-Proton umu-run ""` to create a  wine prefix.

2. Enable umu in Heroic

3. Add a game

4. Select GE-Proton as the wine version

5. Select a prefix created by umu as the wine prefix for the game

6. Click the winetricks button in the game's settings

7. Run `cd $HOME/Games/Heroic/Prefixes/foo` and inspect the directory contents
In both of these cases, can we have Heroic not create the pfx subdirectory when umu is enabled?

I pushed a potential fix, could you retest ?

R1kaB3rN · 2024-07-26T21:11:34Z

I pushed a potential fix, could you retest ?

Just retested and, yes, the bug is fixed.

R1kaB3rN · 2024-07-26T21:34:40Z

Since Open-Wine-Components/umu-launcher@7e00027 updates to the runtime can be disabled.

So to prevent the runtime from being updated when using winetricks, just set UMU_RUNTIME_UPDATE=0.

R1kaB3rN · 2024-07-26T21:40:06Z

Also, because Heroic is calling umu separately for the font and dll list, it looks like it can lead to the runtime being downloaded and installed multiple times if umu is not already installed for the user.

Steps to reproduce:

Delete $HOME/.local/share/umu
Enable umu in Heroic
Add a game
Select GE-Proton as the wine version
Select an empty directory (e.g., $HOME/Games/Heroic/Prefixes/foo) as the wine prefix for the game
Click the winetricks button in the game's settings
Watch the umu runtime being downloaded multiple times in the winetricks window

Like I mentioned, I think this can be avoided by not calling winetricks for the fonts and dlls through umu. Or maybe to just get dll or font list in sequence instead async?

Etaash-mathamsetty · 2024-07-26T23:00:11Z

Since Open-Wine-Components/umu-launcher@7e00027 updates to the runtime can be disabled.

So to prevent the runtime from being updated when using winetricks, just set UMU_RUNTIME_UPDATE=0.

I don't understand why we would need to disable runtime updates?

Etaash-mathamsetty · 2024-07-26T23:00:49Z

Also, because Heroic is calling umu separately for the font and dll list, it looks like it can lead to the runtime being downloaded and installed multiple times if umu is not already installed for the user.

Steps to reproduce:

Delete $HOME/.local/share/umu

Enable umu in Heroic

Add a game

Select GE-Proton as the wine version

Select an empty directory (e.g., $HOME/Games/Heroic/Prefixes/foo) as the wine prefix for the game

Click the winetricks button in the game's settings

Watch the umu runtime being downloaded multiple times in the winetricks window

Like I mentioned, I think this can be avoided by not calling winetricks for the fonts and dlls through umu. Or maybe to just get dll or font list in sequence instead async?

Good find, I will fix it heroic side but it might be worth including some sort of lock file system in umu itself

Etaash-mathamsetty · 2024-07-27T01:57:31Z

I just checked the code, they aren't running in parrallel

R1kaB3rN · 2024-07-27T15:52:24Z

I don't understand why we would need to disable runtime updates?

The runtime is an environment to run games, so I think the runtime should only be updated during the process of launching a game, not when running winetricks verbs. I think it would be pretty annoying/confusing if the runtime gets updated when the user just wants to modify the wine prefix.

R1kaB3rN · 2024-07-28T20:47:23Z

A file lock system was merged in Open-Wine-Components/umu-launcher@a505a90

imLinguin

The feature works as expected

R1kaB3rN reviewed Apr 29, 2024

View reviewed changes

src/backend/launcher.ts Outdated Show resolved Hide resolved

R1kaB3rN reviewed Apr 29, 2024

View reviewed changes

src/backend/launcher.ts Show resolved Hide resolved

Etaash-mathamsetty force-pushed the umu branch from 74913ca to eb9c900 Compare May 31, 2024 02:35

arielj reviewed Jun 1, 2024

View reviewed changes

arielj mentioned this pull request Jun 4, 2024

umu-launcher support #3801

Closed

arielj mentioned this pull request Jun 5, 2024

[Exp]: ULWGL support #3480

Closed

4 tasks

arielj reviewed Jun 5, 2024

View reviewed changes

src/backend/launcher.ts Outdated Show resolved Hide resolved

imLinguin and others added 12 commits June 8, 2024 10:03

[Exp]: ULWGL support

9e87a99

improv: fetch ulwgl id only on Linux

ca4b5af

add fallback gameid

d59aad4

less agressive 6h caching

f76cb56

sideload support, additional checks for ulwgl support

14c4f79

use run verb in shutdownWine

d7f6272

cache: allow custom data validity checks to be defined

d08399b

useful for use cases where we want to make sure that data won't be removed when offline etc.. in case of ULWGL we invalidate entries that are null

use ulwgl-run

f972c7c

lint

aad0792

update from ulwgl to umu

217b7d8

fix lint

788d7db

automatically download umu runtime

62cf6b9

imLinguin added this to the 2.15.0 milestone Jul 12, 2024

R1kaB3rN reviewed Jul 19, 2024

View reviewed changes

CommandMC previously requested changes Jul 21, 2024

View reviewed changes

CommandMC added 4 commits July 26, 2024 00:50

Fixup auto-update

494b40b

- `isInstalled('umu')` was missing an await - `isUmuSupported` would return false if UMU wasn't installed, so the auto-install would never actually install UMU if it was missing

Make getUmuPath search for the umu-run binary on $PATH

aad2ae0

Remove PROTON_VERB env var from Winetricks env

accd1e5

This seems to completely break it, not sure why yet but it's not necessary anyways

Call umu-run with an empty executable when running Winetricks GUI

bd0ab0d

Running it with "" makes it try to run a file called "", which we don't want

Etaash-mathamsetty added 3 commits July 25, 2024 23:52

Merge branch 'main' into umu

bfbcbbb

Merge branch 'main' into umu

23c4df4

fix getWineFlagsArray

460f695

Etaash-mathamsetty added the pr:ready-for-review Feature-complete, ready for the grind! :P label Jul 26, 2024

fixes for winetricks

bd374a2

disable umu runtime updates when running winetricks

ccb36e7

R1kaB3rN mentioned this pull request Jul 27, 2024

fix: prevent downloading resources more than once Open-Wine-Components/umu-launcher#160

Merged

imLinguin approved these changes Aug 3, 2024

View reviewed changes

Etaash-mathamsetty merged commit 6105bf2 into main Aug 3, 2024
9 checks passed

Etaash-mathamsetty deleted the umu branch August 3, 2024 17:56

Heroic-Games-Launcher locked and limited conversation to collaborators Aug 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Exp]: UMU support #3724

[Exp]: UMU support #3724

Etaash-mathamsetty commented Apr 27, 2024 •

edited

Loading

R1kaB3rN commented Apr 29, 2024

R1kaB3rN Apr 29, 2024 •

edited

Loading

imLinguin Jun 18, 2024

Etaash-mathamsetty commented Apr 30, 2024 •

edited

Loading

arielj commented Jun 1, 2024

arielj Jun 1, 2024

imLinguin Jun 18, 2024

Arias800 commented Jun 5, 2024

arielj commented Jun 5, 2024 •

edited

Loading

arielj commented Jun 5, 2024

Arias800 commented Jun 6, 2024 •

edited

Loading

arielj commented Jun 6, 2024 •

edited

Loading

Arias800 commented Jun 7, 2024

R1kaB3rN commented Jun 7, 2024

R1kaB3rN Jul 19, 2024

CommandMC left a comment

CommandMC Jul 21, 2024

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

Etaash-mathamsetty commented Jul 27, 2024

R1kaB3rN commented Jul 27, 2024

R1kaB3rN commented Jul 28, 2024

imLinguin left a comment

[Exp]: UMU support #3724

[Exp]: UMU support #3724

Conversation

Etaash-mathamsetty commented Apr 27, 2024 • edited Loading

R1kaB3rN commented Apr 29, 2024

R1kaB3rN Apr 29, 2024 • edited Loading

Choose a reason for hiding this comment

imLinguin Jun 18, 2024

Choose a reason for hiding this comment

Etaash-mathamsetty commented Apr 30, 2024 • edited Loading

arielj commented Jun 1, 2024

arielj Jun 1, 2024

Choose a reason for hiding this comment

imLinguin Jun 18, 2024

Choose a reason for hiding this comment

Arias800 commented Jun 5, 2024

arielj commented Jun 5, 2024 • edited Loading

arielj commented Jun 5, 2024

Arias800 commented Jun 6, 2024 • edited Loading

arielj commented Jun 6, 2024 • edited Loading

Arias800 commented Jun 7, 2024

R1kaB3rN commented Jun 7, 2024

R1kaB3rN Jul 19, 2024

Choose a reason for hiding this comment

CommandMC left a comment

Choose a reason for hiding this comment

CommandMC Jul 21, 2024

Choose a reason for hiding this comment

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

R1kaB3rN commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

Etaash-mathamsetty commented Jul 26, 2024

Etaash-mathamsetty commented Jul 27, 2024

R1kaB3rN commented Jul 27, 2024

R1kaB3rN commented Jul 28, 2024

imLinguin left a comment

Choose a reason for hiding this comment

Etaash-mathamsetty commented Apr 27, 2024 •

edited

Loading

R1kaB3rN Apr 29, 2024 •

edited

Loading

Etaash-mathamsetty commented Apr 30, 2024 •

edited

Loading

arielj commented Jun 5, 2024 •

edited

Loading

Arias800 commented Jun 6, 2024 •

edited

Loading

arielj commented Jun 6, 2024 •

edited

Loading