Service stops too early during shutdown

[ZakKemble]

When shutting down or rebooting a machine the log2ram service will sync logs before the journald service has stopped, causing missed log messages and corrupted or uncleanly shut down errors next boot -

Last few lines that journald saves when log2ram is not used (order reversed):

Jul 26 20:16:06 raspberrypi systemd-journald[103]: Journal stopped
Jul 26 20:16:06 raspberrypi systemd-shutdown[1]: Syncing filesystems and block devices.
Jul 26 20:16:05 raspberrypi kernel: systemd-shutdow: 39 output lines suppressed due to ratelimiting
Jul 26 20:16:05 raspberrypi systemd[1]: Failed to set timeout to 600s: Invalid argument
Jul 26 20:16:05 raspberrypi systemd[1]: Hardware watchdog 'Broadcom BCM2835 Watchdog timer', version 0
Jul 26 20:16:05 raspberrypi systemd[1]: Shutting down.

Last few lines with log2ram (order reversed):

Jul 26 20:21:10 raspberrypi systemd[1]: systemd-ask-password-wall.path: Succeeded.
Jul 26 20:21:10 raspberrypi systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
Jul 26 20:21:10 raspberrypi log2ram[1136]: wtmp
Jul 26 20:21:10 raspberrypi log2ram[1136]: log2ram.log
Jul 26 20:21:10 raspberrypi log2ram[1136]: sending incremental file list
Jul 26 20:21:10 raspberrypi systemd[1]: systemd-ask-password-console.path: Succeeded.
Jul 26 20:21:10 raspberrypi systemd[1]: Stopped target Local Encrypted Volumes.
Jul 26 20:21:10 raspberrypi systemd[1]: Stopping Load/Save Random Seed...
Jul 26 20:21:10 raspberrypi systemd[1]: Stopping Log2Ram...

Then dmesg shows these errors at next boot:

[ 9.927387] systemd[1]: Started Log2Ram.
[ 9.935257] systemd[1]: Starting Journal Service...
[ 10.245285] systemd[1]: Started Journal Service.
[ 10.315507] systemd-journald[566]: Received request to flush runtime journal from PID 1
[ 10.315898] systemd-journald[566]: File /var/log/journal/d44ed59bf19d41c3aa19611fee7fa666/system.journal corrupted or uncleanly shut down, renaming and replacing.
[ 11.469025] random: crng init done
[ 11.469041] random: 7 urandom warning(s) missed due to ratelimiting
[ 13.751834] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control off
[ 20.691192] systemd-journald[566]: File /var/log/journal/d44ed59bf19d41c3aa19611fee7fa666/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.

[ZakKemble]

Adding RequiredBy=systemd-journald.service to the [Install] section of the log2ram.service file seems to have done the trick. However, this means that if log2ram fails at startup then journald will also not start.
There's still the issue of the hourly disk sync which will sync the journal logs without closing them first, but I suppose this one isn't too bad since the same thing will happen from improper shutdown (power loss etc.) anyway...
journalctl --sync before disk sync could be handy too.

[azlux]

As you said: RequiredBy will stuck the start is something fail.
https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Before=

". Note that when two units with an ordering dependency between them are shut down, the inverse of the start-up order is applied. i.e. if a unit is configured with After= on another unit, the former is stopped before the latter if both are shut down. "

It's not possible to have log2ram start after journactl but stop after journactl.

[ZakKemble]

It's not possible to have log2ram start after journactl but stop after journactl.

No, I wanted log2ram to start before journald (which it does at the moment), and log2ram to stop after journald has stopped (which it currently does not do).

At the moment log2ram does this:

1. Boot up
2. log2ram starts
3. journald starts
4. Begin shutdown
5. log2ram stops
6. journald stops

This means log2ram syncs the journald logs before journald stops. When using RequiredBy 5 and 6 are swapped around so that log2ram can sync the properly closed log files.

The issue with journald failing to start if log2ram fails to start can probably be hacked around by using SuccessExitStatus=1.

[azlux]

Hi,
I've indeed make a mistake into my sentence.
But still, the doc say it's not possible, you will make it work by introducing a conflict.

[azlux]

Since the pull have been merge, Can I close this issue ?

[MegaV0lt]

I had to remove my hdd on the (debian) server because it wasn't starting as log2 ram failed...
I manualy removed log2ram. No idea what was the problem. any solution to get it fail safe?

[azlux]

@MegaV0lt An old version havn't timeout on the systemd service. It's not the case since many version. Are you up-to-date ?

[MegaV0lt]

No idea. I think since the required change. I was uninstalling log2ram - changing the server hardware and reinstalling the actual version. Then the problem occurred.

[azlux]

Found it, required don't allow the service to stop. I will rollback until I found a better solution

[azlux]

I will close this issue. The change have been rollback because making issue for many users.