Add possibility to add kernel drivers to initramdisk

[dkubek]

Before this change there was no possibility for developers to specify what kernel drivers should be included in the upgrade/target initramfs. This includes third-party drivers, which are necessary for system upgrades in some spefic use cases.

Changes include:

• A new model KernelModule (analogous to the DracutModule model) has been created, to handle kernel drivers.
• Added an include_kernel_drivers field in the UpgradeInitramfsTasks model to handle a list of these drivers.
• Data in the include_kernel_drivers field is processed correctly to detect conflicting paths.
• Modified the generate-iniram.sh script to accept and process the new data
• Added checks for kernel drivers, in the CheckInitramfsTasks actor.
• Updated the unit-tests accordingly.

Jira ref.: OAMG-4512
BZs: 2175229

Template example:

from leapp.actors import Actor
from leapp.models import (
    KernelModule,
    RpmTransactionTasks,
    TargetInitramfsTasks,
    TargetUserSpaceUpgradeTasks,
    UpgradeInitramfsTasks
)
from leapp.tags import ChecksPhaseTag, IPUWorkflowTag


class AddKernelDriverMYDRIVER(Actor):
    """
    Install the <mydriver> driver during the upgrade

    Install the <mydriver> kernel module in the upgrade & target initramfs.
    In this scenario it requires the package with the module is installed
    on the target system and inside the target userspace container.

    In case of the scenario when the module should be copied from a directory
    existing on the host system, specify the path from where it should
    be copied/installed instead, e.g.:
        KernelModule(name='<mydriver>', module_path='/path/to/the/module')
    """

    name = 'add_kernel_driver_<mydriver>'
    consumes = ()
    produces = (RpmTransactionTasks, TargetInitramfsTasks, TargetUserSpaceUpgradeTasks, UpgradeInitramfsTasks)
    tags = (IPUWorkflowTag, ChecksPhaseTag)

    def process(self):
        # IMPORTANT: For these package installations the (custom) repository
        # must be enabled! Ideal solution is to define the repos inside the
        # /etc/leapp/files/leapp_upgrade_repositories.repo file or using the
        # --enablerepo option when running leapp.
        # this will create task to install the package with desired driver
        # into the target userspace container
        # <pkg-with-driver> - could be e.g. kmod-<mydriver>
        self.produce(TargetUserSpaceUpgradeTasks(install_rpms=['<pkg-with-driver>']))

        # and we want the package to be installed also during the upgrade,
        # so the driver can be used also on the upgraded system
        self.produce(RpmTransactionTasks(to_install=['<pkg-with-driver>']))

        # this will require installation of the module in the upgrade and the
        # target initramfs
        k_module = KernelModule(name='<mydriver>')
        self.produce(UpgradeInitramfsTasks(include_kernel_modules=[k_module]))
        self.produce(TargetInitramfsTasks(include_kernel_modules=[k_module]))

[github-actions]

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

• review please @oamg/developers to notify leapp developers of the review request
• /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build. If you need a different version of leapp from PR#42, use /packit test oamg/leapp#42

To launch regression testing public members of oamg organization can leave the following comment:

• /rerun to schedule basic regression tests using this pr build and latest upstream leapp build as artifacts
• /rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
• /rerun-sst to schedule sst tests using this pr build and latest upstream leapp build as artifacts
• /rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

Fixed

[pirat89]

be aware of trailing whitespaces :)

[pirat89]

added template example for the custom actor installing the third-party driver to the description

[MichalHe]

I've tested the patch using my custom kernel module. The test used a simple custom actor that requested the test kernel module to be present in the upgrade initramfs and target initramfs.

Pre reboot, upgrade initramfs preprated

[root@leapp-20230707134128 ~]# lsinitrd /boot/initramfs-upgrade.x86_64.img | grep test
Arguments:  -v -v -v -v --force --conf '/dev/null' --confdir '/var/empty' --install 'systemd-nspawn' --add 'sys-upgrade-redhat' --add 'sys-upgrade' --add 'sys-upgrade-redhat' --add 'sys-upgrade' --add-drivers 'test' --nomdadmconf --nolvmconf --no-hostonly --kver '5.14.0-332.el9.x86_64' --kernel-image 'vmlinuz-upgrade.x86_64'
-rw-r--r--   1 root     root         6080 May 31 14:33 usr/lib/modules/5.14.0-332.el9.x86_64/extra/leapp/test.ko
-rw-r--r--   1 root     root         5168 May 31 14:33 usr/lib/modules/5.14.0-332.el9.x86_64/kernel/crypto/async_tx/raid6test.ko.xz

In the initramfs

cmdline:/lib/modules/5.14.0-332.el9.x86_64/extra/leapp# modprobe test
[   32.881143] test: loading out-of-tree module taints kernel.
[   32.881195] test: module verification failed: signature and/or required key missing - tainting kernel
[   32.887405] Hello World
[   32.887405]  This is a test

The module is not being added to target initramfs

However, the kernel module is not being added to the target initramfs - the include_kernel_modules field that is added to the TargetInitramfsTasks model through inheritance is not used anywhere - the target_initramfs_generator is not modified in this PR. Therefore, even if a message was emitted:

[root@leapp-20230707134128 ~]# ./leapp-inspector messages --type TargetInitramfsTasks
######################################################################
                          PRODUCED MESSAGES
######################################################################
Stamp: 2023-07-10T11:22:27.891450Z
Actor: persistentnetnamesconfig
Phase: Applications
Type: TargetInitramfsTasks
Message_data:
{
    "include_dracut_modules": [],
    "include_files": [],
    "include_kernel_modules": []
}
----------------------------------------------------------------------
Stamp: 2023-07-10T08:12:34.486155Z
Actor: add_my_kernel_driver
Phase: Checks
Type: TargetInitramfsTasks
Message_data:
{
    "include_dracut_modules": [],
    "include_files": [],
    "include_kernel_modules": [
        {
            "module_path": "/root/test.ko",
            "name": "test"
        }
    ]
}

The target_initramfs_generator ended with the following message in the logs:

Jul 10 11:22:28 localhost upgrade[61186]: 2023-07-10 11:22:28.958 DEBUG    PID: 1085 leapp.workflow.Finalization.target_initramfs_generator: No additional files or modules required to add int
o the target initramfs.

[matejmatuska]

/packit copr-build

[MichalHe]

I've retested the patch again in the same manner:

In upgrade initramfs

cmdline:/# insmod /lib/modules/5.14.0-337.el9.x86_64/extra/leapp/test.ko
[   28.164590] test: loading out-of-tree module taints kernel.
[   28.164654] test: module verification failed: signature and/or required key missing - tainting kernel
cmdline:/# [   28.172835] Hello World
[   28.172835]  This is a test

In target initramfs

cmdline:/# insmod /lib/modules/5.14.0-337.el9.x86_64/extra/leapp/test.ko
cmdline:# dmesg
....
[   55.073049] Hello World
                This is a test

Target Initramfs generator - logs

[  466.235900] upgrade[68485]: 2023-07-14 13:57:03.310 INFO     PID: 1 leapp.workflow.Finalization: Executing actor target_initramfs_generator
[  466.270903] upgrade[69578]: 2023-07-14 13:57:03.344 DEBUG    PID: 1085 leapp.workflow.Finalization.target_initramfs_generator: Copying kernel module "test" to "/lib/modules/5.14.0-337.el9.x86_64/extra/leapp/test.ko".
[  466.277399] upgrade[69578]: 2023-07-14 13:57:03.352 DEBUG    PID: 1085 leapp.workflow.Finalization.target_initramfs_generator: External command has started: ['depmod', '5.14.0-337.el9.x86_64', '-a']
[  471.014799] upgrade[69578]: 2023-07-14 13:57:08.89  DEBUG    PID: 1085 leapp.workflow.Finalization.target_initramfs_generator: External command has finished: ['depmod', '5.14.0-337.el9.x86_64', '-a']
[  471.017346] upgrade[69578]: 2023-07-14 13:57:08.90  DEBUG    PID: 1085 leapp.workflow.Finalization.target_initramfs_generator: External command has started: ['dracut', '-f', '--kver', '5.14.0-337.el9.x86_64', '--add-drivers', 'test']

lsinitrd target_initramfs

[root@leapp-20230622171431 vagrant]# lsinitrd /boot/initramfs-5.14.0-337.el9.x86_64.img | grep test
Arguments:  -f --kver '5.14.0-337.el9.x86_64' --add-drivers 'test'
-rw-r--r--   1 root     root         6080 May 31 14:33 usr/lib/modules/5.14.0-337.el9.x86_64/extra/leapp/test.ko
-rw-r--r--   1 root     root         5180 May 31 14:33 usr/lib/modules/5.14.0-337.el9.x86_64/kernel/crypto/async_tx/raid6test.ko.xz

lsinitrd upgrade_initramfs

[root@leapp-20230622171431 ~]# lsinitrd /boot/initramfs-upgrade.x86_64.img | grep test
Arguments:  -v -v -v -v --force --conf '/dev/null' --confdir '/var/empty' --install 'systemd-nspawn' --add 'sys-upgrade-redhat' --add 'sys-upgrade' --add 'sys-upgrade-redhat' --add 'sys-upgrade' --add-drivers 'test' --nomdadmconf --nolvmconf --no-hostonly --kver '5.14.0-337.el9.x86_64' --kernel-image 'vmlinuz-upgrade.x86_64'
-rw-r--r--   1 root     root         6080 May 31 14:33 usr/lib/modules/5.14.0-337.el9.x86_64/extra/leapp/test.ko
-rw-r--r--   1 root     root         5180 May 31 14:33 usr/lib/modules/5.14.0-337.el9.x86_64/kernel/crypto/async_tx/raid6test.ko.xz

[MichalHe]

I've tested the patch and it works. LGTM

[MichalHe]

@dkubek I will merge this once the integration tests are done running.

[MichalHe]

/rerun

[github-actions]

Copr build succeeded: https://copr.fedorainfracloud.org/coprs/build/6177803

[github-actions]

Testing Farm request for RHEL-8.6-rhui/6177803 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

[github-actions]

Testing Farm request for RHEL-7.9-rhui/6177803 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

[MichalHe]

I've performed manually a successful 8>9 upgrade using internal vagrant box.

[fernflower]

lgtm. Given that we need the builds today and manual testing + e2e testing has been done successfully, I'd suggest we merge it.