nvidia-open-gpu-kernel-modules fails to build on Yocto 4.1. #3
Description
Hello
We want to use Nvidia dGPU Driver on Arm platform Soc，We refer to nvidia-open-gpu-kernel-modules in meta-tegra-holoscan
We modify the bb file content as follows:
# Copyright (c) 2022-2024, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
SUMMARY = "NVIDIA Linux Open GPU Kernel Modules"
LICENSE = "MIT | GPL-2.0-only"
LIC_FILES_CHKSUM = "file://COPYING;md5=1d5fa2a493e937d5a4b96e5e03b90f7c"
SRC_URI = "git://github.com/NVIDIA/open-gpu-kernel-modules.git;branch=${@d.getVar('PV').split('.')[0]};protocol=https"
SRCREV = "${AUTOREV}"
inherit module
S = "${WORKDIR}/git"
# 修改内核模模块符号标的位置，一般Yocto会自动寻找! 由于不在工作空间，所以这个需要打开。
MODULES_MODULE_SYMVERS_LOCATION = "kernel-open"
EXTRA_OEMAKE += " \
TARGET_ARCH='${TARGET_ARCH}' \
SYSSRC='${STAGING_KERNEL_DIR}' \
SYSOUT='${STAGING_KERNEL_BUILDDIR}' \
CXX='${CXX}' \
"
# The Mellanox OFED symvers file must be explicitly provided since the mlnx-ofed recipe
# doesn't use the kernel-module prefix required for automatic inclusion by module.class
# python __anonymous() {
# d.setVar('KBUILD_EXTRA_SYMBOLS', "${STAGING_INCDIR}/mlnx-ofed-kernel-dkms/Module.symvers")
# }
# 完全跳过do_split_packages
# do_package() {
# :
# }
# do_package_write_rpm() {
# :
# }
do_install:append() {
install -m 0644 -D ${S}/kernel-open/nvidia/nv-p2p.h ${D}${includedir}/nvidia/nv-p2p.h
# 删除 nvidia-modeset.ko 规避后面问题
rm -f ${D}/lib/modules/${KERNEL_VERSION}/kernel/drivers/video/nvidia-modeset.ko
}
# 内核模块自动加载的变量，模块加载时需要传递如下参数，目前不确认是否需要使用，先删除。
# KERNEL_MODULE_PROBECONF = "nvidia"
# module_conf_nvidia = "options nvidia NVreg_OpenRmEnableUnsupportedGpus=1 NVreg_DmaRemapPeerMmio=0"
# 配置了一个虚拟包名称，不用作为显示的依赖安装
RPROVIDES:${PN} += " \
kernel-module-nvidia \
kernel-module-nvidia-drm \
kernel-module-nvidia-modeset \
kernel-module-nvidia-peermem \
kernel-module-nvidia-uvm \
"
# 这里是防止报依赖被安装的，我们的系统中不需要，这个可以忽略
# This package conflicts with the Tegra kernel modules provided by nvidia-kernel-oot
# (which are typically installed with the nvidia-kernel-oot-display meta package, but
# may also be installed by accident by installing all of nvidia-kernel-oot).
# RCONFLICTS:${PN} += " \
# nv-kernel-module-nvidia \
# nv-kernel-module-nvidia-drm \
# nv-kernel-module-nvidia-modeset \
# nv-kernel-module-nvidia-peermem \
# nv-kernel-module-nvidia-uvm \
# "
We found that Yocto will cause an exception in split_kernel_module_packages in do_package because the driver module uses CPP syntax to build modules. The exception information is as follows:
DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are ['/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-core/glibc/glibc_2.36.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/binutils/binutils-cross_2.39.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/gcc/gcc-cross_12.2.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/gcc/gcc-runtime_12.2.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/gcc/libgcc_12.2.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/../meta-xilinx/meta-xilinx-core/recipes-bsp/u-boot/u-boot-tools-xlnx_2023.2.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-connectivity/openssl/openssl_3.0.8.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/bison/bison_3.8.2.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/dwarfsrcfiles/dwarfsrcfiles.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-devtools/rpm/rpm_4.18.0.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-extended/xz/xz_5.2.6.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-kernel/dtc/dtc_1.6.1.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-kernel/kmod/kmod_30.bb:do_populate_sysroot', 'virtual:native:/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/recipes-support/gmp/gmp_6.2.1.bb:do_populate_sysroot']
NOTE: Installed into sysroot: []
NOTE: Skipping as already exists in sysroot: ['glibc', 'binutils-cross-aarch64', 'gcc-cross-aarch64', 'gcc-runtime', 'libgcc', 'quilt-native', 'kern-tools-native', 'u-boot-tools-xlnx-native', 'openssl-native', 'bison-native', 'dwarfsrcfiles-native', 'patch-native', 'pkgconfig-native', 'pseudo-native', 'rpm-native', 'bc-native', 'xz-native', 'dtc-native', 'kmod-native', 'gmp-native', 'python3-setuptools-native', 'flex-native', 'python3-native', 'util-linux-native', 'gnutls-native', 'swig-native', 'linux-libc-headers', 'zlib-native', 'mpfr-native', 'libtool-native', 'gnu-config-native', 'libmpc-native', 'zstd-native', 'texinfo-dummy-native', 'readline-native', 'perl-native', 'elfutils-native', 'gettext-minimal-native', 'lua-native', 'file-native', 'sqlite3-native', 'libgcrypt-native', 'bzip2-native', 'popt-native', 'attr-native', 'python3-picobuild-native', 'python3-wheel-native', 'python3-installer-native', 'm4-native', 'libnsl2-native', 'gdbm-native', 'ncurses-native', 'libffi-native', 'libtirpc-native', 'util-linux-libuuid-native', 'libpcre2-native', 'libcap-ng-native', 'libunistring-native', 'nettle-native', 'libidn2-native', 'libpcre-native', 'make-native', 'curl-native', 'libmicrohttpd-native', 'libarchive-native', 'libgpg-error-native', 'libcap-native', 'python3-flit-core-native', 'unzip-native']
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function sstate_task_prefunc
DEBUG: Python function sstate_task_prefunc finished
DEBUG: Executing python function do_package
DEBUG: Executing python function package_convert_pr_autoinc
DEBUG: Python function package_convert_pr_autoinc finished
DEBUG: Executing python function package_prepare_pkgdata
NOTE: Installed into pkgdata-sysroot: ['gcc-runtime', 'glibc', 'libgcc', 'linux-libc-headers']
DEBUG: Python function package_prepare_pkgdata finished
DEBUG: Executing python function perform_packagecopy
DEBUG: Python function perform_packagecopy finished
DEBUG: Executing python function split_and_strip_files
DEBUG: runstrip: ['aarch64-nova-linux-strip', '--strip-debug', '--remove-section=.comment', '--remove-section=.note', '--preserve-dates', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/nvidia.ko']
DEBUG: runstrip: ['aarch64-nova-linux-strip', '--strip-debug', '--remove-section=.comment', '--remove-section=.note', '--preserve-dates', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/nvidia-uvm.ko']
DEBUG: runstrip: ['aarch64-nova-linux-strip', '--strip-debug', '--remove-section=.comment', '--remove-section=.note', '--preserve-dates', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/nvidia-modeset.ko']
DEBUG: runstrip: ['aarch64-nova-linux-strip', '--strip-debug', '--remove-section=.comment', '--remove-section=.note', '--preserve-dates', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/nvidia-peermem.ko']
DEBUG: runstrip: ['aarch64-nova-linux-strip', '--strip-debug', '--remove-section=.comment', '--remove-section=.note', '--preserve-dates', '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/nvidia-drm.ko']
DEBUG: Python function split_and_strip_files finished
DEBUG: Executing python function fixup_perms
DEBUG: Python function fixup_perms finished
DEBUG: Executing python function split_kernel_module_packages
aarch64-nova-linux-objcopy: /home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/.debug/nvidia-modeset.ko: no group info for section '.text._ZN11DisplayPort6AuxBus14fecTransactionEPhPPtj'
aarch64-nova-linux-objcopy: /home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/.debug/nvidia-modeset.ko: no group info for section '.text._ZN11DisplayPort6AuxBus14fecTransactionEPhPPtj'
aarch64-nova-linux-objcopy: /home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/.debug/nvidia-modeset.ko: file format not recognized
ERROR: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:split_kernel_module_packages(d)
0003:
File: '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/classes-recipe/kernel-module-split.bbclass', lineno: 184, function: split_kernel_module_packages
0180: module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
0181: module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
0182: module_pattern = module_pattern_prefix + kernel_package_name + '-module-%s' + module_pattern_suffix
0183:
*** 0184: modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', file_regex=module_regex, output_pattern=module_pattern, description='%s kernel module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name, kernel_version))
0185: if modules:
0186: d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
0187:
0188: # If modules-load.d and modprobe.d are empty at this point, remove them to
File: '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/classes-global/package.bbclass', lineno: 240, function: do_split_packages
0236: d.setVar('pkg_postinst:' + pkg, postinst)
0237: if postrm:
0238: d.setVar('pkg_postrm:' + pkg, postrm)
0239: if callable(hook):
*** 0240: hook(f, pkg, file_regex, output_pattern, m.group(1))
0241:
0242: d.setVar('PACKAGES', ' '.join(packages))
0243: return list(split_packages)
0244:
File: '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/classes-recipe/kernel-module-split.bbclass', lineno: 92, function: frob_metadata
0088: vals[m.group(1)] = m.group(2)
0089: return vals
0090:
0091: def frob_metadata(file, pkg, pattern, format, basename):
*** 0092: vals = extract_modinfo(file)
0093:
0094: dvar = d.getVar('PKGD')
0095:
0096: # If autoloading is requested, output /etc/modules-load.d/<name>.conf and append
File: '/home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/../../sources/poky/meta/classes-recipe/kernel-module-split.bbclass', lineno: 74, function: extract_modinfo
0070: raise msg
0071: cmd = "%sobjcopy -j .modinfo -O binary %s %s" % (d.getVar("HOST_PREFIX") or "", tmpkofile, tmpfile)
0072: else:
0073: cmd = "%sobjcopy -j .modinfo -O binary %s %s" % (d.getVar("HOST_PREFIX") or "", file, tmpfile)
*** 0074: subprocess.check_call(cmd, shell=True)
0075: # errors='replace': Some old kernel versions contain invalid utf-8 characters in mod descriptions (like 0xf6, 'ö')
0076: f = open(tmpfile, errors='replace')
0077: l = f.read().split("\000")
0078: f.close()
File: '/home/common/xilinx/petalinux/2024.1/sysroots/x86_64-petalinux-linux/usr/lib/python3.10/subprocess.py', lineno: 369, function: check_call
0365: if retcode:
0366: cmd = kwargs.get("args")
0367: if cmd is None:
0368: cmd = popenargs[0]
*** 0369: raise CalledProcessError(retcode, cmd)
0370: return 0
0371:
0372:
0373:def check_output(*popenargs, timeout=None, **kwargs):
Exception: subprocess.CalledProcessError: Command 'aarch64-nova-linux-objcopy -j .modinfo -O binary /home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/package/lib/modules/6.1.60-xilinx-v2023.2/kernel/drivers/video/.debug/nvidia-modeset.ko /home/gaojie/workspace/xilinx/versal_vck190/xilinx-yocto/build_nova/vm1402_2110/tmp/work/novastar_vm1402_rxcard-nova-linux/nvidia-open-gpu-kernel-modules/550.107.02-r0/tmppd63p8sz' returned non-zero exit status 1.
DEBUG: Python function split_kernel_module_packages finished
DEBUG: Python function do_package finished
Do you have similar issues when building? Is there any good solution?
My solution was to delete nvidia-modeset.ko
I couldn't install nvidia-modeset.ko and nvidia-drm.ko due to dependencies