Skip to content

Kernel bug on daily tests with debug kernel #2410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
EvgeniiMekhanik opened this issue Apr 28, 2025 · 1 comment
Open

Kernel bug on daily tests with debug kernel #2410

EvgeniiMekhanik opened this issue Apr 28, 2025 · 1 comment
Assignees
@EvgeniiMekhanik
Labels
bug
Milestone
0.9 - LA

Comments

@EvgeniiMekhanik
Copy link
Contributor 

Apr 26 12:40:16 192.168.122.209 [74167.415461] [tdb] Start Tempesta DB
Apr 26 12:40:16 192.168.122.209 [74167.560341] [tempesta fw] Initializing Tempesta FW kernel module...
Apr 26 12:40:18 192.168.122.209 [74168.947456] [tempesta fw] Warning: Vhost default doesn't have certificate with matching SAN/CN.
[74168.947456]     Maybe that's fine, but it's worth checking the
[74168.947456]     config - if there is no relations between the
[74168.947456]     names, then host name confusion attack is possible.
Apr 26 12:40:18 192.168.122.209 [74169.034264] [tempesta fw] Configuration processing is completed.
Apr 26 12:40:18 192.168.122.209 [74169.090613] [tdb] Opened table /opt/tempesta/db/filter0.tdb: size=16777216 rec_size=20 base=00000000a1ec113b
Apr 26 12:40:18 192.168.122.209 [74169.501233] [tdb] Opened table /opt/tempesta/db/cache0.tdb: size=268435456 rec_size=0 base=00000000d6f20111
Apr 26 12:40:18 192.168.122.209 [74169.527013] [tdb] Opened table /opt/tempesta/db/sessions0.tdb: size=16777216 rec_size=312 base=000000000bcf9318
Apr 26 12:40:18 192.168.122.209 [74169.553579] [tdb] Opened table /opt/tempesta/db/client0.tdb: size=16777216 rec_size=624 base=00000000aff1e057
Apr 26 12:40:18 192.168.122.209 [74169.614190] [tempesta fw] Open listen socket on: 0.0.0.0:443
Apr 26 12:40:18 192.168.122.209 [74169.615912] [tempesta fw] Open listen socket on: 0.0.0.0
Apr 26 12:40:18 192.168.122.209 [74169.658144] [tempesta fw] Tempesta FW is ready
Apr 26 12:40:19 192.168.122.209 [74169.951723] BUG: unable to handle page fault for address: ffff88802f889000
Apr 26 12:40:19 192.168.122.209 [74169.952373] #PF: supervisor read access in kernel mode
Apr 26 12:40:19 192.168.122.209 [74169.952836] #PF: error_code(0x0000) - not-present page
Apr 26 12:40:19 192.168.122.209 [74169.953292] PGD 175001067 P4D 175001067 PUD 175002067 PMD 17fd8c067 PTE 800fffffd0776060
Apr 26 12:40:19 192.168.122.209 [74169.954024] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC KASAN NOPTI
Apr 26 12:40:19 192.168.122.209 [74169.954527] CPU: 2 PID: 26 Comm: ksoftirqd/2 Tainted: G           OE     5.10.35.tfw-39c65d3-debug #1
Apr 26 12:40:19 192.168.122.209 [74169.955342] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Apr 26 12:40:19 192.168.122.209 [74169.956202] RIP: 0010:key_128_enc_update+0x23/0x29 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74169.956739] Code: 41 5e 41 5d 41 5c c3 c5 7a 6f 06 c5 7a 6f 6e 60 4c 01 46 18 45 31 db 4c 8b 6e 50 4d 85 ed 0f 84 c6 01 00 00 49 83 f8 10 7c 06 <c5> fa 6f 09 eb 52 4e 8d 14 19 4d 89 c4 c5 f1 ef c9 49 83 fc 08 7c
Apr 26 12:40:19 192.168.122.209 [74169.958367] RSP: 0018:ffff888100feea80 EFLAGS: 00010206
Apr 26 12:40:19 192.168.122.209 [74169.958841] RAX: 0000000000000010 RBX: ffff88802f889000 RCX: ffff88802f889000
Apr 26 12:40:19 192.168.122.209 [74169.959465] RDX: ffff88802f889000 RSI: ffff888100feed60 RDI: ffff88810d90a830
Apr 26 12:40:19 192.168.122.209 [74169.960111] RBP: ffff888100feef58 R08: 00000000000003d9 R09: 000000000000000d
Apr 26 12:40:19 192.168.122.209 [74169.960751] R10: 00000000000001a3 R11: 0000000000000000 R12: ffff888100fdb388
Apr 26 12:40:19 192.168.122.209 [74169.961375] R13: 0000000000000003 R14: ffff888100feeb38 R15: ffff888100fef2d8
Apr 26 12:40:19 192.168.122.209 [74169.962005] FS:  0000000000000000(0000) GS:ffff888147f00000(0000) knlGS:0000000000000000
Apr 26 12:40:19 192.168.122.209 [74169.962727] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 26 12:40:19 192.168.122.209 [74169.963237] CR2: ffff88802f889000 CR3: 000000010e400003 CR4: 0000000000770ee0
Apr 26 12:40:19 192.168.122.209 [74169.963867] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 26 12:40:19 192.168.122.209 [74169.964494] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 26 12:40:19 192.168.122.209 [74169.965122] PKRU: 55555554
Apr 26 12:40:19 192.168.122.209 [74169.965374] Call Trace:
Apr 26 12:40:19 192.168.122.209 [74169.965644]  ? sg_next+0x20/0x50
Apr 26 12:40:19 192.168.122.209 [74169.965952]  ? scatterwalk_ffwd+0x2e/0x150
Apr 26 12:40:19 192.168.122.209 [74169.966338]  ? gcmaes_crypt_by_sg+0x7cd/0xcf0 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74169.966856]  ? common_rfc4106_set_key+0x80/0x80 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74169.967384]  ? static_protections+0x183/0x2b0
Apr 26 12:40:19 192.168.122.209 [74169.967788]  ? create_prof_cpu_mask+0x30/0x30
Apr 26 12:40:19 192.168.122.209 [74169.968178]  ? lookup_address+0xb0/0xb0
Apr 26 12:40:19 192.168.122.209 [74169.968535]  ? ret_from_fork+0x1f/0x30
Apr 26 12:40:19 192.168.122.209 [74169.968884]  ? lookup_address_in_pgd.part.0.constprop.0+0x215/0x280
Apr 26 12:40:19 192.168.122.209 [74169.969440]  ? __change_page_attr_set_clr+0xb8d/0x14b0
Apr 26 12:40:19 192.168.122.209 [74169.969914]  ? __kasan_check_read+0x11/0x20
Apr 26 12:40:19 192.168.122.209 [74169.970314]  ? _raw_spin_trylock+0x6e/0xe0
Apr 26 12:40:19 192.168.122.209 [74169.970697]  generic_gcmaes_encrypt+0xe0/0x110 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74169.971207]  ? helper_rfc4106_decrypt+0x200/0x200 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74169.971752]  ? __dev_queue_xmit+0x6e9/0xf90
Apr 26 12:40:19 192.168.122.209 [74169.972137]  crypto_aead_encrypt+0x78/0xa0
Apr 26 12:40:19 192.168.122.209 [74169.972511]  simd_aead_encrypt+0xcf/0x120 [crypto_simd]
Apr 26 12:40:19 192.168.122.209 [74169.972992]  crypto_aead_encrypt+0x78/0xa0
Apr 26 12:40:19 192.168.122.209 [74169.973371]  ttls_encrypt+0x2a3/0x360 [tempesta_tls]
Apr 26 12:40:19 192.168.122.209 [74169.973860]  tfw_tls_encrypt+0x8f1/0xce0 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.974348]  ? tfw_tls_connection_recv+0x730/0x730 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.974886]  ? memcpy_fast+0xe/0x10 [tempesta_lib]
Apr 26 12:40:19 192.168.122.209 [74169.975340]  ? ss_skb_get_room_w_frag+0x8f/0xf0 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.975857]  ? __kasan_check_write+0x14/0x20
Apr 26 12:40:19 192.168.122.209 [74169.976243]  ? _raw_spin_lock+0x7b/0xd0
Apr 26 12:40:19 192.168.122.209 [74169.976615]  ? tfw_h2_insert_frame_header+0x722/0x9a0 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.977179]  ? skb_zerocopy_clone+0x63/0x280
Apr 26 12:40:19 192.168.122.209 [74169.977570]  ? __kasan_check_write+0x14/0x20
Apr 26 12:40:19 192.168.122.209 [74169.977986]  ? tfw_h2_stream_sched_insert_active+0x99/0xc0 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.978582]  ? __asan_loadN+0xf/0x20
Apr 26 12:40:19 192.168.122.209 [74169.978915]  ? pvclock_clocksource_read+0xe1/0x1a0
Apr 26 12:40:19 192.168.122.209 [74169.979359]  tcp_tfw_sk_write_xmit+0x148/0x1d0
Apr 26 12:40:19 192.168.122.209 [74169.979790]  ? tfw_tls_connection_recv+0x730/0x730 [tempesta_fw]
Apr 26 12:40:19 192.168.122.209 [74169.980326]  tcp_write_xmit+0x768/0x2a50
Apr 26 12:40:19 192.168.122.209 [74169.980688]  ? tcp_rearm_rto+0x220/0x220
Apr 26 12:40:19 192.168.122.209 [74169.981059]  ? __cgroup_bpf_run_filter_skb+0x397/0x720
Apr 26 12:40:19 192.168.122.209 [74169.981518]  __tcp_push_pending_frames+0x5f/0x1a0
Apr 26 12:40:19 192.168.122.209 [74169.981943]  tcp_push_pending_frames+0xbc/0x200
Apr 26 12:40:19 192.168.122.209 [74169.982352]  tcp_rcv_established+0x986/0xb40
Apr 26 12:40:19 192.168.122.209 [74169.982742]  ? tcp_data_queue+0x1d80/0x1d80
Apr 26 12:40:19 192.168.122.209 [74169.983122]  ? __kasan_check_read+0x11/0x20
Apr 26 12:40:19 192.168.122.209 [74169.983499]  tcp_v4_do_rcv+0x25f/0x360
Apr 26 12:40:19 192.168.122.209 [74169.983843]  tcp_v4_rcv+0x1600/0x1770
Apr 26 12:40:19 192.168.122.209 [74169.984231]  ? tcp_new+0x420/0x420 [nf_conntrack]
Apr 26 12:40:19 192.168.122.209 [74169.984653]  ? tcp_v4_early_demux+0x2f0/0x2f0
Apr 26 12:40:19 192.168.122.209 [74169.985053]  ip_protocol_deliver_rcu+0x46/0x2e0
Apr 26 12:40:19 192.168.122.209 [74169.985459]  ip_local_deliver_finish+0xc6/0xe0
Apr 26 12:40:19 192.168.122.209 [74169.985863]  ip_local_deliver+0x1f5/0x210
Apr 26 12:40:19 192.168.122.209 [74169.986225]  ? ip_local_deliver_finish+0xe0/0xe0
Apr 26 12:40:19 192.168.122.209 [74169.986638]  ? tcp_v4_early_demux+0x297/0x2f0
Apr 26 12:40:19 192.168.122.209 [74169.987036]  ? ip_protocol_deliver_rcu+0x2e0/0x2e0
Apr 26 12:40:19 192.168.122.209 [74169.987467]  ? ip_rcv_finish_core.constprop.0+0x17c/0x890
Apr 26 12:40:19 192.168.122.209 [74169.987950]  ip_rcv_finish+0xcf/0xf0
Apr 26 12:40:19 192.168.122.209 [74169.988274]  ip_rcv+0x16d/0x180
Apr 26 12:40:19 192.168.122.209 [74169.988559]  ? ip_local_deliver+0x210/0x210
Apr 26 12:40:19 192.168.122.209 [74169.988944]  ? ip_rcv_finish_core.constprop.0+0x890/0x890
Apr 26 12:40:19 192.168.122.209 [74169.989435]  ? virtqueue_poll+0xbb/0xd0
Apr 26 12:40:19 192.168.122.209 [74169.989787]  ? ip_local_deliver+0x210/0x210
Apr 26 12:40:19 192.168.122.209 [74169.990163]  __netif_receive_skb_one_core+0x132/0x140
Apr 26 12:40:19 192.168.122.209 [74169.990618]  ? napi_gro_receive+0x260/0x260
Apr 26 12:40:19 192.168.122.209 [74169.990997]  ? __kasan_check_write+0x14/0x20
Apr 26 12:40:19 192.168.122.209 [74169.991380]  ? _raw_spin_lock+0x7b/0xd0
Apr 26 12:40:19 192.168.122.209 [74169.991728]  __netif_receive_skb+0x26/0xb0
Apr 26 12:40:19 192.168.122.209 [74169.992109]  ? generic_exec_single+0xb1/0x170
Apr 26 12:40:19 192.168.122.209 [74169.992499]  process_backlog+0xfe/0x290
Apr 26 12:40:19 192.168.122.209 [74169.992850]  net_rx_action+0x287/0x6b0
Apr 26 12:40:19 192.168.122.209 [74169.993191]  ? napi_complete_done+0x2c0/0x2c0
Apr 26 12:40:19 192.168.122.209 [74169.993592]  ? switch_fpu_return+0x120/0x120
Apr 26 12:40:19 192.168.122.209 [74169.993981]  __do_softirq+0xfe/0x383
Apr 26 12:40:19 192.168.122.209 [74169.994311]  ? tasklet_hi_action+0x30/0x30
Apr 26 12:40:19 192.168.122.209 [74169.994684]  run_ksoftirqd+0x2b/0x40
Apr 26 12:40:19 192.168.122.209 [74169.995016]  smpboot_thread_fn+0x1ba/0x2f0
Apr 26 12:40:19 192.168.122.209 [74169.995384]  ? smpboot_register_percpu_thread+0x190/0x190
Apr 26 12:40:19 192.168.122.209 [74169.995868]  ? __kasan_check_read+0x11/0x20
Apr 26 12:40:19 192.168.122.209 [74169.996248]  ? __kthread_parkme+0x83/0xa0
Apr 26 12:40:19 192.168.122.209 [74169.996614]  ? smpboot_register_percpu_thread+0x190/0x190
Apr 26 12:40:19 192.168.122.209 [74169.997095]  kthread+0x1d9/0x200
Apr 26 12:40:19 192.168.122.209 [74169.997390]  ? __kthread_bind_mask+0x70/0x70
Apr 26 12:40:19 192.168.122.209 [74169.997779]  ret_from_fork+0x1f/0x30
Apr 26 12:40:19 192.168.122.209 [74169.998112] Modules linked in: tempesta_fw(OE) tempesta_db(OE) tempesta_tls(OE) tempesta_lib(OE) xt_mark tcp_diag inet_diag xt_nat xt_tcpudp veth ib_core sha256_ssse3 sha512_ssse3 nvme_tcp nvme_fabrics nvme_core vhost_vsock vmw_vsock_virtio_transport_common vhost vhost_iotlb vsock tls xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_algo xt_addrtype nft_compat nft_masq nft_counter nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 bridge stp llc nf_tables overlay cfg80211 intel_rapl_msr intel_rapl_common isst_if_common sch_fq_codel nfit kvm_intel kvm i2c_i801 rapl i2c_smbus binfmt_misc lpc_ich joydev input_leds mac_hid serio_raw dm_multipath netconsole msr efi_pstore nfnetlink dmi_sysfs qemu_fw_cfg ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear cirrus drm_kms_helper syscopyarea crct10dif_pclmul crc32_pclmul sysfillre
Apr 26 12:40:19 192.168.122.209 ct
Apr 26 12:40:19 192.168.122.209 [74169.998238]  sysimgblt ghash_clmulni_intel fb_sys_fops aesni_intel crypto_simd cryptd glue_helper cec psmouse ahci drm libahci virtio_net net_failover virtio_rng virtio_blk failover [last unloaded: tempesta_lib]
Apr 26 12:40:19 192.168.122.209 [74170.007354] CR2: ffff88802f889000
Apr 26 12:40:19 192.168.122.209 [74170.007691] ---[ end trace 6898ec846442ef0a ]---
Apr 26 12:40:19 192.168.122.209 [74170.008123] RIP: 0010:key_128_enc_update+0x23/0x29 [aesni_intel]
Apr 26 12:40:19 192.168.122.209 [74170.008650] Code: 41 5e 41 5d 41 5c c3 c5 7a 6f 06 c5 7a 6f 6e 60 4c 01 46 18 45 31 db 4c 8b 6e 50 4d 85 ed 0f 84 c6 01 00 00 49 83 f8 10 7c 06 <c5> fa 6f 09 eb 52 4e 8d 14 19 4d 89 c4 c5 f1 ef c9 49 83 fc 08 7c
Apr 26 12:40:19 192.168.122.209 [74170.010267] RSP: 0018:ffff888100feea80 EFLAGS: 00010206
Apr 26 12:40:19 192.168.122.209 [74170.010733] RAX: 0000000000000010 RBX: ffff88802f889000 RCX: ffff88802f889000
Apr 26 12:40:19 192.168.122.209 [74170.011353] RDX: ffff88802f889000 RSI: ffff888100feed60 RDI: ffff88810d90a830
Apr 26 12:40:19 192.168.122.209 [74170.011981] RBP: ffff888100feef58 R08: 00000000000003d9 R09: 000000000000000d
Apr 26 12:40:19 192.168.122.209 [74170.012609] R10: 00000000000001a3 R11: 0000000000000000 R12: ffff888100fdb388
Apr 26 12:40:19 192.168.122.209 [74170.013235] R13: 0000000000000003 R14: ffff888100feeb38 R15: ffff888100fef2d8
Apr 26 12:40:19 192.168.122.209 [74170.013860] FS:  0000000000000000(0000) GS:ffff888147f00000(0000) knlGS:0000000000000000
Apr 26 12:40:19 192.168.122.209 [74170.014560] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 26 12:40:19 192.168.122.209 [74170.015071] CR2: ffff88802f889000 CR3: 000000010e400003 CR4: 0000000000770ee0
Apr 26 12:40:19 192.168.122.209 [74170.015697] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 26 12:40:19 192.168.122.209 [74170.016319] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 26 12:40:19 192.168.122.209 [74170.016947] PKRU: 55555554
Apr 26 12:40:19 192.168.122.209 [74170.017199] Kernel panic - not syncing: Fatal exception in interrupt
Apr 26 12:40:19 192.168.122.209 [74170.018014] Kernel Offset: 0x34000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Apr 26 12:40:19 192.168.122.209 [74170.018995] Rebooting in 1 seconds..
Apr 26 12:40:20 192.168.122.209 [74171.011556] ACPI MEMORY or I/O RESET_REG.
Apr 26 12:40:44 192.168.122.209 [    5.688736] shpchp 0000:07:00.0: pci_hp_register failed with error -16
Apr 26 12:40:44 192.168.122.209 [    5.689360] shpchp 0000:07:00.0: Slot initialization failed
@EvgeniiMekhanik EvgeniiMekhanik added this to the 0.9 - LA milestone Apr 28, 2025
@EvgeniiMekhanik
Copy link
Contributor Author

dmesg.txt

@EvgeniiMekhanik EvgeniiMekhanik self-assigned this May 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EvgeniiMekhanik EvgeniiMekhanik

Labels
bug
Projects
None yet
Milestone
0.9 - LA
Development

No branches or pull requests
1 participant
@EvgeniiMekhanik