Sudo can block/hang sometimes #856

aidanharris · 2024-08-16T01:47:54Z

Describe the bug
Sudo can block/hang sometimes

To Reproduce
Unfortunately I don't have a clear and concise reproducer I can share. I think it might be related to my environment or long command lines. When I strace the sudo process it looks like it's stuck in a loop somewhere,

sendto(3, "<38>Aug 16 01:45:45 sudo: [...] "..., 38, MSG_NOSIGNAL, NULL, 0) = 38
sendto(3, "<38>Aug 16 01:45:45 sudo: [...] "..., 38, MSG_NOSIGNAL, NULL, 0) = 38
sendto(3, "<38>Aug 16 01:45:45 sudo: [...] "..., 38, MSG_NOSIGNAL, NULL, 0) = 38

The text was updated successfully, but these errors were encountered:

squell · 2024-08-21T10:39:23Z

Thanks for the report. We should work on getting this reproducible first. Could you share a bit of your /var/log/auth.log with us containing some of sudo-rs' messages when this loop happened? Our first suspect is that logging might be related here.

(You can do so privately using [email protected] if you don't want to paste them here.)

aidanharris · 2024-08-23T05:37:20Z

I have a reproducer now:

$ sudo docker run -it --rm --pull always ubuntu bash
# apt update
Get:1 http://archive.ubuntu.com/ubuntu noble InRelease [256 kB]
Get:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
Get:3 http://archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]
Get:4 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB]
Get:5 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1808 kB]
Get:6 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [19.3 MB]
Get:7 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [12.7 kB]
Get:8 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [337 kB]
Get:9 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [354 kB]
Get:10 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [404 kB]
Get:11 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [117 kB]
Get:12 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [331 kB]
Get:13 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [518 kB]
Get:14 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [16.9 kB]
Get:15 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [449 kB]
Get:16 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [354 kB]
Get:17 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [11.5 kB]
Fetched 24.7 MB in 1s (19.8 MB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
21 packages can be upgraded. Run 'apt list --upgradable' to see them.
# apt install -y sudo git cargo build-essential libpam0g-dev
…
# useradd -s /bin/bash -m user
# printf "%s\n" "user ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
# su - user
$ git clone https://github.com/trifectatechfoundation/sudo-rs.git
Cloning into 'sudo-rs'...
remote: Enumerating objects: 13358, done.
remote: Counting objects: 100% (2403/2403), done.
remote: Compressing objects: 100% (689/689), done.
remote: Total 13358 (delta 1808), reused 2151 (delta 1681), pack-reused 10955 (from 1)
Receiving objects: 100% (13358/13358), 3.10 MiB | 16.61 MiB/s, done.
Resolving deltas: 100% (8618/8618), done.
$ cd sudo-rs
$ cargo build --release
    Updating crates.io index
  Downloaded glob v0.3.1
  Downloaded log v0.4.22
  Downloaded libc v0.2.155
  Downloaded 3 crates (806.4 KB) in 0.61s
   Compiling libc v0.2.155
   Compiling sudo-rs v0.2.3 (/home/user/sudo-rs)
   Compiling glob v0.3.1
   Compiling log v0.4.22
warning: unknown lint: `non_local_definitions`
  --> src/defaults/settings_dsl.rs:3:17
   |
3  |           #[allow(non_local_definitions)]
   |                   ^^^^^^^^^^^^^^^^^^^^^
   |
  ::: src/defaults/mod.rs:25:1
   |
25 | / defaults! {
26 | |     always_query_group_plugin = false
27 | |     always_set_home           = false
28 | |     env_reset                 = true
...  |
53 | |                                 "PYTHONINSPECT", "PYTHONUSERBASE", "RUBYLIB", "RUBYOPT", "...
54 | | }
   | |_- in this macro invocation
   |
   = note: `#[warn(unknown_lints)]` on by default
   = note: this warning originates in the macro `add_from` which comes from the expansion of the macro `defaults` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: unknown lint: `non_local_definitions`
  --> src/defaults/settings_dsl.rs:12:17
   |
12 |           #[allow(non_local_definitions)]
   |                   ^^^^^^^^^^^^^^^^^^^^^
   |
  ::: src/defaults/mod.rs:25:1
   |
25 | / defaults! {
26 | |     always_query_group_plugin = false
27 | |     always_set_home           = false
28 | |     env_reset                 = true
...  |
53 | |                                 "PYTHONINSPECT", "PYTHONUSERBASE", "RUBYLIB", "RUBYOPT", "...
54 | | }
   | |_- in this macro invocation
   |
   = note: this warning originates in the macro `add_from` which comes from the expansion of the macro `defaults` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: unknown lint: `non_local_definitions`
  --> src/defaults/settings_dsl.rs:22:17
   |
22 |           #[allow(non_local_definitions)]
   |                   ^^^^^^^^^^^^^^^^^^^^^
   |
  ::: src/defaults/mod.rs:25:1
   |
25 | / defaults! {
26 | |     always_query_group_plugin = false
27 | |     always_set_home           = false
28 | |     env_reset                 = true
...  |
53 | |                                 "PYTHONINSPECT", "PYTHONUSERBASE", "RUBYLIB", "RUBYOPT", "...
54 | | }
   | |_- in this macro invocation
   |
   = note: this warning originates in the macro `add_from` which comes from the expansion of the macro `defaults` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: unknown lint: `non_local_definitions`
  --> src/defaults/settings_dsl.rs:3:17
   |
3  |           #[allow(non_local_definitions)]
   |                   ^^^^^^^^^^^^^^^^^^^^^
   |
  ::: src/defaults/mod.rs:25:1
   |
25 | / defaults! {
26 | |     always_query_group_plugin = false
27 | |     always_set_home           = false
28 | |     env_reset                 = true
...  |
53 | |                                 "PYTHONINSPECT", "PYTHONUSERBASE", "RUBYLIB", "RUBYOPT", "...
54 | | }
   | |_- in this macro invocation
   |
   = note: this warning originates in the macro `add_from` which comes from the expansion of the macro `defaults` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: `sudo-rs` (lib) generated 10 warnings (6 duplicates)
    Finished release [optimized] target(s) in 34.08s
$ sudo cp -fv target/release/sudo /usr/bin/sudo-rs
'target/release/sudo' -> '/usr/bin/sudo-rs'
$ sudo chmod -v u+s /usr/bin/sudo-rs
mode of '/usr/bin/sudo-rs' changed from 0755 (rwxr-xr-x) to 4755 (rwsr-xr-x)
$ /usr/bin/sudo env CC=clang-18 CXX=clang++-18 FOO="........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................" id
uid=0(root) gid=0(root) groups=0(root)
$ /usr/bin/sudo-rs env CC=clang-18 CXX=clang++-18 FOO="........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................" id
# hangs here

squell · 2024-08-23T13:10:27Z

We can reproduce! Thanks for the clear test case.

squell · 2024-08-23T14:35:28Z

We wrote up a patch (as suspected the issue was the logger), you can check it out from the linked PR to verify that it fixes the issue on your side.

aidanharris · 2024-08-23T14:45:43Z

We wrote up a patch (as suspected the issue was the logger), you can check it out from the linked PR to verify that it fixes the issue on your side.

I can confirm that fixes the issue.

Closing #856 This particular code has been struck by lightning a couple of times now and is pretty hard to reason about using formal reasoning (e.g. a loop invariant like `self.cursor < LIMIT` would seen to have to hold, but it doesn't really). This PR doesn't improve that situation (i.e. the formal reasoning), but it does ensure termination. As a minor side-fix, the code could also panic on line 54 if we hit the middle of a UTF8 character in exactly the wrong position, that is fixed as well. I'll work on an implementation of this writer in a more declarative style.

squell · 2024-08-24T09:54:23Z

Note that this fix has the potential to break a syslog line in the middle of a UTF8 character (as it was a quick patch after we diagnosed the issue). An IMO better solution is in #858.

aidanharris added the bug Something isn't working label Aug 16, 2024

squell mentioned this issue Aug 23, 2024

Ensure termination of syslog writer #857

Merged

squell linked a pull request Aug 23, 2024 that will close this issue

Ensure termination of syslog writer #857

Merged

japaric closed this as completed in #857 Aug 23, 2024

squell mentioned this issue Aug 23, 2024

Refactor SysLog #858

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sudo can block/hang sometimes #856

Sudo can block/hang sometimes #856

aidanharris commented Aug 16, 2024

squell commented Aug 21, 2024

aidanharris commented Aug 23, 2024

squell commented Aug 23, 2024

squell commented Aug 23, 2024

aidanharris commented Aug 23, 2024

squell commented Aug 24, 2024

Sudo can block/hang sometimes #856

Sudo can block/hang sometimes #856

Comments

aidanharris commented Aug 16, 2024

squell commented Aug 21, 2024

aidanharris commented Aug 23, 2024

squell commented Aug 23, 2024

squell commented Aug 23, 2024

aidanharris commented Aug 23, 2024

squell commented Aug 24, 2024