`jemalloc` (`jemallocator`) C toolchain / compilation issues

[dpc]

Describe the bug

It seems like Rust tikv-jemalloc-sys crate fails on compilation of vendored C jemalloc code,
but only on NixOS, only with newer C toolchains and only when optimizations are NOT enabled.

The developers investigating it point out to odd behavior of the toolchains suggesting some misconfiguration/misbehavior. I have been experiencing it with clang, some other user with gcc, but both of us are on NixOS, and problem doesn't repro on other distros as far as we can tell. (Edit: it does repro on other distros if Nix toolchain is used e.g. from a nix develop shell).

Steps To Reproduce

Compile a Rust project using tikv-jemalloc crate in debug mode on NixOS.

Expected behavior

It should compile.

Screenshots

NA

Additional context

NA

Metadata

NA

Notify maintainers

---

Note for maintainers: Please tag this issue in your PR.

---

Add a 👍 reaction to issues you find important.

[jr1221]

It looks like systemd carries a patch to build on musl, could be related:

nixpkgs/pkgs/os-specific/linux/systemd/default.nix

Line 261 in 9a1c989

(musl-patches + "/0012-distinguish-XSI-compliant-strerror_r-from-GNU-specif.patch")

[bkchr]

#370498 this looks related.

[anoadragon453]

This issue unfortunately still occurs after #370881 (which addressed #370498) merged and hit nixos-unstable.

[anoadragon453]

I was able to work around this for now by ignoring the error when compiling:

env.CFLAGS = "-Wno-error=int-conversion";

[bkchr]

have been experiencing it with clang, some other user with gcc, but both of us are on NixOS, and problem doesn't repro on other distros as far as we can tell.

BTW, locally I could fix the issue by reverting the default gcc to gcc13.

[th7nder]

Best (in my opinion) workaround

is setting CFLAGS=-DJEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE.

    devShells.default = mkShell {
      inherit buildInputs;

      # [...]
      CFLAGS = "-DJEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE";
    };

I think it's the best one, as it's specific to this issue and doesn't break/cause other functionalities.

Other possible ways:

• @anoadragon453's env.CFLAGS = "-Wno-error=int-conversion"; - I think it's not ideal, because it disregards a warning which actually has it merits. The warning can cause an undefined behaviour - wrong codepath is used and it erases error message.
• @bkchr's downgrade to gcc13, I didn't test.

Real solution

Something is setting the _FORTIFY_SOURCE and it causes the test compilation to fail, we can:

• understand why _FORTIFY_SOURCE is there, maybe get rid of it _FORTIFY_SOURCE somehow
• force CFLAGS=-O1 even when we're running a debug build in the build.rs. (btw. there might be a bug in there as it is setting LDFLAGS=CFLAGS, which causes a lot of weird stuff when looking at the config.log of the tikv-jemallocator-build).
• maybe don't pass those CFLAGS in the jemalloc-sys/build.rs at all? Not sure whether we should compile dependency with the same flags as cargo build in the debug mode. We'd let the autoconfigure figure out flags on its own, so not passing -O0 (disabling optimization flag) and it'd compile even with _FORTIFY_SOURCE, no matter the debug or release.

_FORTIFY_SOURCE.

Looks like nix gcc wrapper,

nixpkgs/pkgs/stdenv/generic/make-derivation.nix

Line 309 in 99b0506

else remove "pie" knownHardeningFlags);

sets _FORTIFY_SOURCE by default.
When I run it on the latest nixos-unstable nixpkgs environment:

th7nder@stormheim ~/w/e/jemallocator (main) [0|1]> gcc -dM -E - < /dev/null | grep FORTIFY
#define _FORTIFY_SOURCE 3

You could disable this by specifying in your flake.nix:

hardeningDisable = [ "fortify" ];

But I think disabling it at a nix level, is not a solution, because we'd disable fortify on everything that's compiled.
It's essentially a problem with how the underlying C binary is build when building the crate.

So if we wanted to keep building debug jemalloc when building debug crate tikv-jemalloc-sys with nix-wrapped gcc, we could detect it and add -U_FORTIFY_SOURCE. It'd effectively get rid of this define, because it won't work if the crate is compiled without optimizations.
It's alternative to not passing the debug CFLAGS at all.

What do you think guys?
@BusyJay

Deeper

In the current nix:

1. Something is setting _FORTIFY_SOURCE
2. It causes a warning, because detection program for strerror is compiled with-O0 flag
3. Because -Wall, warning is an error and compilation of a test program (see point 3 in explanation) fails.
4. It does not set the flag JEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE) which causes the next warning about the conversion to int in the actual code, and again, compilation fails.

The Deepest

1. jemalloc adapts to different versions of strerror_r based on the defs of _GNU_SOURCE and JEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE.
2. I was looking through the compilation logs in nix and _GNU_SOURCE is set, so only is not set JEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE.
3. jemalloc autoconfigure sets this flags based on whether a program compiles.

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
], [
  char *buffer = (char *) malloc(100);
  char *error = strerror_r(EINVAL, buffer, 100);
  printf("%s\n", error);

4. In local non-nix compilation (basically clone https://github.com/tikv/jemallocator and cargo -vv build with GCC 11, looking at the config.log. It passes:

configure:14110: checking whether strerror_r returns char with gnu source is compilable
configure:14135: cc -o conftest -std=gnu11 -Wall -Wextra -Wsign-compare -Wundef -Wno-format-zero-length -Wpointer-arith -Wno-missing-braces -Wno-missing-field-initializers -Wno-missing-attributes -pipe -g3 -fvisibility=hidden -Wimplicit-fallthrough -O3 -funroll-loops -D_GNU_SOURCE -Werror -herror_on_warning -O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -m64 -Wall -O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -m64 -Wall -D_GNU_SOURCE -D_REENTRANT -O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -m64 -Wall conftest.c -lm  -pthread >&5
configure:14135: $? = 0
configure:14143: result: yes

5. In the nix compilation, compilation fails, but not because of the wrong header and strerror compatibility. It fails because of some warning about FORTIFY_SOURCE being enabled, while optimisation disabled.

configure:14110: checking whether strerror_r returns char with gnu source is compilable
configure:14135: gcc -o conftest -std=gnu11 -Wall -Wextra -Wsign-compare -Wundef -Wno-format-zero-length -Wpointer-arith -Wno-missing-braces -Wno-missing-field-initializers -Wno-missing-attributes -pipe -g3 -fvisibility=hidden -Wimplicit-fallthrough -O3 -funroll-loops -D_GNU_SOURCE -Werror -O0 -ffunction-sections -fdata-sections -fPIC -m64 -Wall -O0 -ffunction-sections -fdata-sections -fPIC -m64 -Wall -D_GNU_SOURCE -D_REENTRANT -O0 -ffunction-sections -fdata-sections -fPIC -m64 -Wall conftest.c -lm  -pthread >&5
In file included from /nix/store/hn4s3cq7368jkb2db02gzxhdzfa3g9zp-glibc-2.40-36-dev/include/errno.h:25,
                 from conftest.c:101:
/nix/store/hn4s3cq7368jkb2db02gzxhdzfa3g9zp-glibc-2.40-36-dev/include/features.h:422:4: error: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Werror=cpp]
  422 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
      |    ^~~~~~~
cc1: all warnings being treated as errors
configure:14135: $? = 1
configure: failed program was:
| /* confdefs.h */

[olivia-fl]

Another option for tikv-jemalloc-sys: pull rust-jemalloc-sys into buildInputs, which has a setup hook to set JEMALLOC_OVERRIDE and disable the vendored jemalloc entirely.