settle on default profile

[Gankra]

At least on macos, even though we're generating split-debuginfo the binary is still stuffed with a bunch of debuginfo like function names that ideally would only be in the dSYM! I need to audit the flags we're picking for their actual platform-specific semantics and maybe add post-process steps like strip.

(Current profile is inherits="release", debug=true, and split-debuginfo="packed". iirc adding the builtin strip flag to this caused at least one platform to clobber its own split-debuginfo but I need to go back and take proper notes. If that persists we can just manually invoke strip.)

[Gankra]

rebranding this issue: figure out the default profile! people were seeing blowups in their binary sizes so I just changed the default profile to "empty". Next commend will copy-paste some comments from the code about this.

[Gankra]

set debug=true? (probably not)

We want full debuginfo for good crashreporting/profiling. This ostensibly wouldn't bloat the final binary because we use split-debuginfo but the behaviour of split-debuginfo is... Not What You Expect! I removed this flag because it seemed like it was blowing stuff up

set split-debuginfo=packed? (probably not)

Ensures that all debuginfo(? don't think so now...) is pulled out of the binary and tossedinto a separate file from the final binary. This should ideally be uploaded to something like a symbol server to be fetched on demand. This is already the default on windows (.pdb) and macos (.dsym), but is rather bleeding on other platforms (.dwp) -- it requires Rust 1.65, which as of this writing in the latest stable rust release! If anyone ever makes a big deal with building final binaries with an older MSRV we may need to more intelligently select this.

It increasingly seems like dwp is kinda whack, and if the other platforms do the right thing by default I think we don't need to touch this? Also I'm pretty sure both windows and macos still default to including the unwind-tables in binaries, even panic=abort ones. This is pretty reasonable as it makes more things like profilers Just Work, but we should at least have a story for the user opting into Mega Stripped production binaries!

set codegen-units=1? (probably not)

Ok so there's an inherent tradeoff in compilers where if the compiler doeseverything in a very serial/global way, it can discover more places whereoptimizations can be done and theoretically make things faster/smallerusing all the information at its fingertips... at the cost of your buildstaking forever. Compiler optimizations generally take super-linear time,so if you let the compiler see and think about EVERYTHING your buildscan literally take days for codebases on the order of LLVM itself.

To keep compile times tractable, we generally break up the programinto "codegen units" (AKA "translation units") that get compiledindependently and then combined by the linker. This keeps the super-linearscaling under control, but prevents optimizations like inlining acrossunits. (This process is why we have things like "object files" and "rlibs",those are the intermediate artifacts fed to the linker.)

Compared to C, Rust codegen units are quite monolithic. Where each Cfile might gets its own codegen unit, Rust prefers scoping them toan entire crate. As it turns out, neither of these answers is right inall cases, and being able to tune the unit size is useful.

Large C++ codebases like Firefox have "unified" builds where they basicallyconcatenate files together to get bigger units. Rust provides theopposite: the codegen-units=N option tells rustc that it should try tobreak up a crate into at most N different units. This is done with someheuristics and constraints to try to still get the most out of each unit(i.e. try to keep functions that call eachother together for inlining).

In the --release profile, codegen-units is set to 16, which attemptsto strike a balance between The Best Binaries and Ever Finishing Compiles.In principle, tuning this down to 1 could be profitable, but LTO(see the next TODO) does most of that work for us. As such we can probablyleave this alone to keep compile times reasonable.

set lto="thin" (or "fat")? (probably "fat"!)

LTO, Link Time Optimization, is basically hijacking the step where wewould link together everything and going back to the compiler (LLVM) todo global optimizations across codegen-units (see the previous TODO).Better Binaries, Slower Build Times.

LTO can be "fat" (or "full") or "thin".

Fat LTO is the "obvious" implementation: once you're done individuallyoptimizing the LLVM bitcode (IR) for each compilation unit, you concatenateall the units and optimize it all together. Extremely serial, extremelyslow, but thorough as hell. For enormous codebases (millions of lines)this can become intractably expensive and crash the compiler.

Thin LTO is newer and more complicated: instead of unconditionally puttingeverything together, we want to optimize each unit with other "useful" unitspulled in for inlining and other analysis. This grouping is done withsimilar heuristics that rustc uses to break crates into codegen-units.This is much faster than Fat LTO and can scale to arbitrarily bigcodebases, but does produce slightly worse results.

Release builds currently default to lto=false, which, despite the name,actually still does LTO (lto="off" really turns it off)! Specifically itdoes Thin LTO but only between the codegen units for a single crate.This theoretically negates the disadvantages of codegen-units=16 whilestill getting most of the advantages! Neat!

Since most users will have codebases significantly smaller than An EntireBrowser, we can probably go all the way to default lto="fat", and theycan tune that down if it's problematic. If a user has "nightly" and "stable"builds, it might be the case that they want lto="thin" for the nightliesto keep them timely.

Aside: you may be wondering "why still have codegen units at all if using> Fat LTO" and the best answer I can give you is "doing things in parallel> at first lets you throw out a lot of junk and trim down the input before> starting the really expensive super-linear global analysis, without losing> too much of the important information". The less charitable answer is that> compiler infra is built around codegen units and so this is a pragmatic hack.>> Thin LTO of course really benefits from still having codegen units.

set panic="abort"? (probably not)

PROBABLY NOT, but here's the discussion anyway!

The default is panic="unwind", and things can be relying on unwindingfor correctness. Unwinding support bloats up the binary and can makecode run slower (because each place that can unwind is essentiallyan early-return the compiler needs to be cautious of).

panic="abort" immediately crashes the program if you panic,but does still run the panic handler, so you can get things likebacktraces/crashreports out at that point.

See RUSTFLAGS="-Cforce-unwind-tables" for the semi-orthogonal flagthat adjusts whether unwinding tables are emitted at all.

Major C++ applications like Firefox already build with this flag,the Rust ecosystem largely works fine with either.

set strip="true"? (absolutely not)

Despite the name, it doesn't "strip" debuginfo in the way the strip cli does -- it just make rustc prevent the debuginfo from existing anywhere and even makes pdbs completely disappear from the build, cursed!!!

[Gankra]

So the net conclusion of this right now is just "set lto=full", which is indeed a reasonable thing for dist to do. The only real argument I have is whether "thin" is a safer/better default with a comment saying "turn this to full, or set codegen-units=1 if you want to push harder, but results are mixed".

[Gankra]

I opted for "thin" to err more on the side of "a lot of applications are going to get built and never downloaded, and it's wasteful to opt people into super expensive builds if that's the case". The user can tune it more aggressively if they want (in the future I'll have docs suggesting reasonable tweaks).