cmd/go: add -debug flag (default true) to control DWARF/etc info in binary

[josharian]

Many gophers know that they can strip DWARF from their binaries with -ldflags=-w. But this is fairly cryptic, and it doesn't give the compiler the opportunity to save time and space by not generating that DWARF in the first place.

I propose we add support directly to cmd/go to say -debug=false or -dwarf=false or the like. Then cmd/go would translate that into the appropriate compiler and linker flags.

[mvdan]

I wonder if this would be a net win overall, assuming that using -ldflags=-w or not doesn't affect the compiler, so both versions can share the build cache entirely.

[mvdan]

Here's one case where it could help build speed a lot, though - single production builds from scratch, such as those that happen inside Dockerfiles, or on CI/CD machines. Those often exclude debug information, and they rarely have a warm build cache, given that they're not developer machines. For the same reason, they tend to compile a lot of packages from scratch, so even saving 2% of the compiler's work would be noticeable.

[cherrymui]

it doesn't give the compiler the opportunity to save time and space by not generating that DWARF in the first place.

The compiler has -dwarf=false flag.

[mvdan]

The compiler has -dwarf=false flag.

I think that's what @josharian meant by "Then cmd/go would translate that into the appropriate compiler and linker flags".

Also, I don't think it's quite as simple as doing go build -gcflags=all=-dwarf=false -ldflags=-w. Imagine the case where most of the library packages have already been built with the regular go build, and we just need to re-compile a few packages and re-link the entire binary. -gcflags=all=-dwarf=false will probably force recompiling all the dependencies, defeating the entire purpose of avoiding work in the first place.

I think go build -debug=false could be smarter in that way. For packages it does compile, it would know it can add flags like -dwarf=false to avoid extra work. Similarly, for link operations it would add -w. But it would not need to re-compile any package that was previously compiled with debug information, as it knows that the linker's -w will discard that anyway.

[rsc]

@mvdan:

I think go build -debug=false could be smarter in that way. For packages it does compile, it would know it can add flags like -dwarf=false to avoid extra work. Similarly, for link operations it would add -w. But it would not need to re-compile any package that was previously compiled with debug information, as it knows that the linker's -w will discard that anyway.

FWIW, I don't think we can do that. I don't know how the go command would compute accurate build IDs that would come out the same for a build in which some inputs have expected-to-be-ignored DWARF info versus a build in which no inputs do. Those are different incoming file hashes, and I don't see how to generate the outgoing file hash in a way that gets the same answer in both cases. Builds will become nondeterministic based on the content of the cache. And debugging places where DWARF info accidentally does change other compile results will become very difficult. Even if you somehow know the set of inputs to use, deciding whether you have a cache hit would now require two lookups - one to check for a -dwarf=false build and one to check for a -dwarf=true build. It's all very complicated and not obviously worth the complexity.

So if we make -debug=false mean -gcflags=all=-dwarf=false -ldflags=-w, I think it basically has to mean a full recompilation, as you noted. Whether that's worthwhile is an open question, I think. My instinct is "probably not worthwhile", but maybe the first rebuild is not such a big deal and then everything is cached.

In contrast, if -debug=false meant only -ldflags=-w, that would be a clear win and have no caching complexity - do we know what percentage of compile time is spent on DWARF, versus what percentage of link time?

If the main purpose of the flag is to be a clearer statement of intent, then only applying to the link step might be fine: -debug=false is much clearer than -ldflags=-w, and it could be made to work no matter what the underlying toolchain, making it a portability win too.

[josharian]

do we know what percentage of compile time is spent on DWARF

IIRC 1–2%

versus what percentage of link time?

I don’t know this number offhand, but I think people mainly disable DWARF for smaller binaries, not faster link (or compile) times. Faster builds is an accidental benefit.

it basically has to mean a full recompilation

I’m not so sure. ‘go test’ builds without DWARF, so in the common case in which people run tests before generating a binary (both locally and in CI/CD), it might actually be the DWARF-containing build that requires significant recompilation.

If the main purpose of the flag is to be a clearer statement of intent

FWIW, a clearer statement of intent was my primary goal in filing this proposal. But as per above, I think we can also get faster builds.

[mvdan]

FWIW, I don't think we can do that.

@jayconrod brought up the same point on the golang-tools call a few weeks ago, and it's true that I wasn't thinking of the pitfalls of forcing the build cache this way.

[rsc]

@josharian

I’m not so sure. ‘go test’ builds without DWARF, so in the common case in which people run tests before generating a binary (both locally and in CI/CD), it might actually be the DWARF-containing build that requires significant recompilation.

Looking at the code, it looks like OmitDebug is only set for the package main in a test or in "go run", not recursively down the tree. So I don't think those are doing full rebuilds necessarily. Would be nice if we were already doing full recursive non-DWARF builds, of course, but I don't think we are. (There are also implications for using the pre-cgo-ed packages like net.a without a C compiler toolchain installed if we do the recursive no-dwarf.)

It still seems like the linker is where the win mostly is.

Does anyone want to gather numbers about how much speedup there is in compiler + linker, vs how much recompilation is incurred?

[rsc]

Discussion here seems blocked on someone gathering specific numbers showing that adding this flag would be helpful in practice. Anyone interested?

[cherrymui]

I just wanted to mention that in the past we started with nearly all DWARF generation in the linker, and we have moved a good portion to the compiler, and we are planning to move more. (Not sure how this will affect the decision here.)

[josharian]

Discussion here seems blocked on someone gathering specific numbers showing that adding this flag would be helpful in practice.

I can gather numbers, but it probably won't happen particularly soon.

But to re-iterate, I think the primary value here is clarity, and performance benefits are secondary.