make it possible to use stage0 libtest on compiletest

[onur-ozkan]

With #119899, building the library tree will require a stage 1 compiler. This is because compiletest is defined as a ToolStd (since #68019) in order to use the in-tree library. As a result, #119899 makes certain development workflows more difficult as changes on the compiler tree will now require recompiling compiletest each time.

This PR allows switching ToolStd to ToolBootstrap with a simple boolean option in bootstrap.toml to allow compiletest to use the stage 0 libtest instead.

The changes under src/ci are clearly intended to make sure that compiletest doesn't break during future bootstrap beta bumps.

[rustbot]

r? @Kobzol

rustbot has assigned @Kobzol.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

[Kobzol]

I would set this to false in the dist profile, in order to be enabled globally on CI, and then have one runner that sets it to true, to test that compiletest works also with stage0, if we want to gate on this.

[onur-ozkan]

Not sure if using it in just one runner gives enough coverage.

Here's what I did: compiletest-use-stage0-libtest is set to false in the dist profile and run.sh script overrides it to true unless $DEPLOY or $DEPLOY_ALT is set.

[Kobzol]

I'm pretty sure that most dist builders don't even build compiletest, as they don't run any tests. So if you want to have partial coverage of both true/false, we should do the split inside the test builders, not between test and dist, IMO.

[Kobzol]

That being said, I'm not sure if we need that much coverage of the true value? It's just a hack to make local rebuilds faster. I would put it maybe into one runner per OS.

Although, if we have both true and false on CI as blocking, can't that break beta bumps? 🤔

[onur-ozkan]

That being said, I'm not sure if we need that much coverage of the true value? It's just a hack to make local rebuilds faster. I would put it maybe into one runner per OS.

Yeah, probably... I can do that for Linux, Mac and Windows (let me know if I am missing any other) as those are the most popular ones used by devs.

Although, if we have both true and false on CI as blocking, can't that break beta bumps? 🤔

compiletest is always built using the in-tree version of libtest. So if something breaks libtest, it would fail during development already; it wouldn't make it to beta without being caught. I guess that could happen if libtest is changed before bumping stage0 compiler, which will require cfg(bootstrap) thing there, I am not sure if this is acceptable. cc @jieyouxu

[jieyouxu]

Hm yeah, I think there's a window where indeed cfg(bootstrap) might be needed, a library/testing-devex contributor who wants to modify libtest may need to cfg(bootstrap)-gate bit.

[jieyouxu]

That being said, following #139317, the libtest usages in compiletest should be mostly isolated in one single module. We could add a doc comment on that about possibly needing cfg(bootstrap)-gating libtest changes.

Personally, I think this approach (possibility of compiletest using stage0 libtest for local dev) is reasonable, and I can't really think of a better solution.

[rustbot]

This PR modifies src/bootstrap/defaults.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.