Building an out-of-tree rustc driver against a locally built toolchain no longer works

[RalfJung]

Sometimes (rarely, but sometimes) I have to build out-of-tree Miri against a patched version of rustc. This used to work as follows:

# in the rustc folder
./x build --stage 2 library
# in the miri folder
cargo overwrite set src-rustc-stage2 # a toolchain that points to `build/host/stage2`
./miri build

However, this no longer works. I now get a bunch of errors like this

error[E0463]: can't find crate for `rustc_abi`
  --> src/lib.rs:63:1
   |
63 | extern crate rustc_abi;
   | ^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |

As a work-around, what I currently do is I make a rustc PR and trigger a try build to get a prebuilt toolchain that I can then build Miri against. Needless to say that makes the edit-compile-test cycle pretty painful.^^

I am not entirely sure what happens, but I seem to recall some discussion of not copying the rustc_* crates into the sysroot any more as that copy step was causing trouble, and instead making them available via some other means (#144252). That would likely explain the problem -- the crates aren't in the sysroot so when I build Miri out-of-tree they can't be used. Is there some way to make such out-of-tree builds possible again without re-introducing whatever problems were solved when bootstrap changed how the rustc crates are handled?

Cc @Kobzol @jieyouxu

[jieyouxu]

X-link to thread: #t-infra/bootstrap > Building miri against a locally built rustc

[RalfJung]

@jieyouxu suggested to try ./x build library --stage=3 and then use the stage2 dir, which actually seems to work -- not that I can explain why. It doesn't actually build anything new on top of a stage 2 build, so it's not a problem in terms of taking longer than it should, it's just counter-intuitive.

[RalfJung]

Closing as we gave a solution :)