Record failed tests with `--record`, and rerun them with `--rerun` by jdonszelmann · Pull Request #154586 · rust-lang/rust

jdonszelmann · 2026-03-30T13:17:37Z

This adds two parameters to x test:

`--record`

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

`--rerun`

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

rustbot · 2026-03-30T13:17:41Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

rustbot · 2026-03-30T13:17:44Z

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: bootstrap
bootstrap expanded to 6 candidates
Random selection from Mark-Simulacrum, clubby789, jieyouxu

jyn514 · 2026-03-30T13:27:05Z

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

jdonszelmann · 2026-03-30T13:30:31Z

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

WaffleLapkin · 2026-03-30T15:44:27Z

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

WaffleLapkin · 2026-03-30T16:03:30Z

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

jyn514 · 2026-03-30T16:17:58Z

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

jdonszelmann · 2026-03-30T16:19:42Z

I agree with jyn here, shouldn't make such a big difference

jdonszelmann · 2026-03-30T16:20:45Z

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

jdonszelmann · 2026-03-30T16:20:50Z

with a warning of course

jyn514 · 2026-03-30T16:21:44Z

I think x test --rerun with no previous --record should behave exactly the same as x test.

jdonszelmann · 2026-03-30T17:10:17Z

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

jdonszelmann · 2026-03-30T18:53:18Z

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

clubby789 · 2026-03-31T12:18:40Z

I haven't looked at the full implementation yet, but why not always record failed tests?

jyn514 · 2026-03-31T12:32:23Z

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

clubby789 · 2026-03-31T12:36:10Z

Sorry, always *except when using --rerun 😅

jieyouxu · 2026-04-16T01:56:13Z

(Scheduled for re-review this weekend, ran out of time last weekend)

jieyouxu · 2026-04-20T01:11:38Z

See #154586 (comment)
@rustbot author

jieyouxu

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

jieyouxu · 2026-05-14T01:21:40Z

+                Ok(f) => Some(f),
+                Err(e) => {
+                    println!(
+                        "Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",


Nit: s/failutes/failures/.

jieyouxu · 2026-05-19T19:15:37Z

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

rerun failed tests

rustbot · 2026-06-04T12:25:26Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

jdonszelmann · 2026-06-04T13:11:25Z

@bors r=clubby789,jieyouxu

rust-bors · 2026-06-04T13:11:28Z

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

…uwer Rollup of 5 pull requests Successful merges: - #154586 (Record failed tests with `--record`, and rerun them with `--rerun`) - #157296 (delegation: split resolution and lowering) - #156171 (Fix a coroutine UI test which is missing `#[coroutine]`) - #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes) - #157426 (rustc-dev-guide subtree update)

Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu Record failed tests with `--record`, and rerun them with `--rerun` This adds two parameters to `x test`: ## `--record` Writes a file, by default `build/failed-tests`, but this can be overwritten with ```toml [build] record_failed_tests_path = "somepath" ``` with a list of all tests that fail that run. ## `--rerun` Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e. ``` x test tests/ui --record x test --rerun ``` Will run all failed uitests. No need to pass tests/ui to the rerun invocation. The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli. This makes it so: ``` x test tests/ui --rerun ``` will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests. Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026

rustbot assigned clubby789 Mar 30, 2026