Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Slightly reorganize to first present the slow command then the fast c…
…ommands
  • Loading branch information
mark-i-m committed Jun 17, 2020
commit c645df3e6ef90fba1473d23dae465b711121b390
103 changes: 74 additions & 29 deletions src/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ There are no hard hardware requirements, but building the compiler is
computationally expensive, so a beefier machine will help, and I wouldn't
recommend trying to build on a Raspberry Pi :P

- x86 and ARM are both supported (TODO: confirm)
- Recommended >=30GB of free disk space; otherwise, you will have to keep
clearing incremental caches. More space is better, the compiler is a bit of a
hog; it's a problem we are aware of.
Expand Down Expand Up @@ -125,34 +124,60 @@ In the top level of the repo:
cp config.toml.example config.toml
```

Then, edit `config.toml`. You will need to search for, uncomment, and update
Then, edit `config.toml`. You may want to search for, uncomment, and update
the following settings:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should consider having a (minimal) config.toml.dev or something that has the recommended settings.

That said, I do not recommend debug = true -- but maybe my advice is out of date. I do recommend debug-assertions=true, as well as ccache for LLVM, though I've never tried using the system LLVM, that may be better indeed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without debug = true, you won't see any debug! logging. @eddyb recommended debug = true and debuginfo = 1 a while back, maybe we could change debuginfo to default to 1 instead of 2 when debug is set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've started this hackmd to collect various suggestions that have come up: https://hackmd.io/@ux-jKBcgRTSHsH042VF3BA/rkXEozKTU/edit


- `debug = true`: enables debug symbols and `debug!` logging, takes a bit longer to compile.
- `incremental = true`: enables incremental compilation of the compiler itself,
which can significantly speed things up. This is turned off by default
because it's technically unsound; sometimes this will cause weird crashes.
Also, it can consume a lot of disk space.
because it's technically unsound; sometimes it will cause weird crashes.
Also, it can consume a lot of disk space. This has the same effect as the
`-i` or `--incremental` flags.
- `llvm-config`: enables building with system LLVM. [See this chapter][sysllvm]
for more info. This avoids building LLVM, which can take a while.

[sysllvm]: ./building/suggested.html#building-with-system-llvm

### `./x.py` Intro

`rustc` is a bootstrapping compiler because it is written in Rust. Where do you
`rustc` is a _bootstrapping_ compiler because it is written in Rust. Where do you
get the original compiler from? We use the current beta compiler
to build the compiler. Then, we use that compiler to build itself. Thus,
`rustc` has a 2-stage build.
`rustc` has a 2-stage build. You can read more about bootstrapping
[here][boot], but you don't need more to contribute.

[boot]: ./building/bootstrapping.md

We have a special tool `./x.py` that drives this process. It is used for
compiling the compiler, the standard libraries, and `rustdoc`. It is also used
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've always heard of the "standard library" as a single piece of code, but I might be wrong and this is a nitpick, really 😄
I need those nits to make a review. Because there isn't much to say otherwise...

Suggested change
compiling the compiler, the standard libraries, and `rustdoc`. It is also used
compiling the compiler, the standard library, and `rustdoc`. It is also used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly I said libraries because there is std, core, alloc, proc_macro, test, etc... not sure what else to call them.

for driving CI and building the final release artifacts.

Unfortunately, a proper 2-stage build takes a long time depending on your
hardware, but it is the only correct way to build everything (e.g. it's what
the CI and release processes use). **However, in most cases, you can get by
without a full 2-stage build**. In the following section, we give instructions
for how to do "the correct thing", but then we also give various tips to speed
things up.

### Building and Testing `rustc`

For most contributions, you only need to build stage 1, which saves a lot of time.
After updating `config.toml`, as mentioned above, you can use `./x.py`:
To do a full 2-stage build of the whole compiler, you should run this (after
updating `config.toml` as mentioned above):

```sh
./x.py build
```

In the process, this will also necessarily build the standard libraries, and it
will build `rustdoc` (which doesn't take too long).

To build and test everything:

```sh
./x.py test
```

For most contributions, you only need to build stage 1, which saves a lot of time:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like if we think this is what people should do most of the time, we should suggest it first, maybe?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I also think it'd be good to integrate the x.py check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that my own workflow at this point is basically:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Use x.py check while editing; try whenever possible to structure my changes into things that the compiler can fully enforce, and make commits as I go. This also helps with reviewing.
  • Run x.py build and x.py test --bless when necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway I don't know, I'm not sure what specific suggestion I have, but I think we could convey this information better. I was toying with the idea of a flowchart (see this mermaid live editor example) but I think it's maybe not that effective.

I guess I feel like it'd be nice to present the information in a "summary form", maybe something like this instead?

Command When to use it
x.py check Quick check to see if things compile; rust-analyzer can run this automatically for you
x.py build --stage 1 Build just the 1st stage of the compiler; this is faster than building stage 2 and usually good enough
x.py build --stage 1 --keep-stage 1 Build the 1st stage of the compiler and skips rebuilding the library; this is useful after you've done an ordinary stage1 build to skip compilation time, but it can cause weird problems. (Just do a regular build to resolve.)
x.py test --stage 1 Run the test suite using the stage1 compiler
x.py test --stage 1 --bless ...

...and then give more details below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I definitely agree that I think we should make the "default" commands the ones that most people need rather than what release does (e.g. ./x.py build would do a stage-1 incremental build), but it also seems odd to have the default command be technically unsound...

I'm not really sure what to make of this situation...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, also, I forgot to mention --bless...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for the most common commands first, with a disclaimer that they may not work and you should ask on Zulip.

Since it’s a getting started guide, I think the command that beginners will use more should be at the top.

Maybe mention that after a major rust release, you should run the sound command since now the older part was compiled with a different compiler? At least, I think I needed to do that after 1.44

Copy link
Member

@camelid camelid Jun 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe say not to worry about using bless often since you can just compare the git diff later? At first I thought it was better to edit stderr manually, until I realized I could just diff later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added Niko's table to the top of the section.


```shell
# Build the compiler (stage 1)
Expand All @@ -170,33 +195,36 @@ does not need to be rebuilt, which is usually true, which will save some time.
However, if you are changing certain parts of the compiler, this may lead to
weird errors. Feel free to ask on [zulip][z] if you are running into issues.

To run the compiler's UI test suite (the bulk of the test suite):
This runs a ton of tests and takes a long time to complete. If you are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unsure that the tests take all available CPU cores to run, and in my experience I was under the impression that the tests ran faster if I passed the --jobs flag. I am sure of nothing though, I should probably dive into the sources for the bootstrap to find out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TMK, compiletest will use all available CPUs and run as many parallel tasks as possible. I might be wrong though. I just remember that on my laptop compiletest will max out CPU.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the tests ran faster if I passed the --jobs flag

This depends on what you have codegen-units set to in config.toml. I have it set to codegen-units = 0 but I think the default might be 1.

working on `rustc`, you can usually get by with only the [UI tests][uitests]. These
test are mostly for the frontend of the compiler, so if you are working on LLVM
or codegen, this shortcut will _not_ test your changes. You can read more about the
different test suites [in this chapter][testing].

[uitests]: ./tests/adding.html#ui
[testing]: https://rustc-dev-guide.rust-lang.org/tests/intro.html

```
# UI tests
# First build
./x.py test --stage 1 src/test/ui

# Subsequent builds
./x.py test --stage 1 src/test/ui --keep-stage 1
```

This will build the compiler first, if needed.

This will be enough for most people. Notably, though, it mostly tests the
compiler frontend, not codegen or debug info. You can read more about the
different test suites [in this chapter][testing].

[testing]: https://rustc-dev-guide.rust-lang.org/tests/intro.html

If you only want to check that the compiler builds (without actually building
it) you can run the following:
While working on the compiler, it can be helpful to see if the code just
compiles (similar to `cargo check`) without actually building it. You can do
this with:

```shell
./x.py check
```

To format the code:
This command is really fast (relative to the other commands). It usually
completes in a couple of minutes on my laptop.

Finally, the CI ensures that the codebase is using consistent style. To format
the code:

```shell
# Actually format
Expand All @@ -206,15 +234,27 @@ To format the code:
./x.py fmt --check
```

You can use `RUSTC_LOG=XXX` to get debug logging. [Read more here][logging].
*Note*: we don't use stable `rustfmt`; we use a pinned version with a special
config, so this may result in different style from normal `rustfmt` if you have
format-on-save turned on. It's a good habit to run `./x.py fmt` before every
commit, as this reduces conflicts later.

On last thing: you can use `RUSTC_LOG=XXX` to get debug logging. [Read more
here][logging]. Notice the `C` in `RUSTC_LOG`. Other than that, it uses normal
[`env_logger`][envlog] syntax.

[envlog]: https://crates.io/crates/env_logger
[logging]: ./compiler-debugging.html#getting-logging-output

### Building and Testing `std`/`core`/`alloc`/`test`/`proc_macro`/etc.

To contribute to `libstd`, you don't need to build the compiler unless you are
As before, technically the proper way to build one of these libraries is to use
the stage-2 compiler, which of course requires a 2-stage build, described above
(`./x.py build`).

In practice, though, you don't need to build the compiler unless you are
planning to use a recently added nightly feature. Instead, you can just build
stage 0.
stage 0 (i.e. which basically just uses the current beta compiler).

```sh
./x.py build --stage 0 src/libstd
Expand All @@ -224,13 +264,16 @@ stage 0.
./x.py test --stage 0 src/libstd
```

(The same works for `src/liballoc`, `src/libcore`, etc.)

### Building and Testing `rustdoc`

`rustdoc` uses `rustc` internals (and, of course, the standard library), so you
will have to build the compiler and `std` once before you can build `rustdoc`.
As before, you can use `./x.py build` to do this.

The following command will build all of them. Stage 1 should be sufficient,
even though the release version will use the full 2-stage build.
However, in practice, stage 1 should be sufficient. The first time you build,
the stage-1 compiler will also be built.

```sh
# First build
Expand All @@ -240,7 +283,11 @@ even though the release version will use the full 2-stage build.
./x.py build --stage 1 --keep-stage 1 src/tools/rustdoc
```

You can also use `./x.py check` here to do a fast check build.
As with the compiler, you can do a fast check build:

```sh
./x.py check
```

Rustdoc has two types of tests: content tests and UI tests.

Expand Down Expand Up @@ -421,5 +468,3 @@ master.
- [The compiler's documentation (rustdocs)](https://doc.rust-lang.org/nightly/nightly-rustc/)
- [The Forge](https://forge.rust-lang.org/) has more documentation about various procedures.
- `#contribute`, `#compiler`, and `#rustdoc` on [Discord](https://discord.gg/rust-lang).

TODO: am I missing any?