Skip to content

Implement edition hygiene for keywords#50307

Merged
bors merged 9 commits into
rust-lang:masterfrom
petrochenkov:keyhyg2
May 18, 2018
Merged

Implement edition hygiene for keywords#50307
bors merged 9 commits into
rust-lang:masterfrom
petrochenkov:keyhyg2

Conversation

@petrochenkov

@petrochenkov petrochenkov commented Apr 29, 2018

Copy link
Copy Markdown
Contributor

Determine "keywordness" of an identifier in its hygienic context.
cc #49611

I've resurrected proc as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: proc is removed again.

@rust-highfive

Copy link
Copy Markdown
Contributor

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 29, 2018
@petrochenkov

Copy link
Copy Markdown
Contributor Author

r? @nikomatsakis
cc @Manishearth

@Manishearth

Copy link
Copy Markdown
Member

I've resurrected proc as an Edition-2015-only keyword for testing purposes, but it should probably be buried again.

We don't need keywords that are keywords on old editions but not new, we need it the other way around 😄

@Manishearth

Copy link
Copy Markdown
Member

Haven't really gone through this completely, but could you include some form of the changes from 0eabb1b so that it's less fragile? (and liberally sprinkle comments that link things together)

@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 30, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I continue to be nervous about the idea that macros can observe the different here -- what happens if I have a macro from a 2018 Edition crate that accepts r#async? Can a 2015 crate pass in async and have it work?

Put another way:

If I have a macro that accepts a async in its input, can I transition my crate to 2018 Edition without a breaking change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With scheme implemented in this PR keyword hygiene and raw identifiers are entirely orthogonal.
(So this PR doesn't make changes to any interactions between raw identifiers and macros.)

If macro accepts async literally (not as $i: ident or something) in its left hand side, then it will not appear in the macro output, so we are safe even without keyword hygiene (as consumes_async!(async) shows, yeah).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the answers to my question are here, maybe?

@nikomatsakis

Copy link
Copy Markdown
Contributor

This looks pretty good so far! (modulo my questions about the desired behavior of raw idents in macros)

@petrochenkov

Copy link
Copy Markdown
Contributor Author

Update: I'll return to this once I'm done with proc macro 1.2 API review, likely this weekend.

Comment thread src/libsyntax_pos/symbol.rs Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we keeping proc reserved on 2015? We'll surely never use it for anything on the 2015 epoch.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM saw the comment explaining this

@bors

bors commented May 7, 2018

Copy link
Copy Markdown
Collaborator

☔ The latest upstream changes (presumably #50454) made this pull request unmergeable. Please resolve the merge conflicts.

@Manishearth

Copy link
Copy Markdown
Member

any updates

@petrochenkov

petrochenkov commented May 10, 2018

Copy link
Copy Markdown
Contributor Author

@Manishearth
I still need to do the plumbing mentioned in TODO (thread edition info from crate metadata into libsyntax/ext/expand.rs where it's needed).
I'll work on this tomorrow.

@Manishearth

Manishearth commented May 10, 2018 via email

Copy link
Copy Markdown
Member

@petrochenkov

Copy link
Copy Markdown
Contributor Author

PR updated, all tests are now fixed.
It's too late today, I'll cleanup stuff / remove proc tomorrow and it should be ready.

@rust-highfive

Copy link
Copy Markdown
Contributor

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[01:00:15] ............test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[01:00:45] ........................................................................................
[01:01:06] .....................................................................................ii.............
[01:02:04] .................................................i..................................................
[01:02:09] ..i.ii.........test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[01:03:02] ..........iiiiiii...................................................................................
[01:03:24] ....................................................................................................
[01:03:44] ....................................................................................................
[01:04:05] ...................................................................................
---
travis_time:start:test_run-pass-fulldeps
Check compiletest suite=run-pass-fulldeps mode=run-pass (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:08:48] 
[01:08:48] running 88 tests
[01:10:38] ..........................................FFF...........................test [run-pass] run-pass-fulldeps/myriad-closures.rs has been running for over 60 seconds
[01:14:11] failures:
[01:14:11] 
[01:14:11] ---- [run-pass] run-pass-fulldeps/plugin-args-1.rs stdout ----
[01:14:11]  
[01:14:11]  
[01:14:11] error: auxiliary build of "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" failed to compile: 
[01:14:11] status: exit code: 101
[01:14:11] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-1.stage2-x86_64-unknown-linux-gnu.aux" "-Crpath" "-O" "-Zunstable-options" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-1.stage2-x86_64-unknown-linux-gnu.aux"
[01:14:11] ------------------------------------------
[01:14:11] 
[01:14:11] ------------------------------------------
[01:14:11] stderr:
---
[01:14:11]    |     ^^^^^^^^^^^^^^
[01:14:11]    |
[01:14:11]    = note: #[warn(unused_imports)] on by default
[01:14:11] 
[01:14:11] error[E0063]: missing field `edition` in initializer of `syntax::ext::base::SyntaxExtension`
[01:14:11]   --> /checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs:51:9
[01:14:11] 51 |         NormalTT {
[01:14:11] 51 |         NormalTT {
[01:14:11]    |         ^^^^^^^^ missing `edition`
[01:14:11] error: aborting due to previous error
[01:14:11] 
[01:14:11] For more information about this error, try `rustc --explain E0063`.
[01:14:11] 
[01:14:11] 
[01:14:11] ------------------------------------------
[01:14:11] 
[01:14:11] thread '[run-pass] run-pass-fulldeps/plugin-args-1.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3033:9
[01:14:11] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[01:14:11] 
[01:14:11] ---- [run-pass] run-pass-fulldeps/plugin-args-2.rs stdout ----
[01:14:11]  
[01:14:11] error: auxiliary build of "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" failed to compile: 
[01:14:11] status: exit code: 101
[01:14:11] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-2.stage2-x86_64-unknown-linux-gnu.aux" "-Crpath" "-O" "-Zunstable-options" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-2.stage2-x86_64-unknown-linux-gnu.aux"
[01:14:11] ------------------------------------------
[01:14:11] 
[01:14:11] ------------------------------------------
[01:14:11] stderr:
---
[01:14:11]    |     ^^^^^^^^^^^^^^
[01:14:11]    |
[01:14:11]    = note: #[warn(unused_imports)] on by default
[01:14:11] 
[01:14:11] error[E0063]: missing field `edition` in initializer of `syntax::ext::base::SyntaxExtension`
[01:14:11]   --> /checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs:51:9
[01:14:11] 51 |         NormalTT {
[01:14:11] 51 |         NormalTT {
[01:14:11]    |         ^^^^^^^^ missing `edition`
[01:14:11] error: aborting due to previous error
[01:14:11] 
[01:14:11] For more information about this error, try `rustc --explain E0063`.
[01:14:11] 
[01:14:11] 
[01:14:11] ------------------------------------------
[01:14:11] 
[01:14:11] thread '[run-pass] run-pass-fulldeps/plugin-args-2.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3033:9
[01:14:11] 
[01:14:11] ---- [run-pass] run-pass-fulldeps/plugin-args-3.rs stdout ----
[01:14:11]  
[01:14:11] error: auxiliary build of "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" failed to compile: 
[01:14:11] thread 'main' panicked at 'Some tests failed', tools/compiletest/src/main.rs:492:22
[01:14:11] status: exit code: 101
[01:14:11] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-3.stage2-x86_64-unknown-linux-gnu.aux" "-Crpath" "-O" "-Zunstable-options" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/plugin-args-3.stage2-x86_64-unknown-linux-gnu.aux"
[01:14:11] ------------------------------------------
[01:14:11] 
[01:14:11] ------------------------------------------
[01:14:11] stderr:
---
[01:14:11]    |     ^^^^^^^^^^^^^^
[01:14:11]    |
[01:14:11]    = note: #[warn(unused_imports)] on by default
[01:14:11] 
[01:14:11] error[E0063]: missing field `edition` in initializer of `syntax::ext::base::SyntaxExtension`
[01:14:11]   --> /checkout/src/test/run-pass-fulldeps/auxiliary/plugin_args.rs:51:9
[01:14:11] 51 |         NormalTT {
[01:14:11] 51 |         NormalTT {
[01:14:11]    |         ^^^^^^^^ missing `edition`
[01:14:11] error: aborting due to previous error
[01:14:11] 
[01:14:11] For more information about this error, try `rustc --explain E0063`.
[01:14:11] 
---
[01:14:11] test result: FAILED. 85 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out
[01:14:11] 
[01:14:11] 
[01:14:11] 
[01:14:11] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "3.9.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:14:11linux-gnu/release
41260 ./src/llvm/test/CodeGen/X86
40772 ./src/libcompiler_builtins
40268 ./src/libcompiler_builtins/compiler-rt

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors

bors commented May 13, 2018

Copy link
Copy Markdown
Collaborator

☔ The latest upstream changes (presumably #50235) made this pull request unmergeable. Please resolve the merge conflicts.

@petrochenkov petrochenkov changed the title [WIP] Implement edition hygiene for keywords Implement edition hygiene for keywords May 13, 2018
@petrochenkov petrochenkov added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 13, 2018
@rust-highfive

Copy link
Copy Markdown
Contributor

The job x86_64-gnu-debug of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:03:48]    Compiling syntax_pos v0.0.0 (file:///checkout/src/libsyntax_pos)
[00:03:49]    Compiling rustc_errors v0.0.0 (file:///checkout/src/librustc_errors)
[00:03:51]    Compiling backtrace v0.3.6
[00:04:03]    Compiling proc_macro v0.0.0 (file:///checkout/src/libproc_macro)
[00:04:03] error[E0425]: cannot find function `is_path_segment_keyword` in module `token`
[00:04:03]    --> libproc_macro/lib.rs:821:19
[00:04:03]     |
[00:04:03] 821 |            token::is_path_segment_keyword(ast::Ident::with_empty_ctxt(ident.sym)) {
[00:04:03]     |                   ^^^^^^^^^^^^^^^^^^^^^^^ not found in `token`
[00:04:04] error: aborting due to previous error
[00:04:04] 
[00:04:04] For more information about this error, try `rustc --explain E0425`.
[00:04:04] error: Could not compile `proc_macro`.
[00:04:04] error: Could not compile `proc_macro`.
[00:04:04] 
[00:04:04] Caused by:
[00:04:04]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name proc_macro libproc_macro/lib.rs --color always --error-format json --crate-type dylib --emit=dep-info,metadata -C prefer-dynamic -C debug-assertions=off -C overflow-checks=on -C metadata=f65e917d1a6aeb8a -C extra-filename=-f65e917d1a6aeb8a --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps --extern rustc_errors=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-24d16f00de0b3e02.rmeta --extern rustc_data_structures=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-750c848698d19171.rmeta --extern syntax_pos=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax_pos-54e4cced589011ca.rmeta --extern syntax=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax-b878b60a3b79382b.rmeta` (exit code: 101)
[00:04:04] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:04:04] expected success, got: exit code: 101
[00:04:04] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1091:9
[00:04:04] travis_fold:end:stage0-rustc

[00:04:04] travis_time:end:stage0-rustc:start=1526541407644825260,finish=1526541444689757796,duration=37044932536

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@kennytm kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 17, 2018
@petrochenkov

Copy link
Copy Markdown
Contributor Author

Needs rebase.

@petrochenkov

Copy link
Copy Markdown
Contributor Author

@bors r=nikomatsakis

@bors

bors commented May 17, 2018

Copy link
Copy Markdown
Collaborator

📌 Commit d8bbc1e has been approved by nikomatsakis

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 17, 2018
@withoutboats withoutboats mentioned this pull request May 17, 2018
9 tasks
@bors

bors commented May 18, 2018

Copy link
Copy Markdown
Collaborator

⌛ Testing commit d8bbc1e with merge df40e61...

bors added a commit that referenced this pull request May 18, 2018
Implement edition hygiene for keywords

Determine "keywordness" of an identifier in its hygienic context.
cc #49611

I've resurrected `proc` as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: `proc` is removed again.
@bors

bors commented May 18, 2018

Copy link
Copy Markdown
Collaborator

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing df40e61 to master...

@cramertj

cramertj commented Jun 6, 2018

Copy link
Copy Markdown
Member

@petrochenkov Looking over this PR I'm unable to find a way to get the current edition while parsing (not necessarily in a macro). Should the edition be passed down into the parser, or is there something else I should be using to determine if something is a keyword in the current edition while parsing?

@petrochenkov

petrochenkov commented Jun 9, 2018

Copy link
Copy Markdown
Contributor Author

@cramertj

a way to get the current edition

hygiene::default_edition()
(aka edition of the "root" hygienic context (no expansion) aka DUMMY_SP.edition(), edition needs to be kept in global data like other hygiene-related things)

@petrochenkov

Copy link
Copy Markdown
Contributor Author

@cramertj

is there something else I should be using to determine if something is a keyword

For a given identifier is_reserved_ident should automatically return the correct answer adjusted for editions, without additional user intervention.

carols10cents added a commit to rust-lang/book that referenced this pull request Oct 23, 2018
`async` was added in rust-lang/rust#50307 and is
not yet implemented

`try` was added in rust-lang/rust#52602 and is
not yet stable: rust-lang/rust#31436
@petrochenkov petrochenkov deleted the keyhyg2 branch June 5, 2019 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants