Skip to content

rustbuild: Tweak LLVM distribution layout#53955

Merged
bors merged 1 commit into
rust-lang:masterfrom
alexcrichton:fix-dist-again
Sep 6, 2018
Merged

rustbuild: Tweak LLVM distribution layout#53955
bors merged 1 commit into
rust-lang:masterfrom
alexcrichton:fix-dist-again

Conversation

@alexcrichton

Copy link
Copy Markdown
Member

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at $sysroot/lib. The LLVM tools we ship are
in two locations:

  • LLD is shipped at $sysroot/lib/rustlib/$host/bin/rust-lld
  • Other LLVM tools are shipped at $sysroot/bin

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to ../lib and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at $sysroot/bin, not LLD!

This commit is targeted at fixing this situation by making two changes:

  • LLVM tools other than LLD are moved in the distribution to
    $sysroot/lib/rustlib/$host/bin. This moves them next to LLD and should
    position them for...
  • The LLVM shared object is moved to $sysroot/lib/rustlib/$host/lib

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813

@rust-highfive

Copy link
Copy Markdown
Contributor

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 4, 2018
@alexcrichton

Copy link
Copy Markdown
Member Author

cc @japaric, this is the change I mentioned on IRC

r? @kennytm

@rust-highfive rust-highfive assigned kennytm and unassigned aturon Sep 4, 2018

@japaric japaric left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. (You can r=me if that makes sense)

@cuviper

cuviper commented Sep 4, 2018

Copy link
Copy Markdown
Member

cc #53852 @Keruspe - this may solve your issue by moving libLLVM under rustlib/.

@alexcrichton

Copy link
Copy Markdown
Member Author

@bors: try

Gonna run this through bors to double check it's working as expected.

@bors

bors commented Sep 4, 2018

Copy link
Copy Markdown
Collaborator

⌛ Trying commit ace1929 with merge b07b6be...

bors added a commit that referenced this pull request Sep 4, 2018
rustbuild: Tweak LLVM distribution layout

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813
@bors

bors commented Sep 4, 2018

Copy link
Copy Markdown
Collaborator

☀️ Test successful - status-travis
State: approved= try=True

@Keruspe

Keruspe commented Sep 5, 2018

Copy link
Copy Markdown
Contributor

This would partly fix #53852 too, but stable, beta and nightly would install the same file (once it reaches stable), so adding a way to customize LLVM_VERSION_SUFFIX will still be needed

@alexcrichton

Copy link
Copy Markdown
Member Author

@bors: r=japaric

@bors

bors commented Sep 5, 2018

Copy link
Copy Markdown
Collaborator

📌 Commit 60584a36dbac1427f341ab073c1e9cce0e89d869 has been approved by japaric

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 5, 2018
@alexcrichton

Copy link
Copy Markdown
Member Author

@bors: p=1

(fixing a regression for a number of targets on nightly)

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In rust-lang#53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes rust-lang#53813
@alexcrichton

Copy link
Copy Markdown
Member Author

@bors: r=japaric

@cuviper

cuviper commented Sep 5, 2018

Copy link
Copy Markdown
Member

@Keruspe if you're trying to install multiple versions to a shared sysroot, how do you deal with the existing conflicts? At least bin/ and lib/rustlib/$target/codegen-backends/ are not uniquely named.

@Keruspe

Keruspe commented Sep 5, 2018 via email

Copy link
Copy Markdown
Contributor

@alexcrichton

Copy link
Copy Markdown
Member Author

@bors: r=japaric

@bors

bors commented Sep 5, 2018

Copy link
Copy Markdown
Collaborator

📌 Commit bce09b6 has been approved by japaric

@bors

bors commented Sep 6, 2018

Copy link
Copy Markdown
Collaborator

⌛ Testing commit bce09b6 with merge 27e5457...

bors added a commit that referenced this pull request Sep 6, 2018
rustbuild: Tweak LLVM distribution layout

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813
@bors

bors commented Sep 6, 2018

Copy link
Copy Markdown
Collaborator

☀️ Test successful - status-appveyor, status-travis
Approved by: japaric
Pushing 27e5457 to master...

@bors bors merged commit bce09b6 into rust-lang:master Sep 6, 2018
tromey added a commit to tromey/rust that referenced this pull request Sep 7, 2018
We're shipping a rust-enabled lldb, but the "lldb" executable is not
installed into the "bin" directory by rustup.  See the discussion in
rust-lang/rustup#1492 for
background on this decision.  There, we agreed to have rust-lldb
prefer the rust-enabled lldb if it is installed.

This patch changes dist.rs to put lldb into rustlib, following what
was done for the other LLVM tools in rust-lang#53955, and then fixes rust-lldb
to prefer that lldb, if it exists.

See issue rust-lang#48168
@alexcrichton alexcrichton deleted the fix-dist-again branch September 27, 2018 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rust-lld fails with "dyld: Library not loaded: @rpath/libLLVM.dylib" on 1.30.0-nightly (02cb8f2a4 2018-08-29)

8 participants