Skip to content

Wrap all of x.py in if __name__ == '__main__': to avoid problems with multiprocessing#111049

Closed
Zalathar wants to merge 1 commit into
rust-lang:masterfrom
Zalathar:x-py
Closed

Wrap all of x.py in if __name__ == '__main__': to avoid problems with multiprocessing#111049
Zalathar wants to merge 1 commit into
rust-lang:masterfrom
Zalathar:x-py

Conversation

@Zalathar

@Zalathar Zalathar commented May 1, 2023

Copy link
Copy Markdown
Member

#110427 caused bootstrap.py to start using the multiprocessing module, which unfortunately broke the bootstrap script on some systems.

It seems that in some situations, Python's multiprocessing module needs to be able to safely import a program's entry-point module without executing the program.

Fixes #111046.

@rustbot

rustbot commented May 1, 2023

Copy link
Copy Markdown
Collaborator

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 1, 2023
@Zalathar

Zalathar commented May 1, 2023

Copy link
Copy Markdown
Member Author

The easier fix would be to just revert #110427, but filing this PR was simple enough that I figured I'd give the option of doing this instead.

@albertlarsan68

Copy link
Copy Markdown
Member

If you could revert the PR in addition of your changes, it would be great.

@jyn514

jyn514 commented May 1, 2023

Copy link
Copy Markdown
Member

Why? This fixes the bug, doesn't it? I don't think we gain anything by reverting

@albertlarsan68

albertlarsan68 commented May 1, 2023

Copy link
Copy Markdown
Member

I (at least) would like to join the pool before continuing and use the os.get_schedaffinity() function instead of the number of CPUs.
If this could be done in this PR, then no need to revert.
The first is especially important, to avoid trying to use the files before they are extracted (may happen with slower CPUs).

EDIT: Per the multiprocessing documentation, the function to use would be len(os.sched_getaffinity(0))

@Zalathar

Zalathar commented May 1, 2023

Copy link
Copy Markdown
Member Author

I'm not interested in fighting over this.

@Zalathar Zalathar closed this May 1, 2023
@Zalathar Zalathar deleted the x-py branch May 31, 2023 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed to run x

5 participants