incompatible_list_based_execution_strategy_selection: List-based explicit spawn strategy selection.

[philwo]

For more detailed description visit our blog post: https://blog.bazel.build/2019/06/19/list-strategy.html

🙅‍♂️ Current situation:

• The user can set the execution strategy in general and for individual action mnemonics using the flags --spawn_strategy=, --strategy=Mnemonic= and --strategy_regexp=Regex=. The flags take a single strategy name as value. Example: --spawn_strategy=linux-sandbox.
• Bazel has a hardcoded and undocumented list of mnemonics that are known to work well with certain strategies and uses these unless the user overrides them individually using --strategy=Mnemonic= flags. Example: Bazel will use some kind of sandboxing by default if its available on your system, otherwise non-sandboxed execution (e.g. on Windows). However, it will run Javac actions via the persistent worker strategy.
• Each strategy has custom code to deal with the situation that it can't execute a given action - for example, the remote strategy silently falls back to sandboxed execution if an action can't run remotely. The sandbox strategy silently falls back to non-sandboxed local execution if an action doesn't support sandboxing. There's no good way to configure this behavior.

🆕 The option --incompatible_list_based_execution_strategy_selection:

• Allows the user to pass comma-separated lists of strategies to the above mentioned flags: --spawn_strategy=remote,worker,linux-sandbox.
• Each strategy now knows whether it can execute a given action. For any action that it wants to execute, Bazel just picks the first strategy from the given list that claims to be able to execute the action. If no strategy remains, the execution will fail.
• This means you can now have your build fail if any non-sandboxable or non-remotable action sneaks in! More reproducibility and safety for your builds! 🎉
• The strategies no longer do their own custom fallback, simplifying the code and unifying the behavior.

💡 Some ideas how to use this:

• Don't set any flags and Bazel will try to do the best automatically: Use remote execution if it's available, otherwise persistent workers, otherwise sandboxed execution, otherwise non-sandboxed execution.
• I want the best sandboxed build on my Linux machine and no automatic fallback to symlink-only sandboxing, non-sandboxed execution and no persistent workers: --spawn_strategy=linux-sandbox.
• I want persistent workers for actions that support it, but otherwise only sandboxed execution: --spawn_strategy=worker,sandboxed.

🚧 Migration needed:

• If you currently use the strategy selection flags, you'll want to revisit the values you set them to.
• If you currently patch your Bazel version to remove fallback to non-sandboxed execution, this should no longer be necessary then (:wave: @AustinSchuh).
• If you currently use remote execution with --strategy=remote and/or --spawn_strategy=remote and have actions that might not be executed remotely, consider removing those strategy flags completely, in this case bazel will pickup the first available strategy from the default list, which is remote,worker,sandboxed,local. Alternatively, add a strategy to fallback to if remote is not possible for an action, for example --spawn_strategy=remote,sandboxed will fallback to a sandboxed execution if remote is not possible.
• If you are using --config=remote you might need to change your .bazelrc file, where all the remote configs are located, including strategy flags.

[jin]

/sub

[AustinSchuh]

Thanks @philwo ! This looks helpful.

We moved over to just using --spawn_strategy=linux-sandbox based on your advice in late 2017. That means we no longer have a patch for that.

Does this replace --remote_local_fallback_strategy=linux-sandbox ?

[agoulti]

Yes.
Under incompatible_list_based_execution_strategy_selection, remote_local_fallback_strategy will no longer be respected. A target that is marked no-remote will not be eligible for remote execution, so the next available strategy from the list will be applied.

We plan to deprecate remote_local_fallback_strategy by the time incompatible_list_based_execution_strategy_selection becomes the default.

We think that the new strategy list should serve all the fallback needs (and make things more explicit and cleaner). Do let us know if for some reason your case is not being served!

[buchgr]

@ishikhman as parts of your flag work can you please take over ownership of flipping this flag? You probably won't get to it by 0.26 (this thursday) so let's target to flip this in 0.27? I don't expect there to be much trouble except for fixing a few tests.