Make element selection consistent across commands

[BuildStream-Migration-Bot]

See original issue on GitLab
In GitLab by [Gitlab user @cs-shadow] on Jul 3, 2020, 12:12

Summary

BuildStream mainly offers two options to tweak element sections - --except and --deps. However they are not accepted uniformly by all commands.

For example, some commands like show, source checkout etc support --except option to exclude certain elements from the pipeline. However, others like artifact checkout don't.

Similarly, not all commands accept all values for --deps. For example, source track only accepts all and none whereas show accepts all|none|build|run|plan. Out of these, plan is probably the most tricky one, as mentioned in https://gitlab.com/BuildStream/buildstream/-/merge_requests/1893#note_335927435. We need a plan about how to handle --deps plan sensibly. But, it should be generally reasonable to add the other options to all the commands.

Todo

Part of --deps unification was implemented in !1904. Here's what's left to do:

• Unify --deps for the remaining commands
• Provide --except to commands more consistently, review where it should be added
• Have a plan for handling --deps plan, especially important from API perspective if it's going to move out of --deps option
• Implement the said plan

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Jul 3, 2020, 12:13

assigned to [Gitlab user @cs-shadow]

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Jul 3, 2020, 12:13

mentioned in merge request !1975

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 13, 2020, 22:16

mentioned in commit 5965be6bbf23929f70dd4e845e43ca33abcae56a

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 13, 2020, 22:17

mentioned in merge request !2027

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 13, 2020, 22:22

mentioned in merge request !1893

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 13, 2020, 23:26

mentioned in commit fd4b622ff77251d791a227719f25db19d4c10b22

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 14, 2020, 12:44

mentioned in commit abd8615

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Aug 18, 2020, 21:30

marked the task Unify --deps for the remaining commands as completed

[gtristan]

Element excepting

In order to get consensus on what needs to be done for --except option consistency, here is a breakdown of all of the commands with some thought into whether it is a candidate for --except.

• bst build

  When building the specified targets, it does not make sense to omit elements which are required for a build, as that is just not really possible.

  One could potentially --except build dependencies which would have been included by --deps all but not by --deps plan (i.e. the --except option would only be meaningful when running bst build --deps all).

• bst show

  This command shows the --deps elements of the specified targets, here --except makes sense and is implemented.

• bst shell

  This command takes only one target and creates a shell, all of the target dependencies are required and cannot be excepted.

  I could envision a future where the user wants to stage sources or build trees for the entire runtime dependency graph, in order to more easily use debuggers - in such a future the user may want to be selective about which sources are staged in the shell, and at that time it may make sense to introduce a --source-except option for this.

  Such a future idea however, should not be a blocker at this time.

• bst source fetch

  Fetches sources for the --deps elements of the specified targets, here --except makes sense and is implemented.

• bst source push

  Pushes sources for the --deps elements of the specified targets, here --except makes sense.

  Here the --except semantic is not implemented (opened Add --except option to bst source push #1445 to add this)

• bst source track

  Tracks fresh versions of sources for the --deps elements of the specified targets, here --except makes sense and is implemented.

• bst source checkout

  Checkout sources for the the --deps elements of the specified targets, here --except makes sense and is implemented.

• bst workspace *

  Workspace commands never have a --deps argument, they operate on explicit elements and do not manipulate dependency trees. There is no need for --except in any of these.

• bst artifact *

  Here we some ambiguity, especially given issue Commands which accept artifact names and element targets are ambiguous #1411.

  • All artifact commands which have a --deps argument are potential candidates for an --except argument
  • Artifact commands currently support element names or artifact names interchangeably (see Commands which accept artifact names and element targets are ambiguous #1411)
  • When specifying artifact names on the command line, it is unclear how we can support --except, possibly we would want to support ${project-name}/${element-name} syntax (without the cache key) for --except values to apply against artifact dependencies.

Tentative conclusion

To provide consistent support we should add the --except argument to bst source push.

Regarding the artifact commands, we should block on figuring out how we want to support --except arguments for dependencies of targets specified by artifact name, this would be a new feature.

I think that this new feature of --except for artifact names is a decent potential enhancement and can be added without breaking CLI API. I don't think that this addition constitutes a blocker for BuildStream 2.