[RFC] [WIP] use bash arrays for some template fields

[classabbyamp]

this will allow select variables to be defined as arrays or strings. if a string is specified, it will be converted to an array for internal usage. if it is an array, it passes through untouched. this should allow for a smooth transition to using more arrays inside xbps-src, and should solve issues with shell quoting in things like configure_args, make_check_args, etc.

to start, I've only done this with configure_args, and fixed some templates

would like to get some feedback before continuing, as this will be a fairly massive undertaking. what fields would be valuable as arrays? (I'm thinking mostly the *_args variables) should the internal string -> array conversion happen or should we just go all-in?

Testing the changes

• I tested the changes in this PR: YES|briefly|NO

[ci skip]

[tranzystorekk]

*depends variables also seem like a good pick for this.

I haven't checked if there's any convenient mechanism for concatenating arrays, which would be nice in python packages as an alternative to checkdepends="${depends} ...".

If possible, it would be nice if we could make this optional, i.e. still parse variables the old way, but if we detect an array, we skip the conversion step and just take it as-is.

EDIT: sorry I skimmed through the part where you already say it's optional by design.

[classabbyamp]

I haven't checked if there's any convenient mechanism for concatenating arrays

for the same array, just foo+=(new_element), for different:

$ foo=(foo bar baz)
$ bar=("${foo[@]}" idk)
$ declare -p foo
declare -a foo=([0]="foo" [1]="bar" [2]="baz")
$ declare -p bar
declare -a bar=([0]="foo" [1]="bar" [2]="baz" [3]="idk")

[tranzystorekk]

I haven't checked if there's any convenient mechanism for concatenating arrays

for the same array, just foo+=(new_element), for different:

$ foo=(foo bar baz)
$ bar=("${foo[@]}" idk)
$ declare -p foo
declare -a foo=([0]="foo" [1]="bar" [2]="baz")
$ declare -p bar
declare -a bar=([0]="foo" [1]="bar" [2]="baz" [3]="idk")

Hm, for me the added explicitness of "${foo[@]}" is a plus, but probably not everyone would agree.

[ahesford]

Of all the xbps-src improvement proposals, this is the one I most strongly support. Bash arrays will solve a lot of problems in a lot of templates and we should move any variable currently space-separated to an array for proper handling. This includes:

• *depends
• *args
• distfiles
• checksum
• nostrip_files
• ignore_elf*
• make_dirs (either one value per entry to simplify iteration, or just split every component so that space handling in paths would work as expected... not sure yet)
• subpackages
• nopie_files
• shlib_{provides,requires}
• reverts
• alternatives
• font_dirs
• dkms_modules
• register_shell
• tags
• perl_configure_dirs
• archs
• conf_files
• mutable_files
• skip_extraction
• make_check_pre
• make_*_target
• build_helper
• make_cmd
• configure_script
• conflicts
• replaces

We can do this by attrition, but relatively quickly, with a linter that fails if any of the should-be-arrays is a string that will split into an array with more than one element. Thus, the only time we accept the current syntax in future template updates is as a shorthand for a one-element array.

[paper42]

this might be for another issue/PR, but we might also want to consider using associative arrays for build_options like this:

declare -A buildopts=([docs]=0 [wayland]=1 [xorg]=1)

if this was declared before buildopts are used (= usually before configure_args), we could parse the template only once instead of twice if we also used functions for accessing build options instead of variables like currently (because buildopts has to be combined with options from cli args and the config). The only disadvantage I can think of is that it would be ugly to have declare -A right at the top of the template.

[classabbyamp]

I think that should be out of scope for this PR, but it may be useful to do

[classabbyamp]

We can do this by attrition, but relatively quickly, with a linter that fails if any of the should-be-arrays is a string that will split into an array with more than one element.

I like this, and I've done this in the array conversion function directly, instead of e.g. changing xlint. I feel that using xlint for that would just be a mess

[leahneukirchen]

I'm not a big fan of these 0,1,2 parameter... could we use str ary or something?

[leahneukirchen]

or 1 and n ;)

[classabbyamp]

i think licenses could be an array too, but we'd need to keep spdx expressions together for sanity, like licenses=(LGPL-2.1-or-later "BSD-3-Clause WITH BullshitException")

[Duncaen]

We have nothing really to gain from making licenses an array, at the moment its free text that is not interpreted by anything, making it a shell array now might cause conflicts if we would want to actually use SPDX expressions where you can have nesting and AND/OR operators (MIT AND (LGPL-2.1-or-later OR BSD-3-Clause)).

https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/

[classabbyamp]

I'm not a big fan of these 0,1,2 parameter... could we use str ary or something?

i've made it a bit more readable, but I suspect the code will change more once more things are arrays

[classabbyamp]

I think the best way to go about this will be:

1. don't try to convert strings to arrrays
2. don't try to assume strings are single-element arrays
3. have some utility (maybe written in go with shfmt's library) that can do a best-effort conversion of templates

[github-actions]

Pull Requests become stale 90 days after last activity and are closed 14 days after that. If this pull request is still relevant bump it or assign it.