Checklist
What happened?
conda-build's CI tests are still a bit flaky.
gh-5068 and gh-5076 lessened the chance of failures, but there are still some underlying issues:
The serial tests are actually not run in series anymore
I did not change/fix that in the recent pull requests (since tests run somewhat fine and otherwise it's a waste of resources).
We actually should run most of those tests in parallel since the reasons they were marked as serial are most likely connected to reasons handled by the pull requests above.
There may be some tests that should actually be run in series -- we might want to run them separately (or at least within a single worker, e.g., via --dist loadgroup.
The tests (and conda-build in general) use the same global package cache
With the recent changes, the tests should now use individual working directories, but they still use the same shared package cache.
When the tests are run concurrently, this means they can corrupt indexes (or possibly packages when different channels/subdirs are used).
Ideally we want the package cache separated for the tests too.
However, this could increase resource usage significantly since the same packages would be downloaded/extracted multiple times to different caches then.
Hence, we might want to do this selectively (I haven't thought out a plan for this yet, though).
Another problem related to the shared package cache is that conda-build will add the packages it built to the package cache when it builds subpackage builds or runs tests (because that's how conda works: to create an environment, the packages are put into pkgs_dirs beforehand).
For the CI tests, this means that packages with the same name+version+build will get clobbered, e.g., see this recent workaround for such a case.
We may want to change that behavior overall -- not just for the CI tests -- and let conda-build put its own builds into a separate pkgs directory relative to its working directory.
There's also the issue with collisions in the cache from different channel/subdirs in general, but that one should be handled on a case-by-case basis for the tests (and/or ideally upstream in conda by appending a content-dependent hash to the archives/folders, e.g., pkgs/package-version-build-<additional hash>).
Conda Info
No response
Conda Config
No response
Conda list
No response
Additional Context
No response
Checklist
What happened?
conda-build's CI tests are still a bit flaky.gh-5068 and gh-5076 lessened the chance of failures, but there are still some underlying issues:
The
serialtests are actually not run in series anymorexxx && 0 || 'auto'which will always evaluate toauto.I did not change/fix that in the recent pull requests (since tests run somewhat fine and otherwise it's a waste of resources).
We actually should run most of those tests in parallel since the reasons they were marked as
serialare most likely connected to reasons handled by the pull requests above.There may be some tests that should actually be run in series -- we might want to run them separately (or at least within a single worker, e.g., via
--dist loadgroup.The tests (and
conda-buildin general) use the same global package cacheWith the recent changes, the tests should now use individual working directories, but they still use the same shared package cache.
When the tests are run concurrently, this means they can corrupt indexes (or possibly packages when different channels/subdirs are used).
Ideally we want the package cache separated for the tests too.
However, this could increase resource usage significantly since the same packages would be downloaded/extracted multiple times to different caches then.
Hence, we might want to do this selectively (I haven't thought out a plan for this yet, though).
Another problem related to the shared package cache is that
conda-buildwill add the packages it built to the package cache when it builds subpackage builds or runs tests (because that's howcondaworks: to create an environment, the packages are put intopkgs_dirsbeforehand).For the CI tests, this means that packages with the same name+version+build will get clobbered, e.g., see this recent workaround for such a case.
We may want to change that behavior overall -- not just for the CI tests -- and let
conda-buildput its own builds into a separatepkgsdirectory relative to its working directory.There's also the issue with collisions in the cache from different channel/subdirs in general, but that one should be handled on a case-by-case basis for the tests (and/or ideally upstream in
condaby appending a content-dependent hash to the archives/folders, e.g.,pkgs/package-version-build-<additional hash>).Conda Info
No response
Conda Config
No response
Conda list
No response
Additional Context
No response