parallel code should respect user settings for number of cores/threads to use

[achubaty]

Package code should let the user control the number of cores to use.

There exist several mechanisms to specify the number of CPU threads to use for parallel operations (e.g., option Ncpus) one of which should be used when determining the default numCoresToUse.

See also https://parallelly.futureverse.org/#availablecores-vs-paralleldetectcores.

[achubaty]

Also note that there is no mechanism in numCoresToUse to change/update the default number of cores once set!

[eliotmcintire]

The Ncpus option is intended to be used for install.packages. It seems like that should not be related to other uses of parallelism. Are other possibilities?

[eliotmcintire]

https://parallelly.futureverse.org/#availablecores-vs-paralleldetectcores does not really cover the practical situation of how many cores are actually available on shared machines. There is a function in PredictiveEcology/clusters@main ... numActiveThreads that is more like what is more generally of interest. However, it only is designed to work on *nix systems where ps is available.

[achubaty]

yes it does - availableCores, which respects various multicore options, cgroup limits, etc., giving lots of ways for the user or system administrator to configure multicore use.

that clusters package cannot be used because it sets up dependency conflicts and is not on CRAN.

[eliotmcintire]

parallelly now (not sure how recent) has the functionality with parallelly::freeCores(). But this is not about the options("Ncpus"). Can you implement another option that does what you need, but without using options("Ncpus") which is for a very different purpose?