Documentation incoherent with releases

[mboisson]

The documentation says:
Stan Math depends on four libraries:
Boost (version 1.75.0): Boost Home Page
Eigen (version 3.3.9: Eigen Home Page
SUNDIALS (version 5.7.0): Sundials Home Page
Intel TBB (version 2020.3): Intel TBB Home Page
These are distributed under the lib/ subdirectory. Only these versions of the dependent libraries have been tested with Stan Math.

yet, once you extract the latest release (4.0.1), it includes
boost 1.72.0
sundials 5.6.1
tbb_2019_U8

so... which is it ?

[rok-cesnovar]

Hi,

are you referring to: https://github.com/stan-dev/math#required-libraries
If so, that refers to the libraries that are in the develop versions of Stan Math (develop branch).

For the release you need to look at documentation of the release, which is on the release branch: https://github.com/stan-dev/math/tree/v4.0.1#required-libraries

And that correctly states:

Boost (version 1.72.0): Boost Home Page
Eigen (version 3.3.9: Eigen Home Page
SUNDIALS (version 5.6.1): Sundials Home Page
Intel TBB (version 2019_U8): Intel TBB Home Page

FYI, there will also be a release later today.

[mboisson]

ah, I see, thanks.

Any documentation about how to not used the vendored dependencies ? We already have all of those installed on our clusters, and installing stuff multiples times just should not be needed.

[SteveBronder]

There's no docs on how to use nonvendored dependencies but you can start by swapping out the paths to the dependencies below

https://github.com/stan-dev/math/blob/develop/make/libraries#L11

installing stuff multiples times just should not be needed.

Why so? I don't think it's reasonable to expect a package to work with every previous / future versions of dependencies, which is what you end up supporting if you default to the system versions. + the libraries are a few mb in size.

[mboisson]

Why so? I don't think it's reasonable to expect a package to work with every previous / future versions of dependencies, which is what you end up supporting if you default to the system versions. + the libraries are a few mb in size.

Because we already have a few thousands of builds. We can install different versions if they are absolutely needed, but we should not need to install twice the same version with the same options, ever.

I can only imagine how much more space our software stack would take if every single library vendored its own dependencies.... we would have hundreds of copies of Boost.

Versions of well-developed packages like Boost are usually backward compatible. There is no reason to specify an exact version match, only a minimum version should be needed. That's why semantic versioning exists: https://semver.org/

[wds15]

Nope…also boost contains some rare bugs, which we fix…and macOS also deploys fat binaries with most stuff included.

[mboisson]

Those rare bugs should be contributed back upstream, or at least, the patches should be documented. We can also apply patches when we build the external dependencies, if they are really bugs

[wds15]

They are contributed back! It just takes time for the turn around.

[mboisson]

Ok. So... any patch we should be aware of, on top of boost, sundials, tbb or eigen, before we try to use our already existing versions ? i.e. are those patches documented somewhere ?

[wds15]

Easiest is to diff against the libs.

[rok-cesnovar]

If you will be using the just released Stan Math, there actually are no required changes in the downstream libraries.

The only change apart from pruning unnecessary files (examples and unused subpackages of the library) from the downstream libraries is removing prints from Sundials https://github.com/stan-dev/math/blob/develop/lib/upgrade-sundials.sh#L122
This is required for CRAN releases of rstan and can be ignored if you will not be dealing with CRAN.

You can see the rest of the upgrade scripts here: https://github.com/stan-dev/math/tree/develop/lib

[rok-cesnovar]

For Stan Math 4.0.0. there was at least this one #2342

There are changes made to the GTest library but that one is not used outside of Stan Math development.

[mboisson]

Thanks. One last question, and I know this is diverting from the first issue.

A typical software on Linux would build and install through something like :

1. configure step: configure/cmake
2. build step: make
3. install step: make install

STAN does not seem to follow this pattern. I understand the configure step is done by editing makefile files. However, step 3 is pretty crucial, it should copy some binaries, libraries and header files into an installation prefix. Since you do not provide such an install step (as far as I've seen), what is expected to be copied in which directory in order to have a working installation ?

[wds15]

The cmdstan thing is meant to be used in-place. We do not have a make install. So you install the cmdstan somewhere and then do

cd cmdstan-path
make path-to-mymodel/model.stan

done.

[mboisson]

Ok, thanks. From that, I conclude it is not meant to be installed centrally for many users. I will tell my user that they should install it themselves.