Loading `libm` on macOS fails with Python 3.7

[jakirkham]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 05:59:23) 
[Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library
>>> assert find_library("libm") is not None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Note: This works with Python 3.8+. Maybe we are missing a patch with Python 3.7?

Installed packages

# packages in environment at /Users/jkirkham/miniconda/envs/py37:
#
# Name                    Version                   Build  Channel
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
attrs                     22.1.0             pyh71513ae_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
blas                      2.115                  openblas    conda-forge
blas-devel                3.9.0           15_osx64_openblas    conda-forge
ca-certificates           2022.6.15            h033912b_0    conda-forge
cloudpickle               2.1.0              pyhd8ed1ab_0    conda-forge
cytoolz                   0.12.0           py37h994c40b_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
flake8                    5.0.4              pyhd8ed1ab_0    conda-forge
fsspec                    2022.7.1           pyhd8ed1ab_0    conda-forge
importlib-metadata        4.11.4           py37hf985489_0    conda-forge
importlib_metadata        4.11.4               hd8ed1ab_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
ipython                   7.33.0           py37hf985489_0    conda-forge
jedi                      0.18.1             pyhd8ed1ab_2    conda-forge
libblas                   3.9.0           15_osx64_openblas    conda-forge
libcblas                  3.9.0           15_osx64_openblas    conda-forge
libcxx                    14.0.6               hce7ea42_0    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
liblapack                 3.9.0           15_osx64_openblas    conda-forge
liblapacke                3.9.0           15_osx64_openblas    conda-forge
libopenblas               0.3.20          openmp_h4e9756f_1    conda-forge
libzlib                   1.2.12               hfe4f2af_2    conda-forge
llvm-openmp               14.0.4               ha654fa7_0    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h96cf925_1    conda-forge
numpy                     1.21.6           py37h345d48f_0    conda-forge
openblas                  0.3.20          openmp_h2608fb9_1    conda-forge
openssl                   3.0.5                hb81d4ab_1    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.2.3            py37hff173d8_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickle5                   0.0.12           py37h271585c_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0            py37hf985489_3    conda-forge
prompt-toolkit            3.0.30             pyha770c72_0    conda-forge
psutil                    5.9.1            py37h994c40b_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
pycodestyle               2.9.1              pyhd8ed1ab_0    conda-forge
pyflakes                  2.5.0              pyhd8ed1ab_0    conda-forge
pygments                  2.12.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pytest                    7.1.2            py37hf985489_0    conda-forge
python                    3.7.12          hf3644f1_100_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.7                     2_cp37m    conda-forge
pytz                      2022.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py37h69ee0a8_4    conda-forge
readline                  8.1.2                h3899abd_0    conda-forge
setuptools                59.8.0           py37hf985489_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.39.2               hd9f0692_0    conda-forge
tk                        8.6.12               h5dbffcc_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
traitlets                 5.3.0              pyhd8ed1ab_0    conda-forge
typing_extensions         4.3.0              pyha770c72_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                h0d85af4_2    conda-forge
zipp                      3.8.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.12               hfe4f2af_2    conda-forge

Environment info

active environment : base
    active env location : /Users/jkirkham/miniconda
            shell level : 1
       user config file : /Users/jkirkham/.condarc
 populated config files : /Users/jkirkham/.condarc
          conda version : 4.13.0
    conda-build version : 3.21.9
         python version : 3.9.13.final.0
       virtual packages : __osx=12.2.1=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /Users/jkirkham/miniconda  (writable)
      conda av data dir : /Users/jkirkham/miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/jkirkham/miniconda/pkgs
                          /Users/jkirkham/.conda/pkgs
       envs directories : /Users/jkirkham/miniconda/envs
                          /Users/jkirkham/.conda/envs
               platform : osx-64
             user-agent : conda/4.13.0 requests/2.28.1 CPython/3.9.13 Darwin/21.3.0 OSX/12.2.1
                UID:GID : 523832026:1019530959
             netrc file : None
           offline mode : False

[isuruf]

Python 3.7 does not support macos >=11 officially and this is one of the issues with macos 11 support. It's an upstream issue, but 3.7 is in security fix mode only.

[jakirkham]

Thanks Isuru 🙏 That makes sense. So even backporting new patches wouldn't necessarily solve things?

[isuruf]

It would, but it's a lot of code and would take a lot of time do the backport correctly.

[jakirkham]

Yeah agreed. Doesn't seem worth it.

[jakirkham]

TBH not sure the newer Python's do the right thing either. They return a path, but it doesn't exist:

Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:09:13) [Clang 13.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library
>>> from os.path import exists
>>> find_library("m")
'/usr/lib/libm.dylib'
>>> exists(find_library("m"))
False

There is a libm.tbd file in the SDK, but IIUC it is just a shim to handle linking to other system libraries (like libSystem.B.dylib).

Though this seems like an upstream issue.

[isuruf]

That's how it works. See macOS 11 announcement on cached system libraries.