pybind11: v2.9.1+

CHANGELOG.rst

@@ -17,9 +17,8 @@ Changes to "0.14.0"
 Features
 """"""""
 
-- CMake
-
-  - include internally shipped toml11 v3.7.0 #1148
+- include internally shipped toml11 v3.7.0 #1148
+- pybind11: require version 2.9.1+ #1220
 
 Bug Fixes
 """""""""

CMakeLists.txt

@@ -381,9 +381,9 @@ if(openPMD_USE_PYTHON STREQUAL AUTO)
         if(openPMD_USE_INTERNAL_PYBIND11)
             add_subdirectory("${openPMD_SOURCE_DIR}/share/openPMD/thirdParty/pybind11")
             set(openPMD_HAVE_PYTHON TRUE)
-            message(STATUS "pybind11: Using INTERNAL version 2.6.2")
+            message(STATUS "pybind11: Using INTERNAL version 2.9.1")
         else()
-            find_package(pybind11 2.6.2 CONFIG)
+            find_package(pybind11 2.9.1 CONFIG)
             if(pybind11_FOUND)
                 set(openPMD_HAVE_PYTHON TRUE)
                 message(STATUS "pybind11: Found version '${pybind11_VERSION}'")
@@ -399,9 +399,9 @@ elseif(openPMD_USE_PYTHON)
     if(openPMD_USE_INTERNAL_PYBIND11)
         add_subdirectory("${openPMD_SOURCE_DIR}/share/openPMD/thirdParty/pybind11")
         set(openPMD_HAVE_PYTHON TRUE)
-        message(STATUS "pybind11: Using INTERNAL version 2.6.2")
+        message(STATUS "pybind11: Using INTERNAL version 2.9.1")
     else()
-        find_package(pybind11 2.6.2 REQUIRED CONFIG)
+        find_package(pybind11 2.9.1 REQUIRED CONFIG)
         set(openPMD_HAVE_PYTHON TRUE)
         message(STATUS "pybind11: Found version '${pybind11_VERSION}'")
     endif()

NEWS.rst

@@ -11,6 +11,7 @@ Building openPMD-api now requires a compiler that supports C++17 or newer.
 
 Python 3.10 is now supported.
 openPMD-api now depends on `toml11 <https://github.com/ToruNiina/toml11>`__ 3.7.0+.
+pybind11 2.9.1 is now the minimally supported version for Python support.
 
 The following backend-specific members of the ``Dataset`` class have been removed: ``Dataset::setChunkSize()``, ``Dataset::setCompression()``, ``Dataset::setCustomTransform()``, ``Dataset::chunkSize``, ``Dataset::compression``, ``Dataset::transform``.
 They are replaced by backend-specific options in the JSON-based backend configuration.

README.md

@@ -104,7 +104,7 @@ Required:
 
 Shipped internally in `share/openPMD/thirdParty/`:
 * [Catch2](https://github.com/catchorg/Catch2) 2.13.4+ ([BSL-1.0](https://github.com/catchorg/Catch2/blob/master/LICENSE.txt))
-* [pybind11](https://github.com/pybind/pybind11) 2.6.2+ ([new BSD](https://github.com/pybind/pybind11/blob/master/LICENSE))
+* [pybind11](https://github.com/pybind/pybind11) 2.9.1+ ([new BSD](https://github.com/pybind/pybind11/blob/master/LICENSE))
 * [NLohmann-JSON](https://github.com/nlohmann/json) 3.9.1+ ([MIT](https://github.com/nlohmann/json/blob/develop/LICENSE.MIT))
 * [toml11](https://github.com/ToruNiina/toml11) 3.7.0+ ([MIT](https://github.com/ToruNiina/toml11/blob/master/LICENSE))
 
@@ -120,7 +120,7 @@ while those can be built either with or without:
 Optional language bindings:
 * Python:
   * Python 3.6 - 3.10
-  * pybind11 2.6.2+
+  * pybind11 2.9.1+
   * numpy 1.15+
   * mpi4py 2.1+ (optional, for MPI)
   * pandas 1.0+ (optional, for dataframes)
@@ -271,7 +271,7 @@ The following options allow to switch to external installs:
 | CMake Option                    | Values     | Library       | Version |
 |---------------------------------|------------|---------------|---------|
 | `openPMD_USE_INTERNAL_CATCH`    | **ON**/OFF | Catch2        | 2.13.4+ |
-| `openPMD_USE_INTERNAL_PYBIND11` | **ON**/OFF | pybind11      |  2.6.2+ |
+| `openPMD_USE_INTERNAL_PYBIND11` | **ON**/OFF | pybind11      |  2.9.1+ |
 | `openPMD_USE_INTERNAL_JSON`     | **ON**/OFF | NLohmann-JSON |  3.9.1+ |
 | `openPMD_USE_INTERNAL_TOML11`   | **ON**/OFF | toml11        |  3.7.0+ |
 

docs/source/dev/buildoptions.rst

@@ -69,7 +69,7 @@ The following options allow to switch to external installs of dependencies:
 CMake Option                      Values      Installs Library       Version
 ================================= =========== ======== ============= ========
 ``openPMD_USE_INTERNAL_CATCH``    **ON**/OFF  No       Catch2         2.13.4+
-``openPMD_USE_INTERNAL_PYBIND11`` **ON**/OFF  No       pybind11        2.6.2+
+``openPMD_USE_INTERNAL_PYBIND11`` **ON**/OFF  No       pybind11        2.9.1+
 ``openPMD_USE_INTERNAL_JSON``     **ON**/OFF  No       NLohmann-JSON   3.9.1+
 ``openPMD_USE_INTERNAL_TOML11``   **ON**/OFF  No       toml11          3.7.0+
 ================================= =========== ======== ============= ========

docs/source/dev/dependencies.rst

@@ -18,7 +18,7 @@ Shipped internally
 The following libraries are shipped internally in ``share/openPMD/thirdParty/`` for convenience:
 
 * `Catch2 <https://github.com/catchorg/Catch2>`_ 2.13.4+ (`BSL-1.0 <https://github.com/catchorg/Catch2/blob/master/LICENSE.txt>`__)
-* `pybind11 <https://github.com/pybind/pybind11>`_ 2.6.2+ (`new BSD <https://github.com/pybind/pybind11/blob/master/LICENSE>`_)
+* `pybind11 <https://github.com/pybind/pybind11>`_ 2.9.1+ (`new BSD <https://github.com/pybind/pybind11/blob/master/LICENSE>`_)
 * `NLohmann-JSON <https://github.com/nlohmann/json>`_ 3.9.1+ (`MIT <https://github.com/nlohmann/json/blob/develop/LICENSE.MIT>`_)
 * `toml11 <https://github.com/ToruNiina/toml11>`_ 3.7.0+ (`MIT <https://github.com/ToruNiina/toml11/blob/master/LICENSE>`__)
 
@@ -40,7 +40,7 @@ Optional: language bindings
 * Python:
 
   * Python 3.6 - 3.10
-  * pybind11 2.6.2+
+  * pybind11 2.9.1+
   * numpy 1.15+
   * mpi4py 2.1+ (optional, for MPI)
   * pandas 1.0+ (optional, for dataframes)

pyproject.toml

@@ -3,6 +3,6 @@ requires = [
     "setuptools>=42",
     "wheel",
     "cmake>=3.15.0,<4.0.0",
-    "pybind11>=2.6.2,<3.0.0"
+    "pybind11>=2.9.1,<3.0.0"
 ]
 build-backend = "setuptools.build_meta"

share/openPMD/thirdParty/pybind11/CMakeLists.txt

@@ -7,13 +7,18 @@
 
 cmake_minimum_required(VERSION 3.4)
 
-# The `cmake_minimum_required(VERSION 3.4...3.18)` syntax does not work with
+# The `cmake_minimum_required(VERSION 3.4...3.22)` syntax does not work with
 # some versions of VS that have a patched CMake 3.11. This forces us to emulate
 # the behavior using the following workaround:
-if(${CMAKE_VERSION} VERSION_LESS 3.18)
+if(${CMAKE_VERSION} VERSION_LESS 3.22)
   cmake_policy(VERSION ${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION})
 else()
-  cmake_policy(VERSION 3.18)
+  cmake_policy(VERSION 3.22)
+endif()
+
+# Avoid infinite recursion if tests include this as a subdirectory
+if(DEFINED PYBIND11_MASTER_PROJECT)
+  return()
 endif()
 
 # Extract project version from source
@@ -45,13 +50,8 @@ if(NOT pybind11_FIND_QUIETLY)
   message(STATUS "pybind11 v${pybind11_VERSION} ${pybind11_VERSION_TYPE}")
 endif()
 
-# Avoid infinite recursion if tests include this as a subdirectory
-if(DEFINED PYBIND11_MASTER_PROJECT)
-  set(PYBIND11_TEST OFF)
-endif()
-
 # Check if pybind11 is being used directly or via add_subdirectory
-if(CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR AND NOT DEFINED PYBIND11_MASTER_PROJECT)
+if(CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR)
   ### Warn if not an out-of-source builds
   if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_CURRENT_BINARY_DIR)
     set(lines
@@ -80,6 +80,8 @@ if(CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR AND NOT DEFINED PYBIND11_MASTER_
   endif()
 
   set(pybind11_system "")
+
+  set_property(GLOBAL PROPERTY USE_FOLDERS ON)
 else()
   set(PYBIND11_MASTER_PROJECT OFF)
   set(pybind11_system SYSTEM)
@@ -89,6 +91,9 @@ endif()
 option(PYBIND11_INSTALL "Install pybind11 header files?" ${PYBIND11_MASTER_PROJECT})
 option(PYBIND11_TEST "Build pybind11 test suite?" ${PYBIND11_MASTER_PROJECT})
 option(PYBIND11_NOPYTHON "Disable search for Python" OFF)
+set(PYBIND11_INTERNALS_VERSION
+    ""
+    CACHE STRING "Override the ABI version, may be used to enable the unstable ABI.")
 
 cmake_dependent_option(
   USE_PYTHON_INCLUDE_DIR
@@ -105,6 +110,7 @@ set(PYBIND11_HEADERS
     include/pybind11/detail/descr.h
     include/pybind11/detail/init.h
     include/pybind11/detail/internals.h
+    include/pybind11/detail/type_caster_base.h
     include/pybind11/detail/typeid.h
     include/pybind11/attr.h
     include/pybind11/buffer_info.h
@@ -116,14 +122,16 @@ set(PYBIND11_HEADERS
     include/pybind11/eigen.h
     include/pybind11/embed.h
     include/pybind11/eval.h
+    include/pybind11/gil.h
     include/pybind11/iostream.h
     include/pybind11/functional.h
     include/pybind11/numpy.h
     include/pybind11/operators.h
     include/pybind11/pybind11.h
     include/pybind11/pytypes.h
     include/pybind11/stl.h
-    include/pybind11/stl_bind.h)
+    include/pybind11/stl_bind.h
+    include/pybind11/stl/filesystem.h)
 
 # Compare with grep and warn if mismatched
 if(PYBIND11_MASTER_PROJECT AND NOT CMAKE_VERSION VERSION_LESS 3.12)
@@ -180,6 +188,10 @@ if(NOT TARGET pybind11_headers)
 
   target_compile_features(pybind11_headers INTERFACE cxx_inheriting_constructors cxx_user_literals
                                                      cxx_right_angle_brackets)
+  if(NOT "${PYBIND11_INTERNALS_VERSION}" STREQUAL "")
+    target_compile_definitions(
+      pybind11_headers INTERFACE "PYBIND11_INTERNALS_VERSION=${PYBIND11_INTERNALS_VERSION}")
+  endif()
 else()
   # It is invalid to install a target twice, too.
   set(PYBIND11_INSTALL OFF)
@@ -200,6 +212,12 @@ if(PYBIND11_INSTALL)
       "${CMAKE_INSTALL_DATAROOTDIR}/cmake/${PROJECT_NAME}"
       CACHE STRING "install path for pybind11Config.cmake")
 
+  if(IS_ABSOLUTE "${CMAKE_INSTALL_INCLUDEDIR}")
+    set(pybind11_INCLUDEDIR "${CMAKE_INSTALL_FULL_INCLUDEDIR}")
+  else()
+    set(pybind11_INCLUDEDIR "\$\{PACKAGE_PREFIX_DIR\}/${CMAKE_INSTALL_INCLUDEDIR}")
+  endif()
+
   configure_package_config_file(
     tools/${PROJECT_NAME}Config.cmake.in "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
     INSTALL_DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})

share/openPMD/thirdParty/pybind11/README.rst

@@ -3,7 +3,7 @@
 
 **pybind11 — Seamless operability between C++11 and Python**
 
-|Latest Documentation Status| |Stable Documentation Status| |Gitter chat| |CI| |Build status|
+|Latest Documentation Status| |Stable Documentation Status| |Gitter chat| |GitHub Discussions| |CI| |Build status|
 
 |Repology| |PyPI package| |Conda-forge| |Python Versions|
 
@@ -13,19 +13,6 @@
 
 .. start
 
-.. warning::
-
-   Combining older versions of pybind11 (< 2.6.0) with Python 3.9.0 will
-   trigger undefined behavior that typically manifests as crashes during
-   interpreter shutdown (but could also destroy your data. **You have been
-   warned.**)
-
-   We recommend that you update to the latest patch release of Python (3.9.1),
-   which includes a `fix <https://github.com/python/cpython/pull/22670>`_
-   that resolves this problem. If you do use Python 3.9.0, please update to
-   the latest version of pybind11 (2.6.0 or newer), which includes a temporary
-   workaround specifically when Python 3.9.0 is detected at runtime.
-
 
 **pybind11** is a lightweight header-only library that exposes C++ types
 in Python and vice versa, mainly to create Python bindings of existing
@@ -45,7 +32,7 @@ this heavy machinery has become an excessively large and unnecessary
 dependency.
 
 Think of this library as a tiny self-contained version of Boost.Python
-with everything stripped away that isn't relevant for binding
+with everything stripped away that isn’t relevant for binding
 generation. Without comments, the core header files only require ~4K
 lines of code and depend on Python (2.7 or 3.5+, or PyPy) and the C++
 standard library. This compact implementation was possible thanks to
@@ -101,8 +88,8 @@ goodies:
 - pybind11 uses C++11 move constructors and move assignment operators
   whenever possible to efficiently transfer custom data types.
 
-- It's easy to expose the internal storage of custom data types through
-  Pythons' buffer protocols. This is handy e.g. for fast conversion
+- It’s easy to expose the internal storage of custom data types through
+  Pythons’ buffer protocols. This is handy e.g. for fast conversion
   between C++ matrix classes like Eigen and NumPy without expensive
   copy operations.
 
@@ -119,7 +106,7 @@ goodies:
 - Binaries are generally smaller by a factor of at least 2 compared to
   equivalent bindings generated by Boost.Python. A recent pybind11
   conversion of PyRosetta, an enormous Boost.Python binding project,
-  `reported <http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf>`_
+  `reported <https://graylab.jhu.edu/Sergey/2016.RosettaCon/PyRosetta-4.pdf>`_
   a binary size reduction of **5.4x** and compile time reduction by
   **5.8x**.
 
@@ -132,7 +119,7 @@ goodies:
 Supported compilers
 -------------------
 
-1. Clang/LLVM 3.3 or newer (for Apple Xcode's clang, this is 5.0.0 or
+1. Clang/LLVM 3.3 or newer (for Apple Xcode’s clang, this is 5.0.0 or
    newer)
 2. GCC 4.8 or newer
 3. Microsoft Visual Studio 2015 Update 3 or newer
@@ -147,9 +134,9 @@ About
 This project was created by `Wenzel
 Jakob <http://rgl.epfl.ch/people/wjakob>`_. Significant features and/or
 improvements to the code were contributed by Jonas Adler, Lori A. Burns,
-Sylvain Corlay, Eric Cousineau, Ralf Grosse-Kunstleve, Trent Houliston, Axel
+Sylvain Corlay, Eric Cousineau, Aaron Gokaslan, Ralf Grosse-Kunstleve, Trent Houliston, Axel
 Huebl, @hulucc, Yannick Jadoul, Sergey Lyskov Johan Mabille, Tomasz Miąsko,
-Dean Moldovan, Ben Pritchard, Jason Rhinelander, Boris Schaeling,  Pim
+Dean Moldovan, Ben Pritchard, Jason Rhinelander, Boris Schäling, Pim
 Schellart, Henry Schreiner, Ivan Smirnov, Boris Staletic, and Patrick Stewart.
 
 We thank Google for a generous financial contribution to the continuous
@@ -189,3 +176,5 @@ to the terms and conditions of this license.
    :target: https://repology.org/project/python:pybind11/versions
 .. |Python Versions| image:: https://img.shields.io/pypi/pyversions/pybind11.svg
    :target: https://pypi.org/project/pybind11/
+.. |GitHub Discussions| image:: https://img.shields.io/static/v1?label=Discussions&message=Ask&color=blue&logo=github
+   :target: https://github.com/pybind/pybind11/discussions