[BUG] - crash on c++ exception when importing 2 pybind submodules built from different MSVC versions

[Nir-Az]

Issue description

Update: After diving deep in the pybind11 comments inside the code (Thanks God for it!) I am rephrasing the issue.

It looks like different MSVC versions that build modules with pybind11 interfere with each other (due to use of common resources between python modules).
The PYBIND11_INTERNALS_ID string is not constructed from a _MSC_VER and when I miss match modules build with MSVC 14.0
and MSVC >= 14.2 the ID is the same and I get a crash

Here is the crash stack

The "registered_exception_translators" list is not empty, but once you try to access the translator you get access violation.

I saw the "PYBIND11_COMPILER_TYPE" definition can be override,
I guess what I see here is that mismatch between MSVC versions also cause problems and perhaps should be considered as a different ABI.

Question:
What are the benefits of using the same "Internals" between modules? is it memory consumption?
It seems like if I force override the ABI version it works (add_definitions(-DPYBIND11_COMPILER_TYPE="_testing")

This is my workaround for now, I can suggest a PR that integrate MSC version as part of the ABI_ID..

Thanks

Thanks
Nir

[ibell]

Maybe this is related: #1562 ?

[Nir-Az]

Anything new regarding it?

I currently solve it by adding -DPYBIND11_COMPILER_TYPE="<unique_abi_string>" to every module I add.
BTW I get the same issue with working with 2 modules of the same MSVC.
They share resources and the objects multi_map is not thread_safe, so the modules interfere with each other.

[pwuertz]

Also got hit by this bug. The involved pybind11 modules were TensorRT, which NVidia built with MSVC 2017, and other modules built via GitHub Actions, currently on MSVC 2022. All modules end up accessing the same global state with PYBIND11_INTERNALS_ID = "__pybind11_internals_v4_msvc__", while being ABI incompatible.

This is really bad, and should be fixed ASAP or we'll keep running into ABI mismatch induced memory corruptions and crashes for the foreseeable future.

Is there a good reason why pybind11 is doing state sharing by default? I could imagine someone designing a multi-module-project with some common types, but state sharing might as well be an opt-in-feature for this specific case, not a default that is prone to all sorts of ABI pitfalls.

What about @Nir-Az's request of adding _MSC_VER to the PYBIND11_INTERNALS_ID? What's the hold-up here?