numpy: Request for user-defined types? (beyond structured dtypes and dtype=object)

[EricCousineau-TRI]

I realized that I didn't create an upstream issue, so I'm placing this here.

Background for NumPy

Basically, NumPy can allow to do custom dtypes through 3 different mechanisms (that I'm aware of):

1. Structured Types (pybind11 documentation)
2. dtype=object - see this PR: Add ability to create object matrices #1152
3. User-defined dtypes: See NumPy issue: User-defined dtype (non-PyObject) in C needing destructors / deallocation? numpy/numpy#10721 (there may be better docs, but I need to find 'em...)

The focus of this issue supporting (3).

(TODO for myself: Replace above issue links with NumPy / pybind11 docs)

Current State for pybind11

pybind11 itself already supports (1):
https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#structured-types

Supporting (2) was done by Robin Deits, we have refined it in our RobotLocomotion/pybind11 fork, and have made an upstream PR to pybind11 in #1152.

Supporting (3) is a bit of a finicky problem, as alluded in numpy/numpy#10721.
Motivated by RobotLocomotion/drake#8116, I made an old prototype / integration branch here: RobotLocomotion/drake#8452

Related pybind11 issues / PRs:

• How to expose custom type derived from pure python type #1170
• Add ability to create object matrices #1152 (How to create a numpy array of objects? #647)

I also have not revisited this in about ~1.5 years, may come back around to it.

FTR @eric-wieser has provided some suggestions and workarounds on some of the related threads. I'll admit that I haven't yet had time to process them to a full extent, and will post here once I am able. (Sorry Eric!)