Metadata to aid static analysis

[rickeylev]

Static analysis tools need additional information to help understand Python targets correctly. The use case of particular note are type checkers.

See discussion #2455_

At a minimum, these tools need the original source files. For type checkers, pyi files are also needed. Similarly, things like proto (or other code gen) would also benefit from a way to provide pyi files.

• Add PyInfo fields: {direct, transitive} x {original_sources, pyi_files}
• Add py_library.{pyi_{srcs, deps}
• Update py_library et al to populate, merge, propagate this information
• Update pypi generation code to set attributes appropriately.

[aignas]

Just a note for future selves - sometimes whl metadata has extra dependencies for type checking that are not included by default and we could potentially leverage the same heuristics as used in #2425 - i.e. the types packages have special names that are/can be easy to detect.

One question though is what happens if the user hasn't specified the types deps in the hub repo requirements? I am somewhat concerned that this could brake bazel query if the type stubs are not present.

Thinking about this more, I have a feeling that automatically propagating stubs from the external typing stub packages is probably a separate ticket as its scope is too large. Initially (as part of this ticket) we should only handle the pyi files that are present in the whl_library repos.

[rickeylev]

All the items in the todo are done!

If downstream users could try using them / migrate to them, and provide feedback, that would be the last missing piece.

I'm going to leave this open for a bit before closing.