import errors with native namespace packages

[wochinge]

Hi,

we have a code base with native namespace packages.
It has the following structure:

src/
    subpackage_a
        namespace
            # no __init__.py here!
            subpackage_a
    subpackage_b
        namespace
            # no __init__.py here!
            subpackage_b

When I run pytype --keep-going src/subpackage_a/namespace, the type checking fails with a ton of import errors.

As soon as I add __init__.py files to the namespace packages (can be empty or pkgutil style), and then run the same command, the pytype check succeeds. The same happens if I run pytype -k src/subpackage_a/namespace/subpackage_a.

After adding the __init__.py files, I have the following structure:

src/
    subpackage_a
        namespace
            __init__.py
            subpackage_a
    subpackage_b
        namespace
            __init__.py
            subpackage_b

pytype version: 2019.11.27
python version: 3.7.5
OS: MacOS catalina

I know this is probably hard to reproduce for you, but I'd really appreciate any hints on that. Thanks a lot for your help!

[rchen152]

This happens because pytype uses the presence of __init__.py to find packages:

pytype/pytype/tools/environment.py

Line 63 in 158e593

def compute_pythonpath(filenames):

. To work around the issue, you can either add empty __init__.py files or pass pytype the paths to your packages with the pythonpath argument (https://github.com/google/pytype#usage - see the example under the "Config File" subsection).

[wochinge]

Hi @rchen152 ,

thanks for your help!

To work around the issue, you can either add empty init.py files

I can't do this due to the native namespace packages. I'm already working around it by using the pkgutil style so that pytype works and the packing works, too.

pass pytype the paths to your packages with the pythonpath argument (https://github.com/google/pytype#usage - see the example under the "Config File" subsection).

I ran pytype -k -P src/subpackage_a/namespace/:src/subpackage_b/namespace src/subpackage_a/namespace as well as different variations of it, and it still brings up the import errors. My guess would be that pytype completely changes the order in which it goes through the files?

[rchen152]

Hmm, are the import errors for files in your own packages or for external dependencies? (You can run pytype with --unresolved to get a list of the imports that pytype can't find.)

Something else you can try is to run pytype with --verbosity=2 with the __init__.py files present and look at the logging output to see what pythonpath pytype is using, then try passing that in explicitly.

[wochinge]

@rchen152 Thanks that did the trick!

For others:

pytype --keep-going -P src/subpackage_a src/subpackage_a/namespace

[eltoder]

Hi @rchen152

Is it possible to reconsider this and add support for native namespace packages? They have been a standard python feature since python 3.3, so for about 10 years now. If I understand correctly, pytype aims to support python 3.9-3.10, but this 3.3 feature is missing.