Unresolved import if sys.path modified in __init__.py

[farleylai]

For projects that include external repos, it would be handy to insert the relative dependency paths ONLY when importing the necessary module.

Here is an example project layout:

Project root
+-- external repo/
      +-- lib/
           +-- config  # defines Config class
      +-- utils/
+-- mypackageA
     +-- __init__.py
     +-- app.py
main.py

In main.py, when an app instance that depends on the Config.py module in the external repo is created, the sys.path must include external repo/lib at the time of importing app:

from mypackageA import app
app.App()

Instead of setting the PYTHONPATH statically, an alternative is to modify sys.path in mypackageA.__init__.py by inserting paths to external repo/lib for the app module to import Config. However, vscode shows unresolved import warnings in this use case.

[ericsnowcurrently]

Farley Lai (@farleylai), thanks for letting us know about this. If I understood correctly, mypackageA.app.py has something like from config import Config in it. You want the import to work automatically when using the extension, where it currently doesn't. Is that right? Otherwise I'm not clear on how we can help.

To get what you want, the extension would have to automatically add external repo/lib to sys.path. However, there are 2 problems with that:

1. running main.py from the normal commandline already doesn't work like you want and likewise if you tried to distribute this project
2. how is the extension supported to know that it should add external repo/bin to PYTHONPATH?

As to the first one, this situation is not unique to the extension. I recommend using one of the existing approaches out there.

[farleylai]

Farley Lai (@farleylai), thanks for letting us know about this. If I understood correctly, mypackageA.app.py has something like from config import Config in it. You want the import to work automatically when using the extension, where it currently doesn't. Is that right? Otherwise I'm not clear on how we can help.

Eric Snow (@ericsnowcurrently)
Yes, that is what I mean the path is inserted only when importing the specific module.

To get what you want, the extension would have to automatically add external repo/lib to sys.path. However, there are 2 problems with that:

1. running main.py from the normal commandline already doesn't work like you want and likewise if you tried to distribute this project

This alternative I mentioned is to insert the path in myPackageA.__init__.py like the following:

import sys
sys.path.insert(0, 'external repo/lib')

I am saying the programmer has done this such that whenever app.py is imported, the statement from myPackage import app should have the additional path inserted. In this way, the main.py works as expected but VSCode does not seem to evaluate myPackageA.__init__.py when editing.

2. how is the extension supported to know that it should add external repo/bin to PYTHONPATH?

So my first intuition is to have the extension to evaluate myPackageA.__init__.py when editing main.py or app.py. Could that be possible?

As to the first one, this situation is not unique to the extension. I recommend using one of the existing approaches out there.

[ericsnowcurrently]

Thanks for clarifying, Farley Lai (@farleylai). I understand now and was able to quickly reproduce the problem. Based on the information you've provided thus far I was able to fill in the blanks for file content, which I've included below.

Using Jedi ("python.jediEnabled": true; the default), pylint reported the error. Using the language server ("python.jediEnabled": false), it reported the error. Either way I got the same error you reported.

You are correct that the sys.path manipulation in mypackageA/__init__.py should be reflected in all imports of mypackageA submodules. I consider this a bug. However, there is a real possibility that it is a bug in pylint/language server and will have to be resolved upstream. There may be workarounds we can do in the extension. We'll take a look.

---

# main.py

from mypackageA import app
app.App()

# mypackageA/__init__.py

import os.path
import sys
PKG_ROOT = os.path.dirname(__file__)
PROJECT_ROOT = os.path.dirname(PKG_ROOT)
LIB = os.path.join(PROJECT_ROOT, 'external repo', 'lib')
sys.path.insert(0, LIB)

# mypackageA/app.py

from config import Config  # <--- error marked here
class App:
    def __init__(self):
        cfg = Config()
    ...

# external repo/lib/config.py

class Config:
    ...