Error: spawn <python interpreter> ENOENT on test discovery when `"python.testing.cwd"` is set

[patrickkidd]

I have searched stackoverflow as well as GitHub issues here and found some related issues but nothing conclusive or helpful.

Environment data

• VS Code version: 1.54.3
• Extension version (available under the Extensions sidebar): v2021.3.658691958
• OS and version: macOS Big Sur 11.2.3
• Python version (& distribution if applicable, e.g. Anaconda): all
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): n/a
• Relevant/affected Python packages and their versions: pytest | unittest
• Relevant/affected Python-related VS Code extensions and their versions: vscode-python v2021.3.658691958
• Value of the python.languageServer setting: pylance

Expected behaviour

python process for test discovery would spawn successfully or produce more descriptive error.

Actual behaviour

python /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/patrick/dev/vscode-test -s --cache-clear
Test Discovery failed: 
Error: spawn /Users/patrick/.pyenv/shims/python ENOENT

Steps to reproduce:

1. Delete/move ~/.vscode for fresh vscode user env.
2. Create simple python project with one pytest test and open vscode to folder with only the python extension and pytest auto discovery enabled.
3. Observe output in Python Test Log listed below.
4. Copy/paste command line from Python Test Log into macOS terminal.
5. Observe process runs successfully and discovers pytest test.

*NOTE: I am unable to reproduce this problem on a clean system. But, it occurs for any python 3 version installed via system, brew, and pyenv regardless of whether the vscode project points to a symlink or executable.

Logs

Output for Python in the Output panel

User belongs to experiment group 'pythonaacf'
User belongs to experiment group 'pythonSendEntireLineToREPL'
User belongs to experiment group 'pythonNotDisplayLinterPrompt'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'ShowExtensionSurveyPrompt - enabled'
User belongs to experiment group 'CollectLSRequestTiming - control'
> conda --version
> conda info --json
> pyenv root
> python3.7 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3.6 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python2 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> pyenv root
> pyenv root
> pyenv root
> pyenv root
> pyenv root
> pyenv root
Starting Pylance language server.
Python interpreter path: ~/.pyenv/shims/python
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/vscode-test -s --cache-clear
cwd: api
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/vscode-test -s --cache-clear
cwd: api
Error 2021-03-20 20:11:59: Failed to parse discovered Test [Error: spawn /Users/patrick/.pyenv/shims/python ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)] {
  errno: 'ENOENT',
  code: 'ENOENT',
  syscall: 'spawn /Users/patrick/.pyenv/shims/python',
  path: '/Users/patrick/.pyenv/shims/python',
  spawnargs: [
    '/Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py',
    'discover',
    'pytest',
    '--',
    '--rootdir',
    '/Users/patrick/dev/vscode-test',
    '-s',
    '--cache-clear'
  ]
}
> conda --version
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
Error 2021-03-20 20:12:00: Python Extension: displayDiscoverStatus [TypeError: Cannot read property 'uri' of undefined
	at /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/out/client/extension.js:9:495273]

Output for Python Test Log in the Output panel

python /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/patrick/dev/vscode-test -s --cache-clear
Test Discovery failed: 
Error: spawn /Users/patrick/.pyenv/shims/python ENOENT

[karrtikr]

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority.

Thanks for your understanding and patience!

[karrtikr]

Hi Dr. Patrick Stinson (@patrickkidd) ,

It seems you're using the pyenv shims as your Python interpreter, /Users/patrick/.pyenv/shims/python. As a workaround, can you try pointing it to the actually python itself? It should probably be inside /Users/patrick/.pyenv/versions.

Also, please verify that the shim exists. ENOET is generally thrown when the command doesn't exist. (see this). Please try running /Users/patrick/.pyenv/shims/python in terminal.

[patrickkidd]

Hi Dr. Patrick Stinson (@patrickkidd) ,

It seems you're using the pyenv shims as your Python interpreter, /Users/patrick/.pyenv/shims/python. As a workaround, can you try pointing it to the actually python itself? It should probably be inside /Users/patrick/.pyenv/versions.

Also, please verify that the shim exists. ENOET is generally thrown when the command doesn't exist. (see this). Please try running /Users/patrick/.pyenv/shims/python in terminal.

I appreciate your diligence with this issue. Yes. I saw the same potential problems with shims and symlinks in my own troubleshooting and have have no luck trying with multiple interpreters installed from system, brew, and pyenv, pointing to and not pointing to symlinks.

I did try a fresh user and pyenv install on the same machine and the plugin worked. So then for my original user I cleared out my .bash_profile, pyenv, and .vscode user settings are extensions and could not get it to work. Couldn't think of anything else that would effect the plugin and/or python environment. So I got stumped there.

[patrickkidd]

I have a little extra info for this issue. This error goes away if I remove this line: from my workspace settings:

"python.testing.cwd": "api",

The "api" dir in my project repo root is where all the python is. Unfortunately I still need to set the cwd for pytest so now I have other errors related to that to deal with.

Thoughts?

[karrtikr]

I see, can you try running /Users/patrick/.pyenv/shims/python inside the api directory in terminal and see if the shim reports the same ENOET error?

[patrickkidd]

It does not. All of the shims run correctly.

[karrtikr]

I was talking with my team and it was suggested that shims needed a shell to run on. In many parts of the extension like discovery, we use child_process.spawn which does not use a shell by default (unless shell: true option is specified), and hence this fails.

So I think we do not support shims as of now.