rules_python 1.7.0 and jupyterlab

[gfrankliu]

Since I upgraded my rules_python from 1.4.1 to 1.7.0, jupyterlab stops working. bazel run gives "No module named ipykernel_launcher" even though I have requirement("ipykernel"), in deps.

I see this in the 1.7.0 changelog:

(bootstrap) For [--bootstrap_impl=system_python](https://rules-python.readthedocs.io/en/latest/api/rules_python/python/config_settings/index.html#bootstrap_impl), PYTHONPATH is no longer used to add import paths. The sys.path order has changed from [app paths, stdlib, runtime site-packages] to [stdlib, app paths, runtime site-packages].

I suspect this is the root cause. Jupyter spawns completely new, independent OS processes (Kernels). I assume it replies on PYTHONPATH from rules_python to "find" the ipykernel module.

Is there a quick workaround to get jupyterlab working again?

[shayanhoshyari]

Or maybe #3437 ?
Solution for above is switch to modern script bootstrap.

#3473 is gonna fix system_path and issue above, but seems has some todos.

[gfrankliu]

Thanks for the link and suggestion! Is there any known gotchas switching to modern script bootstrap? or is it safe for everyone to switch?

I added common --@rules_python//python/config_settings:bootstrap_impl=script in the .bazelrc and bazel run no longer complains "No module named ipykernel_launcher" but now there are now other errors:

[I 2026-01-19 03:34:03.702 ServerApp] Kernel started: d9d54a2f-6840-4d7e-8920-d62d8a8c8f2f
Traceback (most recent call last):
  File "/home/gfrankliu/.cache/bazel/_bazel_gfrankliu/b71142cf8050563176d73cc4d125110e/execroot/_main/bazel-out/k8-fastbuild/bin/ml/notebook/kernel", line 434, in <module>
    Main()
  File "/home/gfrankliu/.cache/bazel/_bazel_gfrankliu/b71142cf8050563176d73cc4d125110e/execroot/_main/bazel-out/k8-fastbuild/bin/ml/notebook/kernel", line 374, in Main
    module_space = FindModuleSpace(main_rel_path)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gfrankliu/.cache/bazel/_bazel_gfrankliu/b71142cf8050563176d73cc4d125110e/execroot/_main/bazel-out/k8-fastbuild/bin/ml/notebook/kernel", line 209, in FindModuleSpace
    raise AssertionError('Cannot find .runfiles directory for %s' % sys.argv[0])
AssertionError: Cannot find .runfiles directory for /home/gfrankliu/.cache/bazel/_bazel_gfrankliu/b71142cf8050563176d73cc4d125110e/execroot/_main/bazel-out/k8-fastbuild/bin/ml/notebook/kernel

[shayanhoshyari]

Thanks for the link and suggestion! Is there any known gotchas switching to modern script bootstrap? or is it safe for everyone to switch

Only gotcha is no windows support.

I added common --@rules_python//python/config_settings:bootstrap_impl=script in the .bazelrc and bazel run no longer complains "No module named ipykernel_launcher" but now there are now other errors:

I am afraid without knowing your setup further, not really possible to provide any guidance.

[gfrankliu]

I am afraid without knowing your setup further, not really possible to provide any guidance.

Sorry for the false alarm. Those extra errors were due to my earlier attempts trying to fix the "No module named ipykernel_launcher". Now I cleaned all my hacks, and only keep the --@rules_python//python/config_settings:bootstrap_impl=script, all is working properly.

Thanks and I will make that the default since we don't use Windows.

[gfrankliu]

I noticed one side effect when switching to script bootstrap. I see those venv directories and symlinks. Does that mean turning on script bootstrap automatically turns on venv support?
With the new venv directory and files, we will need to modify how we pack the py image layers into the rules_oci image, so this script bootstrap switching is not as transparent as we thought :)

[shayanhoshyari]

common --@rules_python//python/config_settings:bootstrap_impl=script
common --@rules_python//python/config_settings:venvs_site_packages=no
common --@rules_python//python/config_settings:venvs_use_declare_symlink=no

Should work with rules_oci and py_image_layer

[shayanhoshyari]

Though I just ditched py_image_layer and use something like this

#3388 (comment)

This works with venv=yes too (the code I have is for 1.7.0, for 1.8.0, needs some slight updates)

[gfrankliu]

common --@rules_python//python/config_settings:bootstrap_impl=script
common --@rules_python//python/config_settings:venvs_site_packages=no
common --@rules_python//python/config_settings:venvs_use_declare_symlink=no

Should work with rules_oci and py_image_layer

I did set venvs_site_packages=no but didn't know I also need to set venvs_use_declare_symlink=no too.

[gfrankliu]

Though I just ditched py_image_layer and use something like this

#3388 (comment)

This works with venv=yes too (the code I have is for 1.7.0, for 1.8.0, needs some slight updates)

This is indeed a better option. Does it require the latest rules_pkg? I recalls seeing some patches were submitted upstream rules_pkg (bazelbuild/rules_pkg#930), not sure if it is mandatory to upgrade. Do we also need to set symlink to no? I'm using 1.7.0 so should be fine but curious to know what changes in 1.8.0 breaks it. It would be nice to have it in rules_pythons just like rules_py folks had py_image_layer.

[shayanhoshyari]

Some info that might help you. Sorry if not directly answering your question.

Ideal case: everyone use 1 from the rules_python section, and all 3 tar methods from building docker images support it. But this is open source and there are migrations blah blah, so it is more complicated.

1. Building docker image

To build a docker image, you just need some tars to pass to rules_oci or rules_img (latter is newer and much more efficient if you have large images)

1. The simplest way to make a tar is using rules_pkg. And I suspect the reason rules_python is not in the business of making tars is that rules_pkg already exists and is managed by google.
2. If you use rules_img instead of rules_oci it also has its own image_layer.
3. py_image_layer just makes a bunch of tars, and uses tar.bzl (which is rules_pkg's alternative from the company aspect.).

1 and 2 support File.is_symlink=True. Not sure if any support SymlinkEntry (at least 2 and 3 don't).

2. rules_python config

1. bootstrap_impl=script + venvs_site_packages=yes: Latest and greatest. Makes a .venv with same structure as pip and uv. Most experimental and uses SymlinkEntry and File.is_symlink=True. This is the only one that allows stuff like torch, tensorflow, ... from pip installs to work out of the box (find their .so s).
2. bootstrap_impl=script + venvs_site_packages=no: One level less change from 1. uses a empty venv with a .pth that adds dependencies to path. Only uses File.is_symlink=True.
3. bootstrap_impl=script + venvs_site_packages=no + venvs_use_declare_symlink=no: 2 with a workaround so that packaging rules that don't support symlink work. uses no symlink. The symlink from 2 is made at runtime.

[shayanhoshyari]

#3388 (comment)
This is indeed a better option.

Here is a working version https://github.com/shayanhoshyari/issue-reports/tree/main/rules_python/rules_pkg_pyvenv (it is rules_python: 1.8.0. With 1.7.0 some workarounds were needed)

Example: https://github.com/shayanhoshyari/issue-reports/blob/main/rules_python/rules_pkg_pyvenv/example/BUILD#L14-L18

[gfrankliu]

Nice, I will upgrade to 1.8.0 and give it a try.
Does the rule automatically break into three layers (tars) like the other py_image_layer: interpreter, packages and app? When modifying local "app", we don't want to rebuild the one big tar for external "packages", especially with cuda, tensorflow, etc where the "packages" layer is several GB after compression.

[gfrankliu]

Maybe too much to ask :)
Can we add "compress" option to the rule, so we can pass zstd or gzip/pigz? oci_image no longer compresses the layers so it expects the tar is already compressed.

[shayanhoshyari]

Does the rule automatically break into three layers (tars) like the other py_image_layer: interpreter, packages and app? When modifying local "app", we don't want to rebuild the one big tar for external "packages", especially with cuda, tensorflow, etc where the "packages" layer is several GB after compression.

Yes, but it is not configurable as-is. You can modify to make it configurable. Now it is configured via https://github.com/shayanhoshyari/issue-reports/blob/main/rules_python/rules_pkg_pyvenv/private/tar_runfiles.bzl#L64-L67 (breaks to main, nvidia, torch and rest of external)

Can we add "compress" option to the rule, so we can pass zstd or gzip/pigz? oci_image no longer compresses the layers so it expects the tar is already compressed.

Feel free to modify to your need. I did not since I used with rules_img and it will compress when the tar is used with their later rule.

[shayanhoshyari]

(To be honest I made the standalone example as a practice of working with gemini cli)