Skip to content

Commit e194415

Browse files
rickeylevrickeylev
committed
refactor(pystar): load (but don't use) Starlark implementation.
Always loading the code provides several benefits: * It's easier to reason about what code paths are taken. * Iteratively working on them is simply changing an environment variable instead of editing several files. * Ensures the files are loadable on older versions of Bazel. Usage of the Starlark implemenation is controlled by an environment variable, `RULES_PYTHON_USE_PYSTAR=1`. An environment variable must be used the decision on what implemenation to use must be made before a regular build flag is able to run. The Starlark implementation is almost entirely compatible with pre-Bazel 7, except for the `py_internal` symbol. This symbol is special in a couple ways: * It only exists within the `@rules_python` repo * It does not exist prior to Bazel 7. This requires using a repo rule, `@rules_python_internal`, to do some feature/version detection to generate a shim bzl file so that the `py_internal` symbol is always loadable. Regular rules_python code then loads the shim and can act accordingly. Work towards #1069
1 parent a07f300 commit e194415

33 files changed

+471
-47
lines changed

‎.bazelignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ bazel-rules_python
66
bazel-bin
77
bazel-out
88
bazel-testlogs
9+
# Prevent the convenience symlinks within the examples from being
10+
# treated as directories with valid BUILD files for the main repo.
911
examples/bzlmod/bazel-bzlmod
12+
examples/bzlmod/other_module/bazel-other_module
1013
examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation
14+
examples/pip_parse/bazel-pip_parse
1115
examples/py_proto_library/bazel-py_proto_library

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,11 @@ A brief description of the categories of changes:
3232
* (multi-version) The `distribs` attribute is no longer propagated. This
3333
attribute has been long deprecated by Bazel and shouldn't be used.
3434

35+
* Calling `//python:repositories.bzl#py_repositories()` is required. It has
36+
always been documented as necessary, but it was possible to omit it in certain
37+
cases. An error about `@rules_python_internal` means the `py_repositories()`
38+
call is missing in `WORKSPACE`.
39+
3540

3641
### Added
3742

‎MODULE.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ internal_deps = use_extension("@rules_python//python/extensions/private:internal
1515
internal_deps.install()
1616
use_repo(
1717
internal_deps,
18+
"rules_python_internal",
1819
# START: maintained by 'bazel run //tools/private:update_pip_deps'
1920
"pypi__build",
2021
"pypi__click",

‎examples/build_file_generation/WORKSPACE‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,11 @@ local_repository(
7171
path = "../../gazelle",
7272
)
7373

74-
# Next we load the toolchain from rules_python.
75-
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
74+
# Next we load the setup and toolchain from rules_python.
75+
load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains")
76+
77+
# Perform general setup
78+
py_repositories()
7679

7780
# We now register a hermetic Python interpreter rather than relying on a system-installed interpreter.
7881
# This toolchain will allow bazel to download a specific python version, and use that version

‎gazelle/WORKSPACE‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,9 @@ local_repository(
3434
path = "..",
3535
)
3636

37-
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
37+
load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains")
38+
39+
py_repositories()
3840

3941
python_register_toolchains(
4042
name = "python39",

‎internal_setup.bzl‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,13 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
2020
load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
2121
load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS")
2222
load("//python/pip_install:repositories.bzl", "pip_install_dependencies")
23+
load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility
2324

2425
def rules_python_internal_setup():
2526
"""Setup for rules_python tests and tools."""
2627

28+
internal_config_repo()
29+
2730
# Because we don't use the pip_install rule, we have to call this to fetch its deps
2831
pip_install_dependencies()
2932

‎python/BUILD.bazel‎

Lines changed: 30 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,11 @@ bzl_library(
8484
bzl_library(
8585
name = "py_binary_bzl",
8686
srcs = ["py_binary.bzl"],
87-
deps = ["//python/private:util_bzl"],
87+
deps = [
88+
"//python/private:util_bzl",
89+
"//python/private/common:py_binary_macro_bazel_bzl",
90+
"@rules_python_internal//:rules_python_config_bzl",
91+
],
8892
)
8993

9094
bzl_library(
@@ -101,19 +105,31 @@ bzl_library(
101105
bzl_library(
102106
name = "py_info_bzl",
103107
srcs = ["py_info.bzl"],
104-
deps = ["//python/private:reexports_bzl"],
108+
deps = [
109+
"//python/private:reexports_bzl",
110+
"//python/private/common:providers_bzl",
111+
"@rules_python_internal//:rules_python_config_bzl",
112+
],
105113
)
106114

107115
bzl_library(
108116
name = "py_library_bzl",
109117
srcs = ["py_library.bzl"],
110-
deps = ["//python/private:util_bzl"],
118+
deps = [
119+
"//python/private:util_bzl",
120+
"//python/private/common:py_library_macro_bazel_bzl",
121+
"@rules_python_internal//:rules_python_config_bzl",
122+
],
111123
)
112124

113125
bzl_library(
114126
name = "py_runtime_bzl",
115127
srcs = ["py_runtime.bzl"],
116-
deps = ["//python/private:util_bzl"],
128+
deps = [
129+
"//python/private:util_bzl",
130+
"//python/private/common:py_runtime_macro_bzl",
131+
"@rules_python_internal//:rules_python_config_bzl",
132+
],
117133
)
118134

119135
bzl_library(
@@ -125,13 +141,21 @@ bzl_library(
125141
bzl_library(
126142
name = "py_runtime_info_bzl",
127143
srcs = ["py_runtime_info.bzl"],
128-
deps = ["//python/private:reexports_bzl"],
144+
deps = [
145+
"//python/private:reexports_bzl",
146+
"//python/private/common:providers_bzl",
147+
"@rules_python_internal//:rules_python_config_bzl",
148+
],
129149
)
130150

131151
bzl_library(
132152
name = "py_test_bzl",
133153
srcs = ["py_test.bzl"],
134-
deps = ["//python/private:util_bzl"],
154+
deps = [
155+
"//python/private:util_bzl",
156+
"//python/private/common:py_test_macro_bazel_bzl",
157+
"@rules_python_internal//:rules_python_config_bzl",
158+
],
135159
)
136160

137161
# NOTE: Remember to add bzl_library targets to //tests:bzl_libraries

‎python/extensions/private/internal_deps.bzl‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,11 @@
99
"Python toolchain module extension for internal rule use"
1010

1111
load("//python/pip_install:repositories.bzl", "pip_install_dependencies")
12+
load("//python/private:internal_config_repo.bzl", "internal_config_repo")
1213

1314
# buildifier: disable=unused-variable
1415
def _internal_deps_impl(module_ctx):
16+
internal_config_repo()
1517
pip_install_dependencies()
1618

1719
internal_deps = module_extension(

‎python/private/BUILD.bazel‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,11 @@ licenses(["notice"])
2222

2323
filegroup(
2424
name = "distribution",
25-
srcs = glob(["**"]) + ["//python/private/proto:distribution"],
25+
srcs = glob(["**"]) + [
26+
"//python/private/common:distribution",
27+
"//python/private/proto:distribution",
28+
"//tools/build_defs/python/private:distribution",
29+
],
2630
visibility = ["//python:__pkg__"],
2731
)
2832

‎python/private/common/BUILD.bazel‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,194 @@
1111
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
14+
15+
load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
16+
17+
package(
18+
default_visibility = ["//python:__subpackages__"],
19+
)
20+
21+
bzl_library(
22+
name = "attributes_bazel_bzl",
23+
srcs = ["attributes_bazel.bzl"],
24+
)
25+
26+
bzl_library(
27+
name = "attributes_bzl",
28+
srcs = ["attributes.bzl"],
29+
deps = [
30+
":common_bzl",
31+
":providers_bzl",
32+
":py_internal_bzl",
33+
":semantics_bzl",
34+
],
35+
)
36+
37+
bzl_library(
38+
name = "cc_helper_bzl",
39+
srcs = ["cc_helper.bzl"],
40+
deps = [":py_internal_bzl"],
41+
)
42+
43+
bzl_library(
44+
name = "common_bazel_bzl",
45+
srcs = ["common_bazel.bzl"],
46+
deps = [
47+
":common_bzl",
48+
":providers_bzl",
49+
":py_internal_bzl",
50+
"@bazel_skylib//lib:paths",
51+
],
52+
)
53+
54+
bzl_library(
55+
name = "common_bzl",
56+
srcs = ["common.bzl"],
57+
deps = [
58+
":cc_helper_bzl",
59+
":providers_bzl",
60+
":py_internal_bzl",
61+
":semantics_bzl",
62+
],
63+
)
64+
65+
filegroup(
66+
name = "distribution",
67+
srcs = glob(["**"]),
68+
)
69+
70+
bzl_library(
71+
name = "providers_bzl",
72+
srcs = ["providers.bzl"],
73+
deps = [
74+
":semantics_bzl",
75+
"@rules_python_internal//:rules_python_config_bzl",
76+
],
77+
)
78+
79+
bzl_library(
80+
name = "py_binary_macro_bazel_bzl",
81+
srcs = ["py_binary_macro_bazel.bzl"],
82+
deps = [
83+
":common_bzl",
84+
":py_binary_rule_bazel_bzl",
85+
],
86+
)
87+
88+
bzl_library(
89+
name = "py_binary_rule_bazel_bzl",
90+
srcs = ["py_binary_rule_bazel.bzl"],
91+
deps = [
92+
":attributes_bzl",
93+
":py_executable_bazel_bzl",
94+
":semantics_bzl",
95+
"@bazel_skylib//lib:dicts",
96+
],
97+
)
98+
99+
bzl_library(
100+
name = "py_executable_bazel_bzl",
101+
srcs = ["py_executable_bazel.bzl"],
102+
deps = [
103+
":attributes_bazel_bzl",
104+
":common_bazel_bzl",
105+
":common_bzl",
106+
":providers_bzl",
107+
":py_executable_bzl",
108+
":py_internal_bzl",
109+
":semantics_bzl",
110+
],
111+
)
112+
113+
bzl_library(
114+
name = "py_executable_bzl",
115+
srcs = ["py_executable.bzl"],
116+
deps = [
117+
":attributes_bzl",
118+
":cc_helper_bzl",
119+
":common_bzl",
120+
":providers_bzl",
121+
":py_internal_bzl",
122+
"@bazel_skylib//lib:dicts",
123+
],
124+
)
125+
126+
bzl_library(
127+
name = "py_internal_bzl",
128+
srcs = ["py_internal.bzl"],
129+
deps = ["@rules_python_internal//:py_internal_bzl"],
130+
)
131+
132+
bzl_library(
133+
name = "py_library_bzl",
134+
srcs = ["py_library.bzl"],
135+
deps = [
136+
":attributes_bzl",
137+
":common_bzl",
138+
":providers_bzl",
139+
":py_internal_bzl",
140+
"@bazel_skylib//lib:dicts",
141+
],
142+
)
143+
144+
bzl_library(
145+
name = "py_library_macro_bazel_bzl",
146+
srcs = ["py_library_macro_bazel.bzl"],
147+
deps = [":py_library_rule_bazel_bzl"],
148+
)
149+
150+
bzl_library(
151+
name = "py_library_rule_bazel_bzl",
152+
srcs = ["py_library_rule_bazel.bzl"],
153+
deps = [
154+
":attributes_bazel_bzl",
155+
":common_bazel_bzl",
156+
":common_bzl",
157+
":py_library_bzl",
158+
],
159+
)
160+
161+
bzl_library(
162+
name = "py_runtime_macro_bzl",
163+
srcs = ["py_runtime_macro.bzl"],
164+
deps = [":py_runtime_rule_bzl"],
165+
)
166+
167+
bzl_library(
168+
name = "py_runtime_rule_bzl",
169+
srcs = ["py_runtime_rule.bzl"],
170+
deps = [
171+
":attributes_bzl",
172+
":common_bzl",
173+
":providers_bzl",
174+
":py_internal_bzl",
175+
"@bazel_skylib//lib:dicts",
176+
"@bazel_skylib//lib:paths",
177+
],
178+
)
179+
180+
bzl_library(
181+
name = "py_test_macro_bazel_bzl",
182+
srcs = ["py_test_macro_bazel.bzl"],
183+
deps = [
184+
":common_bazel_bzl",
185+
":py_test_rule_bazel_bzl",
186+
],
187+
)
188+
189+
bzl_library(
190+
name = "py_test_rule_bazel_bzl",
191+
srcs = ["py_test_rule_bazel.bzl"],
192+
deps = [
193+
":attributes_bzl",
194+
":common_bzl",
195+
":py_executable_bazel_bzl",
196+
":semantics_bzl",
197+
"@bazel_skylib//lib:dicts",
198+
],
199+
)
200+
201+
bzl_library(
202+
name = "semantics_bzl",
203+
srcs = ["semantics.bzl"],
204+
)

0 commit comments

Comments
 (0)