Skip to content

test_importlib: restore builtins after import#154276

Open
kddnewton wants to merge 1 commit into
python:mainfrom
kddnewton:fix-test-util
Open

test_importlib: restore builtins after import#154276
kddnewton wants to merge 1 commit into
python:mainfrom
kddnewton:fix-test-util

Conversation

@kddnewton

Copy link
Copy Markdown

Our CI (https://github.com/facebookincubator/cinder/actions/runs/29382592631/job/87249270535) was failing on test_pickle and test_pickletool with:

PicklingError: Can't pickle
<class 'importlib._bootstrap.BuiltinImporter'>:
it's not the same object as
importlib._bootstrap.BuiltinImporter

I believe this is happening because we run multiple test modules in the same interpreter, so the following sequence happens:

  • test_importlib.util.import_importlib() imports a source copy of importlib while blocking _frozen_importlib. During this import, importlib._bootstrap._setup() initializes import metadata on existing built-in modules. If builtins.__loader__ or builtins.__spec__ was originally absent, the source copy installs its own BuiltinImporter.
  • Although import_fresh_module() restores sys.modules, it does not restore attributes mutated on existing module objects. Consequently, builtins.__loader__ continues to reference the temporary source BuiltinImporter, while importlib._bootstrap.BuiltinImporter resolves to the restored frozen class.
  • Pickle serializes classes by module and qualified name and verifies that the resolved global is the same object. The two BuiltinImporter class objects therefore cause the identity check to fail.

To fix this we instead snapshot __loader__ and __spec__ before importing importlib and then restore them to their original values after.

Our CI (https://github.com/facebookincubator/cinder/actions/runs/29382592631/job/87249270535)
was failing on test_pickle and test_pickletool with:

```
PicklingError: Can't pickle
<class 'importlib._bootstrap.BuiltinImporter'>:
it's not the same object as
importlib._bootstrap.BuiltinImporter
```

I believe this is happening because we run multiple test
modules in the same interpreter, so the following sequence
happens:

* `test_importlib.util.import_importlib()` imports a source copy of
  `importlib` while blocking `_frozen_importlib`. During this import,
  `importlib._bootstrap._setup()` initializes import metadata on existing
  built-in modules. If `builtins.__loader__` or `builtins.__spec__` was
  originally absent, the source copy installs its own `BuiltinImporter`.
* Although `import_fresh_module()` restores `sys.modules`, it does not restore
  attributes mutated on existing module objects. Consequently,
  `builtins.__loader__` continues to reference the temporary source
  `BuiltinImporter`, while `importlib._bootstrap.BuiltinImporter` resolves to
  the restored frozen class.
* Pickle serializes classes by module and qualified name and verifies that the
  resolved global is the same object. The two `BuiltinImporter` class objects
  therefore cause the identity check to fail.

To fix this we instead snapshot `__loader__` and `__spec__` before importing
`importlib` and then restore them to their original values after.
@bedevere-app

bedevere-app Bot commented Jul 20, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@python-cla-bot

python-cla-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@StanFromIreland

Copy link
Copy Markdown
Member

Can this go under #152659?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants