Skip to content

gh-153568: Pause the GC while converting the AST to Python objects #153583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pablogsal wants to merge 1 commit into
base: main
Choose a base branch
from
+18 −0
Open

gh-153568: Pause the GC while converting the AST to Python objects #153583

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ...NEWS.d/next/Core_and_Builtins/2026-07-11-16-51-51.gh-issue-153568.ast2objgc.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Speed up :func:`ast.parse` by pausing the garbage collector while the AST is
converted to Python objects.
8 changes: 8 additions & 0 deletions Parser/asdl_c.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2117,7 +2117,15 @@ class PartingShots(StaticVisitor):
if (state == NULL) {
return NULL;
}
// Freshly converted AST nodes cannot be part of a reference cycle yet,
// so a garbage collection while building them cannot free anything of
// this tree and only pays to traverse its half-built nodes. Pause the
// collector while the conversion runs.
int gc_was_enabled = PyGC_Disable();

@maurycy maurycy Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My, probably naive, question here is what about free-threading?

There seems to be a single global (per interpret) state:

cpython/Python/gc_free_threading.c

Lines 2527 to 2539 in 9d231cb

int
PyGC_Enable(void)
{
GCState *gcstate = get_gc_state();
return _Py_atomic_exchange_int(&gcstate->enabled, 1);
}
int
PyGC_Disable(void)
{
GCState *gcstate = get_gc_state();
return _Py_atomic_exchange_int(&gcstate->enabled, 0);
}

cpython/Python/gc_free_threading.c

Lines 1656 to 1661 in 9d231cb

static GCState *
get_gc_state(void)
{
PyInterpreterState *interp = _PyInterpreterState_GET();
return &interp->gc;
}

For example: a thread disabled GC (in the window between here) but the parser would enable it again?

@pablogsal pablogsal Jul 18, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this will always be racy because is currently not possible to know if this was us or someone else doing it. This is something we need to consider when reviewing this optimization

@maurycy maurycy Jul 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a more general problem?

Quick ripgrep reveals:

cpython/Modules/_posixsubprocess.c

Lines 1063 to 1067 in 1b9fe5c

/* We need to call gc.disable() when we'll be calling preexec_fn */
if (preexec_fn != Py_None) {
need_to_reenable_gc = PyGC_Disable();
}

cpython/Modules/_posixsubprocess.c

Lines 1326 to 1329 in 1b9fe5c

if (need_to_reenable_gc) {
PyGC_Enable();
}

In reverse, maybe even more places would gain from disabling changing the default GC behaviour (disabling it?).

PyObject *result = ast2obj_mod(state, t);
if (gc_was_enabled) {
PyGC_Enable();
}

return result;
}
Expand Down
8 changes: 8 additions & 0 deletions Python/Python-ast.c
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading