Skip to content

Fix RepositoryVersion.content query failing when >65K units#6784

Merged
dkliban merged 1 commit into
pulp:mainfrom
gerrod3:max-param-query-workaround
Jul 15, 2025
Merged

Fix RepositoryVersion.content query failing when >65K units#6784
dkliban merged 1 commit into
pulp:mainfrom
gerrod3:max-param-query-workaround

Conversation

@gerrod3

@gerrod3 gerrod3 commented Jul 14, 2025

Copy link
Copy Markdown
Contributor

fixes: #6772

This is the orm magic breaks down to this sql:

SELECT *
FROM "core_content" 
WHERE "core_content"."pulp_id" IN (
    SELECT unnest(U0."content_ids") AS "cids" 
    FROM "core_repositoryversion" U0
    WHERE U0."pulp_id" = 01980713-3ed0-7f9a-b1f5-20e774f746ae
)

@gerrod3 gerrod3 force-pushed the max-param-query-workaround branch from 8d3e504 to 231fad6 Compare July 14, 2025 07:54

return content_qs.filter(pk__in=self._get_content_ids())
content_ids = self._get_content_ids()
if isinstance(content_ids, list) and len(content_ids) >= 65535:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about making a constant from this number, e.g. POSTGRESQL_MAX_PARAMS?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From reading this I think it's a hard-limit

Comment thread pulpcore/app/models/repository.py

@dkliban dkliban left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i tested this patch on our production instance. i was able to publish a repository with 267K units. It took a little less than 5 minutes!

@dkliban dkliban merged commit b6aaa23 into pulp:main Jul 15, 2025
13 checks passed
@patchback

patchback Bot commented Jul 15, 2025

Copy link
Copy Markdown

Backport to 3.83: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.83/b6aaa2398d779caeba439938dcddcf43a5490569/pr-6784

Backported as #6786

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

publish task fails for repository with 267000 RPMs

4 participants