Skip to content

Adding filtering and pagination to collectionList APIs - #12436

Open
stevenwinship wants to merge 19 commits into
developfrom
12423-collection-list-search-and-pagination
Open

Adding filtering and pagination to collectionList APIs#12436
stevenwinship wants to merge 19 commits into
developfrom
12423-collection-list-search-and-pagination

Conversation

@stevenwinship

@stevenwinship stevenwinship commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it: Result lists for collections can be quite large so a way to filter and use pagination is needed.

Which issue(s) this PR closes:#12423

Special notes for your reviewer: Switched to family "A" pagination. I did not change the output from "data.items" to "data" since that would be a breaking api change and data also contains count and totalCount.

Suggestions on how to test this: See DataRetrieverApi tests. create many dataverses owned or given access to a user. Test getting the list with and without pagination. Get the list as a superuser. Filter the list with a sort term.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?: included

Additional documentation:

https://dataverse-guide--12436.org.readthedocs.build/en/12436/api/native-api.html#list-dataverse-collections-a-user-can-act-on-based-on-their-permissions

@stevenwinship stevenwinship self-assigned this Jun 4, 2026
@github-actions github-actions Bot added FY26 Sprint 24 FY26 Sprint 24 (2026-05-20 - 2026-06-03) FY26 Sprint 25 FY26 Sprint 25 (2026-06-03 - 2026-06-17) GREI Re-arch Issues related to the GREI Dataverse rearchitecture Original size: 30 Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) SPA These changes are required for the Dataverse SPA labels Jun 4, 2026
@stevenwinship stevenwinship moved this to In Progress 💻 in IQSS Dataverse Project Jun 4, 2026
@coveralls

coveralls commented Jun 4, 2026

Copy link
Copy Markdown

Coverage Status

Coverage is 25.001%12423-collection-list-search-and-pagination into develop. No base build found for develop.

@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

Test Results

404 tests   389 ✅  25m 16s ⏱️
 55 suites   15 💤
 55 files      0 ❌

Results for commit e737ac1.

♻️ This comment has been updated with latest results.

@stevenwinship stevenwinship removed their assignment Jun 4, 2026
@stevenwinship stevenwinship moved this from In Progress 💻 to Ready for Review ⏩ in IQSS Dataverse Project Jun 4, 2026
@stevenwinship stevenwinship modified the milestone: 6.12 Jun 4, 2026
@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from 5a0bf16 to c73b373 Compare June 8, 2026 13:32
@github-actions

This comment has been minimized.

@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from c73b373 to eca44a4 Compare June 9, 2026 20:07
@github-actions

This comment has been minimized.

@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from eca44a4 to 6c33308 Compare June 15, 2026 18:36
@github-actions

This comment has been minimized.

@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from 6c33308 to 98baff1 Compare June 15, 2026 21:19
@github-actions

This comment has been minimized.

@cmbz cmbz added the FY26 Sprint 26 FY26 Sprint 26 (2026-06-17 - 2026-07-01) label Jun 18, 2026
@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from 98baff1 to e166e23 Compare June 22, 2026 17:42
@github-actions

This comment has been minimized.

@cmbz cmbz added the FY27 Sprint 1 FY27 Sprint 1 (2026-07-01 - 2026-07-15) label Jul 1, 2026
@stevenwinship
stevenwinship force-pushed the 12423-collection-list-search-and-pagination branch from e166e23 to e1391a1 Compare July 2, 2026 13:14
@github-actions

This comment has been minimized.

@cmbz cmbz added the FY27 Sprint 3 FY27 Sprint 3 (2026-07-29 - 2026-08-12) label Jul 29, 2026
@github-actions

This comment has been minimized.

@pdurbin

pdurbin commented Aug 6, 2026

Copy link
Copy Markdown
Member

At standup we talked about how we have various patterns for pagination, which I wrote about here: cfc5301

Family Endpoint Request parameters Response keys
A GET /api/datasets/{id}/versions/{versionId}/files offset, limit totalCount, data
A GET /api/datasets/{id}/versions/compareSummary offset, limit totalCount, data
A GET /api/notifications/all offset, limit totalCount, data
A GET /api/files/{id}/versionDifferences offset, limit totalCount, data
B GET /api/search start, per_page total_count, start, count_in_response, items
C GET /api/admin/list-users selectedPage, itemsPerPage userCount, selectedPage, pagination{...}, users
C GET /api/access/datafile/{id}/listRequests start, per_page data, pagination{...}
D GET /api/users/{identifier}/allowedCollections/{permission} offset, pageSize count, items, pageSize, nextOffset, prevOffset
D GET /api/mydata/retrieve/collectionList offset, pageSize count, items, pageSize, nextOffset, prevOffset

@stevenwinship will try to switch to the "A" family that the frontend expects.

Also, there's a Pager.java class at https://github.com/IQSS/dataverse/blob/v6.11/src/main/java/edu/harvard/iq/dataverse/mydata/Pager.java . We might want to create a new Pager class to rule them all. 💍

@stevenwinship
stevenwinship requested a review from qqmyers August 6, 2026 20:36
@stevenwinship stevenwinship removed their assignment Aug 6, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
9.5% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@github-actions

Copy link
Copy Markdown

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:12423-collection-list-search-and-pagination
ghcr.io/gdcc/configbaker:12423-collection-list-search-and-pagination

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

@cmbz cmbz added the FY27 Sprint 4 FY27 Sprint 4 (2026-08-12 - 2026-08-26) label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

FY26 Sprint 24 FY26 Sprint 24 (2026-05-20 - 2026-06-03) FY26 Sprint 25 FY26 Sprint 25 (2026-06-03 - 2026-06-17) FY26 Sprint 26 FY26 Sprint 26 (2026-06-17 - 2026-07-01) FY27 Sprint 1 FY27 Sprint 1 (2026-07-01 - 2026-07-15) FY27 Sprint 2 FY27 Sprint 2 (2026-07-15 - 2026-07-29) FY27 Sprint 3 FY27 Sprint 3 (2026-07-29 - 2026-08-12) FY27 Sprint 4 FY27 Sprint 4 (2026-08-12 - 2026-08-26) GREI Re-arch Issues related to the GREI Dataverse rearchitecture Original size: 30 Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) SPA These changes are required for the Dataverse SPA

Projects

Status: In Review 🔎

Development

Successfully merging this pull request may close these issues.

Update collectionList API: add pagination and filtering

5 participants