feat(intake): add canonical experiment_id filter to evaluations list [ASE-716] - #949
Merged
shanaiabuggy merged 3 commits intoJul 28, 2026
Conversation
…[ASE-716] The evaluations write path is canonically `experiment_ids`; bring the list filter in line so producers stop having to write `experiment_ids` and read back by the deprecated key. - Add a canonical `experiment_id` filter to EvaluationFilter that routes through the same group-membership match (experiment_ids $contains + legacy scalar) as before. - Mark the `experiment_group_id` filter param deprecated=True, kept as a working alias. - _rewrite_group_filter now recognizes both keys; update the list-endpoint filter description. - Add a test: filtering by experiment_id returns the same set as experiment_group_id, both resolving membership over experiment_ids. - Regenerate the OpenAPI spec. Removal of the deprecated experiment_group_id filter param is tracked in ASE-615. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (5)
📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThe evaluations filter contract now defines ChangesEvaluation filter aliasing
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
…ilter Ran `make stainless` after the EvaluationFilter change — the generated Python SDK now exposes the canonical `experiment_id` filter and marks `experiment_group_id` as a deprecated alias. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Contributor
|
BrianNewsom
approved these changes
Jul 28, 2026
Migrate the FE evaluations-list filter call sites off the deprecated experiment_group_id to the canonical experiment_id (leaderboard, Pareto chart, edit modal, session compare). The pin/unpin invalidation query-key moves in lockstep so its partial-key match keeps working. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
shanaiabuggy
enabled auto-merge
July 28, 2026 21:11
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 28, 2026
shanaiabuggy
deleted the
sbuggy/ase-716-add-canonical-experiment_id-filter-to-evaluations-list
branch
July 28, 2026 23:19
ngoncharenko
pushed a commit
that referenced
this pull request
Jul 29, 2026
…[ASE-716] (#949) * feat(intake): add canonical experiment_id filter to evaluations list [ASE-716] The evaluations write path is canonically `experiment_ids`; bring the list filter in line so producers stop having to write `experiment_ids` and read back by the deprecated key. - Add a canonical `experiment_id` filter to EvaluationFilter that routes through the same group-membership match (experiment_ids $contains + legacy scalar) as before. - Mark the `experiment_group_id` filter param deprecated=True, kept as a working alias. - _rewrite_group_filter now recognizes both keys; update the list-endpoint filter description. - Add a test: filtering by experiment_id returns the same set as experiment_group_id, both resolving membership over experiment_ids. - Regenerate the OpenAPI spec. Removal of the deprecated experiment_group_id filter param is tracked in ASE-615. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> * chore(sdk): regenerate Python SDK for the experiment_id evaluations filter Ran `make stainless` after the EvaluationFilter change — the generated Python SDK now exposes the canonical `experiment_id` filter and marks `experiment_group_id` as a deprecated alias. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> * refactor(studio): use the canonical experiment_id evaluations filter Migrate the FE evaluations-list filter call sites off the deprecated experiment_group_id to the canonical experiment_id (leaderboard, Pareto chart, edit modal, session compare). The pin/unpin invalidation query-key moves in lockstep so its partial-key match keeps working. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> --------- Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The evaluations write path is canonically
experiment_ids, but the list filter only exposed the deprecatedexperiment_group_id, forcing producers to write one key and read back by another. This brings the filter in line.experiment_idfilter toEvaluationFilterthat routes through the same group-membership match (experiment_ids $contains+ the legacy scalar) as the existing rewrite.experiment_group_idfilter paramdeprecated=True, kept as a fully-working alias (non-breaking) — mirrors how the request body deprecates itsexperiment_group_idfield._rewrite_group_filternow recognizes both keys; updated the list-endpoint filter description.Testing
test_evaluation_filter_experiment_id_matches_deprecated_alias: membership set via the canonicalexperiment_idswrite field, then assertsfilter[experiment_id]andfilter[experiment_group_id]return the same set.ruffclean; intake CRUD integration suite: 38 passed.Follow-up
Removal of the deprecated
experiment_group_idfilter param is tracked as a breaking change in ASE-615.Resolves ASE-716.
🤖 Generated with Claude Code
Summary by CodeRabbit
experiment_idfiltering for evaluations by experiment-group membership.experiment_group_idas a deprecated compatibility alias.EvaluationFilterschema to reflect the new canonical filter and aliasing.experiment_idmatches the deprecated alias behavior.