test(formatters): skip combo fixtures gracefully when Hub returns 429 - #1195
Merged
jakelorocco merged 4 commits intoJun 3, 2026
Conversation
_resolve_yaml() hard-errors when obtain_io_yaml() fails (e.g. 429 rate-limit on the /revision/main ref-resolution endpoint), causing fixture-setup errors for every YamlJsonCombo that downloads its YAML from Hub. On a bad Hub day this produces 66+ ERROR entries and masks real test failures. Catch LocalEntryNotFoundError and requests.exceptions.RequestException and convert to pytest.skip(), matching the pattern already used by test_adapter_versions_unchanged. Tests that need Hub will skip cleanly instead of hard-erroring when Hub is unavailable. Closes generative-computing#1193 Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Applies the same LocalEntryNotFoundError/RequestException guard to the obtain_io_yaml call in test_read_yaml so a Hub outage produces a skip rather than a CI-breaking FAIL. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
The import must sit after pytest.importorskip("torch") so that
environments without the hf extra skip the file cleanly at
collection time rather than raising ModuleNotFoundError.
Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Makes CI skip summaries immediately diagnosable — e.g. "LocalEntryNotFoundError: ..." vs "RequestException: ..." — without having to read the full message body. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
ajbozarth
approved these changes
Jun 3, 2026
ajbozarth
left a comment
ajbozarth
left a comment
Contributor
There was a problem hiding this comment.
IIUC this LGTM as long as the CI passes and doesn't time out
planetf1
enabled auto-merge
June 3, 2026 19:14
jakelorocco
approved these changes
Jun 3, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jun 3, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Issue
Fixes #1193
Description
When
obtain_io_yaml(..., revision="main")is called,huggingface_hubfirst hits the Hub ref-resolution endpoint (/api/models/{repo}/revision/main) to resolve the branch tip to a commit SHA. Under load, this endpoint returns 429. With no local snapshot cached for the failed revision, the library raisesLocalEntryNotFoundError— and because this happens inside pytest fixture setup (_resolve_yaml()), a single 429 produces ERROR entries for every test that depends on that fixture (~66 on a bad Hub day). That swamps CI output and masks real failures.Two changes:
_resolve_yaml()— wrapobtain_io_yamlin atry/except (LocalEntryNotFoundError, requests.exceptions.RequestException)and callpytest.skip(). Each fixture that needs Hub will skip cleanly when Hub is unavailable instead of hard-erroring and cascading.test_read_yaml— apply the same guard to its directobtain_io_yamlcall, so a Hub outage produces a skip rather than a CI-breaking FAIL.Together these mean: Hub unavailability → skips, never failures. CI only fails on real test breakage. Matches the pattern already used by
test_adapter_versions_unchanged, which "silently tolerates network/auth failures so this test doesn't false-fail when run offline".Skip messages include the exception class name for easier triage, e.g.:
Testing
Attribution
Adding a new component, requirement, sampling strategy, or tool?