feat: parallelize forecast usage-artifact downloads to fix 45-min timeout - #48718
Merged
Conversation
…default) Fixes the serial artifact-download bottleneck in `gh aw forecast --days 30` that caused the Daily Spending Forecast workflow to time out after 45 minutes. Changes: - Add `parallelLoadRunAICs` to forecast_compute.go: fetches AIC values for all completed runs concurrently using a semaphore (default: 8 goroutines). - Update `forecastWorkflow` to call `parallelLoadRunAICs` instead of the previous serial `forecastLoadCachedRunAIC` loop. - Add `DownloadConcurrency int` field to `ForecastConfig` and expose it via a new `--concurrency` CLI flag (0 = use default of 8). - Add tests: parallel returns all values, respects cancellation, handles empty input, and verifies the new CLI flag is registered." Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix timeout issue with daily spending forecast artifact download
feat: parallelize forecast usage-artifact downloads to fix 45-min timeout
Jul 28, 2026
Collaborator
|
@copilot run forecast in pre agent steps to predownload data |
Add a "Prefetch forecast usage artifacts" pre-agent step that runs gh aw forecast with --concurrency 8 and --timeout 25 (discarding output) before the main forecast step. Because forecastDownloadUsageArtifact skips re-downloading artifacts already on disk, the main forecast step reads from the local cache and completes in seconds. Also adds --concurrency 8 to the main forecast step and reduces its --timeout from 30 to 10 minutes since data is warm from the prefetch. Recompiled daily-spending-forecast.lock.yml. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Done. Added a "Prefetch forecast usage artifacts" step before "Run spending forecast" in The prefetch step runs |
Contributor
There was a problem hiding this comment.
Pull request overview
Parallelizes forecast artifact downloads to prevent long-running forecasts from timing out.
Changes:
- Adds bounded concurrent artifact loading and a
--concurrencyflag. - Adds concurrency and cancellation tests.
- Updates the daily forecast workflow with prefetching and shorter timeouts.
Show a summary per file
| File | Description |
|---|---|
pkg/cli/forecast_compute.go |
Implements parallel artifact loading. |
pkg/cli/forecast_command.go |
Adds concurrency configuration and flag. |
pkg/cli/forecast_test.go |
Tests parallel loading and flag registration. |
.github/workflows/daily-spending-forecast.md |
Adds artifact prefetching and concurrency settings. |
.github/workflows/daily-spending-forecast.lock.yml |
Recompiles the workflow. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 5/5 changed files
- Comments generated: 3
- Review effort level: Medium
| samples := make([]ForecastRunSample, 0, len(completed)) | ||
|
|
||
| // Download usage artifacts in parallel, then process results in run order. | ||
| aicMap := parallelLoadRunAICs(ctx, completed, config) |
Comment on lines
+494
to
+495
| got := parallelLoadRunAICs(context.Background(), runs, ForecastConfig{DownloadConcurrency: 3}) | ||
| assert.Equal(t, wantAIC, got) |
| cmd.Flags().Int("sample", 100, "Maximum number of completed runs to sample per workflow") | ||
| cmd.Flags().Bool("eval", false, "Evaluate forecast quality against past data (backtesting mode)") | ||
| cmd.Flags().Int("timeout", 0, "Gracefully stop forecast computation after this many minutes (0 = no timeout)") | ||
| cmd.Flags().Int("concurrency", 0, "Maximum number of concurrent usage-artifact downloads (0 = use default)") |
Contributor
|
🎉 This pull request is included in a new release. Release: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gh aw forecast --days 30was downloading usage artifacts for each completed workflow run serially; as run volume ingithub/gh-awgrew, the download phase alone exceeded the 45-minute job timeout with zero output produced.Changes
Parallel artifact download (
forecast_compute.go): replaces the serial per-runforecastLoadCachedRunAICcall insideforecastWorkflowwith a newparallelLoadRunAICshelper that fans out downloads across a semaphore-bounded goroutine pool (default: 8 concurrent downloads), then processes results in original run order.--concurrencyflag (forecast_command.go): exposesDownloadConcurrencyinForecastConfigand as a CLI flag (default0= use the code default of 8; pass1for serial behaviour). The constantdefaultForecastDownloadConcurrency = 8lives inforecast_compute.go.Context cancellation:
parallelLoadRunAICsrespects context cancellation at the semaphore-acquire step, so the existing--timeoutflag cleanly interrupts in-flight downloads.Tests: covers correct result collection with concurrency, graceful early exit on cancelled context, empty-input edge case, and flag registration.
gh-aw forecast --days 30times out after 45m during artifact download #48715