Skip to content

feat: allow users to configure max cache bytes#91

Open
bruce-y wants to merge 13 commits intomainfrom
by/allow-configuration-max-age-bytes
Open

feat: allow users to configure max cache bytes#91
bruce-y wants to merge 13 commits intomainfrom
by/allow-configuration-max-age-bytes

Conversation

@bruce-y
Copy link
Copy Markdown
Contributor

@bruce-y bruce-y commented Apr 9, 2026

GC flag being disabled causes buildkit to not track last access times. It looks like the max-age option for buildctl overrides any additional options, especially when all of the last accessed values are unset. Disabling this option and only setting max cache size sidesteps this issue.

  ID									RECLAIMABLE	SIZE		LAST ACCESSED
  mgza1bb6tgrhvcb9p6qwjkd3m*                                             	true       	4.10kB		
  puqpq6frx7mn9qmmy4upjhps1*                                             	true       	8.19kB		
  u7xd5e7f1db226w56r91xdji4                                              	true       	8.22kB		
  9dv1zm7rkwd5b2otf2etfrwit                                              	true       	8.31kB		
  q57yee1rx33m6usoo805kjzbp                                              	true       	12.42kB		
  yxus1lvhnfbkgqlyveyjjypuk                                              	true       	16.50kB		
  tgir6bk9qgzuqfk5rsz2td8ru                                              	true       	16.57kB		
  qqf1ths8zwwljx2qtysegvdbi                                              	true       	20.65kB		
  72ajqijwmzmxq361ole36u5ky                                              	true       	27.72kB		
  6nihfcm6ryienxzb86lzfonwf                                              	true       	42.70kB		
  f1ruc4ze87pmsa2w6xv8r34i9                                              	true       	54.42kB		
  osjgpumxh8by8uak1x8a8d3so                                              	true       	116.89kB	
  db63u1g20lw5jmv8yx24bbkdm                                              	true       	5.00MB		
  k95bul72rhsjvl3k5gocqm53x                                              	true       	5.13MB		
  nf5fg6nhcvbesymd7r28kmxdx                                              	true       	23.39MB		
  61fvglf7bpwfxje9x5yq2atwh                                              	true       	35.50MB		
  pyu1wrhv7dlzegazwm9gxwlqk                                              	true       	117.19MB	
  znun8miypkycre8ux12w8y4sg                                              	true       	268.50MB	
  tcbb16ybnv3e2te56mf8hni4v                                              	true       	268.50MB	
  1y0crk9isg6h2klukxov5l8g0                                              	true       	268.50MB	
  27o93rnfl5td8vb6a903zukr9                                              	true       	268.50MB	

View in Codesmith
Codesmith can help with this PR — just tag @codesmith or enable autofix.

  • Autofix CI and bot reviews

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f0fd034. Configure here.

default: "168h"
cache-keep-storage:
description: "Amount of build cache to retain after pruning, in MB (e.g., 1000 for 1GB). If not set, no minimum is enforced."
required: false
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New cache config inputs are ignored during pruning

High Severity

The new cache-max-age and cache-keep-storage inputs are declared in action.yml, parsed in getInputs(), and stored in state — but maybeShutdownBuildkitd() hardcodes cacheMaxAge = null and keepStorageMB = 1000 instead of reading from stateHelper.inputs. Users who set these inputs will see no effect: cache-max-age (which defaults to "168h") is always overridden to null (no age pruning), and cache-keep-storage is always 1000 MB regardless of configuration.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit f0fd034. Configure here.

@bruce-y bruce-y changed the title allow users to configure max age or bytes allow users to configure max cache bytes Apr 9, 2026
@bruce-y bruce-y changed the title allow users to configure max cache bytes feat: allow users to configure max cache bytes Apr 9, 2026
@bruce-y bruce-y requested a review from adityamaru April 9, 2026 03:54
}

const cacheKeepStorageInput = core.getInput("cache-keep-storage");
let cacheKeepStorage = 20480;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bruce-y should we entirely skip running prune if they don't specify this? Do you anticipate any regression if we do that? Our storage agent logic already evict when we near 500GB so we don't really need to default to anything imo


/**
* Prunes buildkit cache data.
* @param keepStorageMB Storage to retain in cache, in MB. Defaults to 20480 (20GB).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

20GB is too small for dockr layer cache, this usually runs to atleast 200-300GB. imo we should default to not pruning if not specified and should maybe have docs around recommended sizes here based on our average stickydisk size.

// Disable automatic garbage collection, since we will prune manually. Automatic GC
// has been seen to negatively affect startup times of the daemon.
gc: false,
gc: true,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR's fix of setting gc: true is correct for getting access-time tracking back. One subtlety worth noting: with gc: true and no explicit gcPolicy entries in the toml, BuildKit will use its default GC policies which can be fairly aggressive (e.g., keeping cache under a few GB, evicting unused entries after 48h). If the intent is "track access times but only prune manually at shutdown," you'd want to also define very permissive gcPolicy entries in the toml to prevent the background GC from fighting with your manual prune logic. That said, if the PR is moving toward capping cache at ~20GB via --keep-storage, the default policies probably won't conflict since they're more lenient than 20GB.

from a devin session I was chatting with

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also @bruce-y can you check we have a buildkit startup time metric? As the comment suggests it would not be good to regress that, and if we see signs of that we might have to dig in further

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants