Skip to content

More fixes for actions that shouldn't upload to remote-cache - #1871

Merged
alexeagle merged 1 commit into
bazelbuild:masterfrom
alexeagle:master
May 24, 2021
Merged

More fixes for actions that shouldn't upload to remote-cache#1871
alexeagle merged 1 commit into
bazelbuild:masterfrom
alexeagle:master

Conversation

@alexeagle

Copy link
Copy Markdown
Contributor

Similar to #1856. I found these by patching that PR and running through a big monorepo and searching execution log for more remote cache uploads over 10MB

@gravypod gravypod left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the right direction but I think we need to define more generally what should and shouldn't be cached by our rules. We might want to also run some tests against RBE/remote cache environments like what you're likely doing here.

Similar to bazelbuild#1856. I found these by patching that PR and running through a big monorepo and searching execution log for more remote cache uploads over 10MB
@alexeagle

Copy link
Copy Markdown
Contributor Author

By the way for anyone who wants to figure out whether an execution log shows big artifacts from rules_docker, here's the command I was running. I'm excluding the mnemonics that are now no-remote-cache and see what else is over 10MB

jq --raw-output 'select(.mnemonic != "RunAndCommitLayer" and .mnemonic != "ImageLayer" and .mnemonic != "GUNZIP" and .mnemonic != "GZIP" and .mnemonic != "JoinLayers") | .actualOutputs[] | select(.digest.sizeBytes | tonumber > 10485760) | [.path,.digest.sizeBytes] | @tsv' /Users/alex.eagle/Desktop/bazel_exec.oYGl.json

@alexeagle
alexeagle merged commit e33dda0 into bazelbuild:master May 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants