Skip to content

feat: Support el_storage_type flag#1257

Merged
barnabasbusa merged 6 commits intoethpandaops:mainfrom
MysticRyuujin:archive-full-support
Dec 2, 2025
Merged

feat: Support el_storage_type flag#1257
barnabasbusa merged 6 commits intoethpandaops:mainfrom
MysticRyuujin:archive-full-support

Conversation

@MysticRyuujin
Copy link
Contributor

@MysticRyuujin MysticRyuujin commented Dec 1, 2025 •
edited
Loading

This PR adds el_storage_type which can be set to either full or archive while maintaining the existing (and client default) settings.

I updated the README (including some markdown lint fixes and spelling issues)

I also added test which currently have .norun appended.

I tested these changes locally with:

kurtosis run --enclave archive-storage . --args-file .github/tests/archive-storage.yaml.norun
kurtosis run --enclave full-storage . --args-file .github/tests/full-storage.yaml.norun

To the best of my knowledge ethereumjs, ethrex, and nimbus-eth1 cannot be ran as archive nodes, but that's documented in the README as well.

Per external feedback from @barnabasbusa I added sanity checks into input_parser.star to fail when either:

  • el_storage_type is set to archive for ethereumjs, ethrex, and nimbus
  • el_storage_type is set to archive for shadowfork networks

@MysticRyuujin MysticRyuujin marked this pull request as ready for review December 2, 2025 16:05
Copilot AI review requested due to automatic review settings December 2, 2025 16:05
@barnabasbusa barnabasbusa merged commit 2eb1e85 into ethpandaops:main Dec 2, 2025
18 checks passed
@EthPandaOpsBot EthPandaOpsBot mentioned this pull request Dec 1, 2025
Merged
@MysticRyuujin MysticRyuujin deleted the archive-full-support branch December 2, 2025 16:08
Copilot AI reviewed Dec 2, 2025
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces support for configuring Ethereum execution layer (EL) client storage types via the el_storage_type parameter, allowing users to explicitly choose between "full" and "archive" node modes. This addresses the need for fine-grained control over node storage behavior, which has different space and performance tradeoffs.

  • Adds el_storage_type parameter that accepts "full", "archive", or empty string (for default behavior)
  • Implements client-specific storage configuration for Geth, Reth, Erigon, Nethermind, and Besu
  • Adds validation to prevent unsupported configurations (archive mode with shadowfork networks or unsupported clients)
  • Updates documentation with comprehensive guidance on storage type behavior and volume size considerations

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/package_io/sanity_check.star Adds el_storage_type to allowed participant parameters for validation
src/package_io/input_parser.star Adds parameter definition, validation logic for shadowfork/archive conflicts and unsupported clients, and passes parameter to participant struct
src/el/reth/reth_launcher.star Configures Reth to use --full flag when el_storage_type is "full" (Reth defaults to archive)
src/el/nethermind/nethermind_launcher.star Configures Nethermind to use --Pruning.Mode=None when el_storage_type is "archive" (Nethermind defaults to full)
src/el/geth/geth_launcher.star Updates Geth archive detection to include el_storage_type check alongside existing el_extra_params checks
src/el/erigon/erigon_launcher.star Configures Erigon to use --prune.mode=full when el_storage_type is "full" (Erigon defaults to archive)
src/el/besu/besu_launcher.star Configures Besu to use --data-storage-format=FOREST for archive mode and adjusts sync mode logic
README.md Adds comprehensive documentation for el_storage_type, fixes markdown formatting issues, and updates various sections for clarity
examples/README.md Adds proper markdown heading for improved formatting
.github/tests/full-storage.yaml.norun Adds test configuration for clients that default to archive (reth, erigon) running in full mode
.github/tests/archive-storage.yaml.norun Adds test configuration for clients that default to full (geth, nethermind, besu) running in archive mode

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/package_io/input_parser.star
for idx, participant in enumerate(result["participants"]):
el_type = participant["el_type"]
el_storage_type = participant["el_storage_type"]

Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing validation for invalid el_storage_type values. The code should validate that el_storage_type is either empty string, "full", or "archive". Currently, if a user provides an invalid value like "partial" or "medium", it would be silently ignored, leading to unexpected behavior.

Consider adding validation like:

if el_storage_type != "" and el_storage_type not in ["full", "archive"]:
    fail(
        "Participant {0}: Invalid el_storage_type '{1}'. Must be either 'full' or 'archive' (or empty string for default behavior).".format(
            idx, el_storage_type
        )
    )
Suggested change
# Validate el_storage_type value
if el_storage_type != "" and el_storage_type not in ["full", "archive"]:
fail(
"Participant {0}: Invalid el_storage_type '{1}'. Must be either 'full' or 'archive' (or empty string for default behavior).".format(
idx, el_storage_type
)
)

Copilot uses AI. Check for mistakes.
README.md
el_tolerations: []

# Persistent storage size for the EL client container (in MB)
# IMPORTANT: Consider settings this if you are setting el_storage_type
Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "settings" should be "setting" for grammatical correctness.

Suggested change
# IMPORTANT: Consider settings this if you are setting el_storage_type
# IMPORTANT: Consider setting this if you are setting el_storage_type

Copilot uses AI. Check for mistakes.
README.md
# If this is emptystring, each client will use its default behavior:
# - reth, erigon: default to archive (use "full" to save space)
# - geth, besu, nethermind: default to full (use "archive" to keep historical data)
# - ethereumjs, ethrex, nimbus-eth1: unused (full only?)
Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The uncertainty indicator "(full only?)" could be more definitive for production documentation. Consider either:

  1. Confirming the behavior and removing the question mark: "unused (full mode only)"
  2. Being explicit about the uncertainty: "unused (these clients do not currently support archive mode configuration)"
  3. Adding a reference to where users can learn more

This would make the documentation more authoritative and helpful to users.

Suggested change
# - ethereumjs, ethrex, nimbus-eth1: unused (full only?)
# - ethereumjs, ethrex, nimbus-eth1: unused (these clients do not currently support archive mode configuration)

Copilot uses AI. Check for mistakes.
barnabasbusa pushed a commit that referenced this pull request Jan 5, 2026
@EthPandaOpsBot
chore(main): release 6.0.0 (#1013)
b992b41 
🤖 I have created a release *beep* *boop*
---


##
[6.0.0](5.0.1...6.0.0)
(2026-01-05)


### ⚠ BREAKING CHANGES

* geth genesis flag, osaka time passthrough for public networks
([#1229](#1229))
* remove mev_flood
([#1091](#1091))

### Features

* Add 'trace' to enabled JSON-RPC modules
([#1235](#1235))
([ea241af](ea241af))
* add `depends_on` to store service files for parallel
([#1208](#1208))
([98206c8](98206c8))
* add `env` parameter to Blockscout configuration for custom environment
variables
([#1262](#1262))
([627619c](627619c))
* add api to dora
([#1120](#1120))
([9dbde5a](9dbde5a))
* add bpo
([#1016](#1016))
([37082b2](37082b2))
* add chainspec support for fusaka-devnet-2
([#1055](#1055))
([0e18733](0e18733))
* add checkpointz
([#1254](#1254))
([c9d72f9](c9d72f9))
* Add cl_devices parameter for mounting host devices to CL containers
([#1251](#1251))
([ab9e55f](ab9e55f))
* add client-language label to ethereum service containers
([#1074](#1074))
([6955763](6955763))
* add custody group and getBlobsV2 metrics on PeerDAS dashboard
([#982](#982))
([e43e569](e43e569))
* add debug port for ethjs
([#1044](#1044))
([459a931](459a931))
* add disk usage as a metric for ethereum-metrics-exporter
([#1026](#1026))
([8e793a5](8e793a5))
* add el genesis files to dora config for extended blob gas display
([#1180](#1180))
([2fd2b41](2fd2b41))
* add engine snooper urls to dora config for block execution time
tracking
([#1083](#1083))
([7ffb9d5](7ffb9d5))
* Add eRPC integration
([#1223](#1223))
([16b72b7](16b72b7))
* add ethrex execution client
([#1131](#1131))
([82e5a71](82e5a71))
* add extra labels
([#1030](#1030))
([12447f8](12447f8))
* add genesis_time as a possible network_param config value
([#1216](#1216))
([b11ce6f](b11ce6f))
* Add Geth dashboard to the grafana module
([#1166](#1166))
([47c90f6](47c90f6))
* Add Kurtosis config for BALs devnet 0
([#1197](#1197))
([9a93b33](9a93b33))
* add log level to ethrex
([#1269](#1269))
([57af564](57af564))
* add maxblobspertx
([#1063](#1063))
([1944080](1944080))
* add mempool-bridge service integration
([#1222](#1222))
([92d2239](92d2239))
* add mev-fulu test
([#1014](#1014))
([67a76e9](67a76e9))
* add MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS as config option
([#1057](#1057))
([97afe9c](97afe9c))
* add more groups to spamoor clients
([#1066](#1066))
([c9ed485](c9ed485))
* add multiple bn nodes per vc
([#1189](#1189))
([7727330](7727330))
* add name and nameoverride func to clients page spamoor
([#1068](#1068))
([e3abf47](e3abf47))
* add new timing parameters
([#1168](#1168))
([1d524c8](1d524c8))
* add nginx file server implementation
([#1065](#1065))
([fd76bba](fd76bba))
* add node index label
([#1086](#1086))
([5aa0d44](5aa0d44))
* add node selectors and tolerations to run_sh
([#1167](#1167))
([c4e0c89](c4e0c89))
* add OTLP collector URL to Lighthouse validator client
([#1252](#1252))
([f507360](f507360))
* add peercount support for nimbusel
([#1092](#1092))
([12409e4](12409e4))
* add per participant checkpoint sync enabled flag
([#1243](#1243))
([2101448](2101448))
* add public ports for mev
([#1023](#1023))
([5d89274](5d89274))
* add public ports for other tools
([#1025](#1025))
([fa9d05e](fa9d05e))
* add sanity check for lack of supernodes
([#1145](#1145))
([70dd011](70dd011))
* add sanity check for perfect peerdas
([#1217](#1217))
([6c51752](6c51752))
* add skip_start
([#1253](#1253))
([338bb88](338bb88))
* Add support for dummy EL in kurtosis config
([#1276](#1276))
([b8007fd](b8007fd))
* add support for extra mounts for CL, EL, and VC clients
([#1136](#1136))
([d385265](d385265))
* add support for MIN_EPOCHS_FOR_BLOCK_REQUESTS
([#1211](#1211))
([17ad84a](17ad84a))
* add support for separate bootnode with bootnodoor
([#1238](#1238))
([f8f4de6](f8f4de6))
* Add support for the helix relay
([#1237](#1237))
([e17cb60](e17cb60))
* add telemetry service name flag to Lighthouse
([#1160](#1160))
([2f61b9c](2f61b9c))
* add Tempo as an additional service to collect Lighthouse tracing data
([#1150](#1150))
([ba328bb](ba328bb))
* add tolerations
([#1137](#1137))
([a4b52da](a4b52da))
* add tx snooper
([#1043](#1043))
([34e1151](34e1151))
* add validator balance
([#1032](#1032))
([3601346](3601346))
* add validator ranges for devnets
([#1176](#1176))
([3fb5084](3fb5084))
* add validator summary dora
([#1177](#1177))
([f289914](f289914))
* **ai:** Add docs
([#1061](#1061))
([1bf0893](1bf0893))
* allow passing custom env vars to the genesis generator
([#1227](#1227))
([a43368e](a43368e))
* allow specifying additional mnemonics
([#1267](#1267))
([dad4ea3](dad4ea3))
* automatically generate a 2/3 ratio for target/max blobs
([#1156](#1156))
([2d1aa15](2d1aa15))
* bump egg (fulu genesis support)
([#1140](#1140))
([601df3b](601df3b))
* configure Blockscout to index from shadowfork block height
([#1221](#1221))
([a1347fe](a1347fe))
* default to ethpandaops/client:devnet images
([#1097](#1097))
([fa4f99a](fa4f99a))
* enable `custom_preset` in checkpointz config
([#1259](#1259))
([8e9913b](8e9913b))
* enable extra env vars to be set during runtime mev, enable pprof by
default
([#1012](#1012))
([94a7f22](94a7f22))
* enable mass das guardian scans in dora
([#1125](#1125))
([0671925](0671925))
* enable prom and grafana to be ran separatly
([#1028](#1028))
([500c3f0](500c3f0))
* enable rpc proxy in dora
([#1212](#1212))
([4de44ce](4de44ce))
* enable tty for prysm
([#1076](#1076))
([1ae1826](1ae1826))
* enable validator block on sentry
([#1224](#1224))
([0f61746](0f61746))
* Extra Files for `*_extra_mounts` support
([#1144](#1144))
([1b889f6](1b889f6))
* feature flag for DisableFinalizedRootCheck
([#1228](#1228))
([c51f183](c51f183))
* fine grained control with public ip addresses per service
([#1111](#1111))
([3f60fa8](3f60fa8))
* make default node a supernode
([#1230](#1230))
([802c045](802c045))
* remove maxBlobsPerTx
([#1113](#1113))
([9f40d0a](9f40d0a))
* remove mev_flood
([#1091](#1091))
([2d3b170](2d3b170))
* rename eip7732 to gloas
([#1157](#1157))
([f0c5522](f0c5522))
* set fulu fork epoch at genesis
([#1261](#1261))
([6ae2474](6ae2474))
* sps setting Qu0b/nethermind sps
([#1225](#1225))
([969a707](969a707))
* Support el_storage_type flag
([#1257](#1257))
([2eb1e85](2eb1e85))
* use dns names instead of ip addresses for services
([#1194](#1194))
([f360a51](f360a51))


### Bug Fixes

* add fulu fork version for mev-boost-relay
([#1088](#1088))
([953ec57](953ec57))
* add input option for blobber
([#1072](#1072))
([293286d](293286d))
* assertoor image for fulu support
([#1240](#1240))
([b0f4fab](b0f4fab))
* besu sync snap if non kurtosis
([#1034](#1034))
([6752218](6752218))
* **blockscout:** make frontend available in kubernetes
([#1033](#1033))
([d3ae571](d3ae571))
* bump egg,fix minimal preset
([#1165](#1165))
([0f877c6](0f877c6))
* bump ethereum-genesis-generator to fix issues with large additional
contracts
([#1019](#1019))
([cb644af](cb644af))
* change default images
([#1099](#1099))
([ba92830](ba92830))
* change lh supernode flag
([#1186](#1186))
([f64ff38](f64ff38))
* change nimbus supernode flag
([#1275](#1275))
([094b3f3](094b3f3))
* cl node discovery on k8s
([#1162](#1162))
([5643dfd](5643dfd))
* cleanup besu
([#1139](#1139))
([5001427](5001427))
* commit boost cb-config
([#1233](#1233))
([87f3e03](87f3e03))
* commit boost integration
([#1204](#1204))
([69e60b3](69e60b3))
* default to empty blob schedule if non defined
([#1115](#1115))
([35c298d](35c298d))
* default to pandaops ethrex image for arm/amd
([#1249](#1249))
([b03a571](b03a571))
* disable page cache in dora
([#1079](#1079))
([1e51446](1e51446))
* dora,assertoor pull through cache
([#1059](#1059))
([69c965f](69c965f))
* downgrade teku from latest to master
([#1155](#1155))
([996c2a1](996c2a1))
* el/cl/vc index calculation bug, due to parallel execution
([#1121](#1121))
([fc4e65e](fc4e65e))
* enable submission pages in dora
([#1031](#1031))
([33e3f7b](33e3f7b))
* ensure proper bpo scheduling
([#1266](#1266))
([57120bf](57120bf))
* erigon db size alloc
([#1096](#1096))
([777d37e](777d37e))
* failed to start network: ethereum-package execution error: Evaluation
error: key osaka_time not in dict
([#1218](#1218))
([ae74385](ae74385))
* fix checkpointz params override
([#1258](#1258))
([9518b72](9518b72))
* geth genesis flag, osaka time passthrough for public networks
([#1229](#1229))
([d58cab7](d58cab7))
* geth peering bug
([#1133](#1133))
([bc62c0c](bc62c0c))
* gloas minimal config values
([#1250](#1250))
([2ec3a94](2ec3a94))
* helix logging type
([#1279](#1279))
([756bfdd](756bfdd))
* **lighthouse:** allow genesis sync when checkpoint sync isn't enabled
([#1192](#1192))
([4053331](4053331))
* make sure builder cl is supernode
([#1188](#1188))
([dfef921](dfef921))
* mev rbuilder remove unused config param
([#1248](#1248))
([3838a5f](3838a5f))
* minimal builds should use latest unstable branches
([#1174](#1174))
([d6d6d5f](d6d6d5f))
* minimal spec
([#1037](#1037))
([2372550](2372550))
* missing dns_name
([#1274](#1274))
([3238be5](3238be5))
* move bootnodoor to additional_services
([#1264](#1264))
([bf40917](bf40917))
* nethermind chainspec, default genesis gas to 60M
([#1039](#1039))
([b839e61](b839e61))
* network params default images
([#1213](#1213))
([33a0db2](33a0db2))
* nimbus checkpoint syncing
([#1181](#1181))
([d464295](d464295))
* only add --target-peers=0 only when the network is kurtosis
([#1119](#1119))
([572cbfc](572cbfc))
* only append blob schedule, if defined
([#1022](#1022))
([43db03a](43db03a))
* override bpo1,2
([#1196](#1196))
([836cbb8](836cbb8))
* pass gas limit to ethrex if network gas_limit was specified
([#1232](#1232))
([39ac09b](39ac09b))
* prysm gzip encoding bug
([#1112](#1112))
([9f5fc45](9f5fc45))
* pull kurtosis images in kurtosis install
([#1048](#1048))
([a00b6dd](a00b6dd))
* rbuilder parallel safe sorting
([#1046](#1046))
([ec5895d](ec5895d))
* readme for additional services
([#1270](#1270))
([ba855e0](ba855e0))
* readme/CI jobs
([#1263](#1263))
([ca6b7d2](ca6b7d2))
* remove default basefee fraction
([#1143](#1143))
([d29e0bf](d29e0bf))
* remove graffiti
([#1082](#1082))
([ee4fff4](ee4fff4))
* remove unused env
([#1153](#1153))
([ea73a95](ea73a95))
* revert prometheus branch
([#1024](#1024))
([1559386](1559386))
* sanity check for all subfields
([#1130](#1130))
([3d2c71c](3d2c71c))
* service ports
([#1021](#1021))
([e83a1ad](e83a1ad))
* set deploy_client_group for mev related uniswap spammer
([#1195](#1195))
([09d09b0](09d09b0))
* set miner gasprice for geth if running kt'
([#1027](#1027))
([161fc14](161fc14))
* sf for erigon/geth post fulu
([#1183](#1183))
([e964e30](e964e30))
* shadowfork enclave edits
([#1070](#1070))
([63689ec](63689ec))
* shadowfork latest bug
([#1045](#1045))
([197cdf8](197cdf8))
* shadowfork upstream to eth-clients
([#1047](#1047))
([7c11a34](7c11a34))
* single-node lighthouse startup issue
([#1073](#1073))
([6d29b3a](6d29b3a))
* some tests
([#1190](#1190))
([fca81b3](fca81b3))
* specify devnet size for persistent flag
([#1054](#1054))
([b4c398c](b4c398c))
* update custom-network to --network
([#1159](#1159))
([1f57a7b](1f57a7b))
* Update mainnet yaml
([#1069](#1069))
([288919b](288919b))
* Update sf to osaka
([#1105](#1105))
([59579bb](59579bb))
* update some tests
([#1122](#1122))
([9488046](9488046))
* use default dora image for fulu networks
([#1128](#1128))
([b1f4e5c](b1f4e5c))
* use reth-rbuilder image as the default mev_builder_image in
network_params.yaml
([#1077](#1077))
([f07f3b6](f07f3b6))
* use self hosted runners
([#1100](#1100))
([2fc4a3c](2fc4a3c))
* use ubuntu-latest
([#1078](#1078))
([d209af4](d209af4))
* validator client compatibility update
([#1114](#1114))
([b826cc9](b826cc9))
* yeet unused mev builder
([#1056](#1056))
([40767fe](40767fe))
* yeet-7907
([#1116](#1116))
([93c6630](93c6630))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@barnabasbusa barnabasbusa barnabasbusa approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MysticRyuujin @barnabasbusa