From deb08d439f80b1b0417dddbdb26ac0575512eba6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomasz=20Zdyba=C5=82?= Date: Thu, 24 Apr 2025 08:14:53 +0200 Subject: [PATCH 1/3] docs: enhance Execution API documentation with block size management details Added a section on block size management to the Execution API documentation, detailing the role of the `maxBytes` parameter in `InitChain` and `ExecuteTxs` methods. This update clarifies how the parameter aids in maintaining consistent block size constraints and its usage in Rollkit for validating block sizes before submission. --- specs/lazy-adr/adr-010-exec-api.md | 31 ++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/specs/lazy-adr/adr-010-exec-api.md b/specs/lazy-adr/adr-010-exec-api.md index 539d3e36b8..ecdd3d92b8 100644 --- a/specs/lazy-adr/adr-010-exec-api.md +++ b/specs/lazy-adr/adr-010-exec-api.md @@ -3,6 +3,7 @@ ## Changelog - 2025.01.13: Initial draft +- 2025.04.24: Various improvements ## Context @@ -155,6 +156,36 @@ The Execution API was designed to be highly generic. As a result, the types intr This design choice ensures maximum flexibility and allows the API to remain independent of specific implementations or formats. It facilitates interoperability across different execution environments while keeping the API lightweight and adaptable. +### Block Size Management + +The Execution API includes `maxBytes` as a return value in both `InitChain` and `ExecuteTxs` methods. This parameter plays a crucial role in block size management: + +1. **Initial Configuration**: During `InitChain`, the execution environment returns the maximum block size allowed by its genesis configuration. This value is used by Rollkit to enforce block size limits during block production. + +2. **Dynamic Adjustment**: In `ExecuteTxs`, the execution environment can return an updated `maxBytes` value. This allows for dynamic adjustment of block size limits based on: + - Network conditions + - Resource availability + - Protocol-specific requirements + - Other runtime factors + +3. **Implementation Requirements**: + - The execution environment must ensure that blocks produced do not exceed the returned `maxBytes` value + - If a block would exceed the limit, the execution environment should return an error + - The value should be consistent with the execution environment's capabilities and constraints + - A protocol overhead buffer is subtracted from the DA layer's max blob size to account for block headers and encoding + +4. **System-wide Coordination**: + - Rollkit gets the initial `maxBytes` value from the DA layer and passes it to the sequencer + - The sequencer uses this value to limit the size of transaction batches it creates + - The sequencer's `PopUpToMaxBytes` method ensures transactions don't exceed the size limit + - This coordination ensures consistent block size constraints across the entire system + - Both Rollkit and sequencer independently validate block sizes to prevent oversized blocks + +5. **Usage in Rollkit**: + - Rollkit uses this value to validate block sizes before submission + - Helps prevent oversized blocks from being produced + - Enables dynamic adjustment of block size limits without protocol changes + ### Sequence Diagrams #### Block production From 3276640068b3a327a9db4cc7198ed78214bf6116 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomasz=20Zdyba=C5=82?= Date: Thu, 24 Apr 2025 08:23:56 +0200 Subject: [PATCH 2/3] docs: add more details to ADR-010 --- specs/lazy-adr/adr-010-exec-api.md | 98 +++++++++++++++++------------- 1 file changed, 55 insertions(+), 43 deletions(-) diff --git a/specs/lazy-adr/adr-010-exec-api.md b/specs/lazy-adr/adr-010-exec-api.md index ecdd3d92b8..ab04df5133 100644 --- a/specs/lazy-adr/adr-010-exec-api.md +++ b/specs/lazy-adr/adr-010-exec-api.md @@ -10,11 +10,26 @@ Introduction of the Execution API makes rollkit very generic and execution-environment agnostic. It removes all ABCI-centric code for full interoperability with other types of VMs. +The Execution API serves as a bridge between Rollkit and various execution environments (VMs), +allowing Rollkit to remain agnostic to the specific implementation details of the execution layer. +This separation enables: + +1. Support for multiple VM types (EVM, WASM, etc.) +2. Easier integration with different execution environments +3. Cleaner separation of concerns between consensus and execution +4. More flexible and maintainable architecture + ## Alternative Approaches 1. Maintain current state: keep ABCI interface and implement other VMs inside ABCI application. + - Pros: No changes required to existing code + - Cons: ABCI-specific code remains, limiting flexibility 2. Migrate to Engine API. + - Pros: Standard interface for EVM-based chains + - Cons: Too specific to EVM, not suitable for other VMs 3. Create new API generic enough to handle any arbitrary VM. + - Pros: Maximum flexibility, clean separation of concerns + - Cons: Requires significant refactoring ## Decision @@ -70,6 +85,7 @@ Initializes the blockchain's state based on genesis information. This method is - Initialize rollup according to the genesis. - Generate an initial `stateRoot` representing the genesis state of the rollup. - Return the maximum allowable block size (`maxBytes`). +- Ensure all necessary state is initialized for subsequent operations. #### `GetTxs` @@ -92,6 +108,7 @@ Transactions returned by execution client will be passed by rollkit to sequencer - Access the mempool and retrieve all available transactions. - If no transactions are available, return an empty slice without error. - Do not remove ("reap") transactions from mempool. +- Ensure transactions are valid and properly formatted. #### `ExecuteTxs` @@ -120,6 +137,7 @@ Executes a given set of transactions, updating the blockchain state. - Enforce block size and validity limits, returning errors if constraints are violated. - Respect the ordering of transactions. - Update the mempool to remove all executed transactions. +- Ensure atomic execution - either all transactions succeed or none do. #### `SetFinal` @@ -140,12 +158,16 @@ Marks a block at the specified height as final, guaranteeing immutability for co - Update the execution client's internal state to reflect that the specified block is final and immutable. - Ensure additional guarantees like cleaning up unnecessary resources associated with blocks deemed final. +- Prevent any modifications to finalized blocks. +- Optimize storage for finalized blocks if possible. #### General Notes 1. **Thread-Safety**: All methods are not expected to be thread-safe, concurrent calls are not planned. 2. **Error Handling**: All methods should follow robust error handling practices, ensuring meaningful errors are returned when issues occur. 3. **Context Usage**: Methods should respect context-based deadlines and cancellations for long-running operations. +4. **State Management**: The execution environment is responsible for maintaining its own state and ensuring consistency. +5. **Atomicity**: Operations that modify state should be atomic - either fully succeed or fully fail. ### Types @@ -186,6 +208,32 @@ The Execution API includes `maxBytes` as a return value in both `InitChain` and - Helps prevent oversized blocks from being produced - Enables dynamic adjustment of block size limits without protocol changes +### Implementation Guidelines + +1. **State Management**: + - Execution environments must maintain their own state + - State transitions should be atomic + - State should be persisted appropriately + - State should be recoverable after crashes + +2. **Error Handling**: + - Return meaningful error messages + - Handle context cancellations gracefully + - Ensure proper cleanup on errors + - Maintain state consistency even after errors + +3. **Performance Considerations**: + - Optimize for common operations + - Consider caching where appropriate + - Handle large state sizes efficiently + - Minimize unnecessary state transitions + +4. **Security**: + - Validate all inputs thoroughly + - Prevent unauthorized state modifications + - Ensure proper access control + - Handle sensitive data appropriately + ### Sequence Diagrams #### Block production @@ -236,49 +284,6 @@ sequenceDiagram end ``` -### Changes in Rollkit - -Because of improved separation of concerns, logic related to actual execution can be removed from rollkit. -Rollkit will turn from fully-fledged ABCI client to kind of orchestrator, gluing together multiple modules. -The role of Rollkit is coordination of Sequencer and Execution Environment and recording results in Data Availability layer. - -Currently, Rollkit exposes ABCI and CometBFT compatible RPCs, manages genesis processing and mempool. -This logic belongs to Execution Environment, and will be moved to Execution API implementations. - -Removal of this logic from Rollkit enables further refactoring, for example removing all CometBFT dependencies. - -#### Main changes in Rollkit packages - -##### `block` - -1. New processing in `Manager` - move from ABCI to Execution API oriented processing. - -##### `node` - -1. Simplification of `Node` interface. -2. Removal of `LightClient` and `FullClient`. -3. Cleanup/simplification of `FullNode` and `LightNode`. - -##### `rpc` - -1. Removal of ABCI and CometBFT methods. -2. Introduction of Rollkit specific methods. - -##### Other packages - -Probably all the packages will be affected by cleanup and refactoring. - -### Testing - -As there is a lot of logic to be removed, corresponding tests can also be removed. -Some of the other test are very ABCI-oriented and should be fixed or re-created. - -Integration tests should be introduced to ensure that entire flow (from `InitChain` to `SetFinal`) works correctly. - -### User documentation - -All the tutorials needs to be updated to accommodate introduction of new binary (Execution API implementation). - ## Status Accepted @@ -290,16 +295,23 @@ Accepted 1. Simplification of rollkit's logic. 2. Better separation of concerns. 3. Removal of ABCI dependencies. +4. Increased flexibility for different execution environments. +5. Cleaner architecture with well-defined boundaries. ### Negative 1. More difficult deployment (another binary is needed). 2. Need to reimplement ABCI execution environment. +3. Additional complexity in coordinating between components. +4. Potential performance overhead from additional abstraction layer. ### Neutral 1. Need to introduce new API exposed by rollkit. +2. Changes to existing deployment procedures. +3. Updates to documentation and tooling required. ## References - [Rollkit EPIC for Execution API](https://github.com/rollkit/rollkit/issues/1802) +- [go-execution repository](https://github.com/rollkit/go-execution) From 88586a167d46423f9caed9a2635d9f7ab9604401 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomasz=20Zdyba=C5=82?= Date: Mon, 28 Apr 2025 14:05:00 +0200 Subject: [PATCH 3/3] docs: address review comments --- specs/lazy-adr/adr-010-exec-api.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/specs/lazy-adr/adr-010-exec-api.md b/specs/lazy-adr/adr-010-exec-api.md index ab04df5133..1d745f41e7 100644 --- a/specs/lazy-adr/adr-010-exec-api.md +++ b/specs/lazy-adr/adr-010-exec-api.md @@ -27,9 +27,6 @@ This separation enables: 2. Migrate to Engine API. - Pros: Standard interface for EVM-based chains - Cons: Too specific to EVM, not suitable for other VMs -3. Create new API generic enough to handle any arbitrary VM. - - Pros: Maximum flexibility, clean separation of concerns - - Cons: Requires significant refactoring ## Decision @@ -201,7 +198,6 @@ The Execution API includes `maxBytes` as a return value in both `InitChain` and - The sequencer uses this value to limit the size of transaction batches it creates - The sequencer's `PopUpToMaxBytes` method ensures transactions don't exceed the size limit - This coordination ensures consistent block size constraints across the entire system - - Both Rollkit and sequencer independently validate block sizes to prevent oversized blocks 5. **Usage in Rollkit**: - Rollkit uses this value to validate block sizes before submission