-
Notifications
You must be signed in to change notification settings - Fork 255
test: testapp bench #3214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: testapp bench #3214
Changes from all commits
fad4f5f
1fc283c
5581431
b8dc969
a8c9ee4
544c67b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,27 +1,75 @@ | ||
| # Test Application | ||
|
|
||
| This application serves as an example to help you understand how to create your own application using this framework. It provides a basic implementation that you can use as a reference when building your own solution. | ||
| Reference implementation of a key-value store rollup using ev-node. Includes a KV executor, HTTP server for transaction submission, and a stress test tool targeting 10M req/s. | ||
|
|
||
| ## Important Note | ||
| ## Build | ||
|
|
||
| When implementing your own application, it's your responsibility to provide the appropriate Executor dependency. This is a crucial component that needs to be properly implemented according to your specific requirements. | ||
| ```bash | ||
| # Build the testapp binary | ||
| go build -o testapp . | ||
|
|
||
| # Build the stress test tool | ||
| go build -o stress-test ./kv/bench/ | ||
| ``` | ||
|
|
||
| ## Installation | ||
| ## Quick Start | ||
|
|
||
| To install and test the application, you can use the following command: | ||
| You need 3 terminals: one for the local DA, one for the testapp node, and one for the stress test. | ||
|
|
||
| ```bash | ||
| go build . | ||
| # Terminal 1: Start local DA (defaults to localhost:7980) | ||
| go run ../../tools/local-da | ||
|
|
||
| # Terminal 2: Initialize and start the testapp | ||
| ./testapp init --evnode.node.aggregator --evnode.signer.passphrase_file examples/passphrase.txt | ||
| ./testapp start --kv-endpoint localhost:9090 --evnode.node.aggregator --evnode.signer.passphrase_file examples/passphrase.txt | ||
|
|
||
| # Terminal 3: Run the stress test | ||
| ./stress-test --addr localhost:9090 --duration 10s --workers 10000 | ||
| ``` | ||
|
|
||
| This will build and install all necessary dependencies for running the test application. | ||
| ## Commands | ||
|
|
||
| | Command | Description | | ||
| | ------------------ | ------------------------------------- | | ||
| | `testapp init` | Initialize configuration and genesis | | ||
| | `testapp start` | Run the node (aliases: `run`, `node`) | | ||
| | `testapp rollback` | Rollback state by one height | | ||
| | `testapp version` | Show version info | | ||
| | `testapp keys` | Manage signing keys | | ||
| | `testapp net-info` | Get info from a running node via RPC | | ||
|
|
||
| ### Key Flags for `start` | ||
|
|
||
| ## Usage | ||
| | Flag | Description | | ||
| | -------------------------- | ------------------------------------------------- | | ||
| | `--kv-endpoint <addr>` | Enable the KV HTTP server (e.g. `localhost:9090`) | | ||
| | `--evnode.node.aggregator` | Run as aggregator (block producer) | | ||
| | `--evnode.node.block_time` | Block interval (default `1s`) | | ||
| | `--evnode.da.address` | DA layer address | | ||
| | `--home <dir>` | Data directory (default `~/.testapp`) | | ||
|
|
||
| This is a reference implementation. Feel free to explore the code and use it as a starting point for your own application. Make sure to: | ||
| ## HTTP Endpoints | ||
|
|
||
| When `--kv-endpoint` is set, the following endpoints are available: | ||
|
|
||
| | Method | Path | Description | | ||
| | ------ | --------------- | --------------------------------------------------- | | ||
| | POST | `/tx` | Submit a transaction (`key=value` body) | | ||
| | GET | `/kv?key=<key>` | Retrieve latest value for a key | | ||
| | GET | `/store` | List all key-value pairs | | ||
| | GET | `/stats` | Get injected/executed tx counts and blocks produced | | ||
|
|
||
| ## Stress Test | ||
|
|
||
| ```bash | ||
| ./stress-test --addr localhost:9090 --duration 10s --workers 1000 | ||
| ``` | ||
|
|
||
| 1. Review the existing code structure | ||
| 2. Understand how the Executor is implemented | ||
| 3. Adapt the implementation to your specific needs | ||
| | Flag | Default | Description | | ||
| | ----------- | ---------------- | ---------------------- | | ||
| | `-addr` | `localhost:9090` | Server host:port | | ||
| | `-duration` | `10s` | Test duration | | ||
| | `-workers` | `1000` | Concurrent TCP workers | | ||
|
|
||
| Remember that this is just an example, and your actual implementation might require different approaches depending on your use case. | ||
| The test sends transactions via raw persistent TCP connections, reports live RPS, and prints a summary table with avg/peak req/s, server-side block stats, and whether the 10M req/s goal was reached. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| foo:bar | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,62 +1,51 @@ | ||
| # KV Executor Benchmark Client | ||
| # KV Executor Stress Test | ||
|
|
||
| This is a command-line client primarily used for benchmarking the KV executor HTTP server by sending transactions. It can also list the current state of the store. | ||
| Stress test for the KV executor HTTP server. Sends transactions via raw TCP connections to maximize throughput, targeting 10M req/s. | ||
|
|
||
| ## Building | ||
|
|
||
| ```bash | ||
| go build -o txclient | ||
| go build -o stress-test ./kv/bench/ | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| The client runs a transaction benchmark by default. | ||
|
|
||
| ### Running the Benchmark | ||
| Run the testapp first, then the stress test alongside it: | ||
|
|
||
| ```bash | ||
| ./txclient [flags] | ||
| ``` | ||
|
|
||
| By default, the benchmark runs for 30 seconds, sending 10 random key-value transactions every second to `http://localhost:40042`. | ||
|
|
||
| **Benchmark Flags:** | ||
|
|
||
| * `-duration <duration>`: Total duration for the benchmark (e.g., `1m`, `30s`). Default: `30s`. | ||
| * `-interval <duration>`: Interval between sending batches of transactions (e.g., `1s`, `500ms`). Default: `1s`. | ||
| * `-tx-per-interval <int>`: Number of transactions to send in each interval. Default: `10`. | ||
| * `-addr <url>`: Specify a different server address. Default: `http://localhost:40042`. | ||
|
|
||
| **Transaction Data for Benchmark:** | ||
|
|
||
| * **Random Data (Default):** If no transaction data flags are provided, the client sends random `key=value` transactions, where keys are 8 characters and values are 16 characters long. | ||
| * **Fixed Key/Value:** Use `-key mykey -value myvalue` to send the *same* transaction `mykey=myvalue` repeatedly during the benchmark. | ||
| * **Fixed Raw Data:** Use `-raw "myrawdata"` to send the *same* raw transaction data repeatedly during the benchmark. | ||
| # Terminal 1: start the testapp with KV endpoint | ||
| ./build/testapp start --kv-endpoint localhost:9090 | ||
|
|
||
| ### List all key-value pairs in the store | ||
|
|
||
| ```bash | ||
| ./txclient -list [-addr <url>] | ||
| # Terminal 2: run the stress test | ||
| ./stress-test --addr localhost:9090 --duration 10s --workers 1000 | ||
|
Comment on lines
+16
to
+20
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Line 17 runs 🤖 Prompt for AI Agents |
||
| ``` | ||
|
|
||
| This will fetch and display all key-value pairs currently in the KV executor's store. It does not run the benchmark. | ||
| ## Flags | ||
|
|
||
| ## Examples | ||
| | Flag | Default | Description | | ||
| |------|---------|-------------| | ||
| | `-addr` | `localhost:9090` | Server host:port | | ||
| | `-duration` | `10s` | Test duration | | ||
| | `-workers` | `1000` | Number of concurrent TCP workers | | ||
| | `-target-rps` | `10000000` | Target requests per second (goal) | | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is this transaction submission or requests on the api?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it is tx execution, how fast the kv executor is able to process txs. |
||
|
|
||
| Run a 1-minute benchmark sending 20 random transactions every 500ms: | ||
| ## Output | ||
|
|
||
| ```bash | ||
| ./txclient -duration 1m -interval 500ms -tx-per-interval 20 | ||
| ``` | ||
| The tool prints live progress (req/s) during the run, then a summary table with: | ||
|
|
||
| Run a 30-second benchmark repeatedly sending the transaction `user1=alice`: | ||
| - Avg req/s and peak req/s | ||
| - Total requests, successes, and failures | ||
| - Server-side stats: blocks produced, txs executed, avg txs per block | ||
| - Whether the 10M req/s goal was reached (displays `SUCCESS` if so) | ||
|
|
||
| ```bash | ||
| ./txclient -duration 30s -key user1 -value alice | ||
| ``` | ||
| ## /stats Endpoint | ||
|
|
||
| List all values from a specific server: | ||
| The KV executor HTTP server exposes a `/stats` endpoint (GET) returning: | ||
|
|
||
| ```bash | ||
| ./txclient -list -addr http://192.168.1.100:40042 | ||
| ```json | ||
| { | ||
| "injected_txs": 52345678, | ||
| "executed_txs": 1234567, | ||
| "blocks_produced": 1000 | ||
| } | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just make running the command faster. I can just remove it from being committed