Skip to content

feat: initial implementation of SPV client in rust-dashcode#75

Merged
QuantumExplorer merged 77 commits into
v0.40-devfrom
feat/dash-spv
Jun 17, 2025
Merged

feat: initial implementation of SPV client in rust-dashcode#75
QuantumExplorer merged 77 commits into
v0.40-devfrom
feat/dash-spv

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Jun 17, 2025

Copy link
Copy Markdown
Member

feat: Add Complete dash-spv SPV Client + Modular Wallet Architecture

Summary

This PR introduces a brand new dash-spv SPV client implementation (26,000+ lines) along with a modular wallet architecture. The dash-spv crate provides a complete SPV (Simplified Payment Verification) client for the Dash network with advanced features like filter synchronization, masternode management, and wallet functionality. Additionally, wallet functionality has been extracted into standalone key-wallet components with cross-platform FFI bindings.

Key Features Added

Complete dash-spv SPV Client - Full-featured SPV client with 26,000+ lines of new code
Advanced Filter Synchronization - BIP157 compact block filters with gap detection and auto-restart
Wallet Integration - UTXO tracking, address monitoring, and transaction processing
Masternode Support - Synchronization and validation of Dash masternode lists
Multi-peer Networking - Robust P2P networking with connection management and message routing
Comprehensive Storage - Both memory and disk-based storage with segmented architecture

New dash-spv Architecture

Client Layer:

  • High-level DashSpvClient with comprehensive configuration options
  • Block processing with dedicated worker threads and async task handling
  • Message handling with request tracking and timeout management
  • Filter synchronization with flow control and progress reporting
  • Wallet utilities and watch manager for address monitoring

Network Layer:

  • Multi-peer connection management with exclusive peer mode
  • TCP connection handling with handshake protocols
  • Peer discovery and connection pooling
  • Message routing and network monitoring
  • Address v2 support and peer persistence

Storage Layer:

  • Memory and disk-based storage implementations
  • Segmented storage for headers (10,000-header segments)
  • Separate storage for filter headers and compact filters
  • State persistence and consistency management
  • Reverse indexing for efficient lookups

Synchronization Layer:

  • Header synchronization with validation
  • Filter header and compact filter sync coordination
  • Masternode list synchronization
  • Interleaved sync processes with state management
  • Timeout handling and recovery mechanisms

Validation Layer:

  • Header validation with proof-of-work verification
  • ChainLock validation for enhanced security
  • InstantLock validation for InstantSend transactions
  • Comprehensive validation framework

Wallet Layer:

  • UTXO tracking with in-memory cache and disk persistence
  • Transaction processing and balance calculations
  • Address indexing and monitoring
  • Watch item management for payment detection

Core Library Enhancements

Enhanced Special Transactions:

  • Improved coinbase payload handling with version-specific parsing
  • Enhanced ProTx parsing with BLS version support
  • Updated provider registration/update service transactions
  • Better error handling and validation

Network Protocol Improvements:

  • Updated network constants and genesis block configuration
  • Enhanced masternode list and quorum management
  • Improved message handling and serialization

Summary by CodeRabbit

  • New Features

    • Introduced a modular, async Dash SPV client library with support for block header sync, BIP157 filter sync, masternode list sync, ChainLock and InstantLock validation, wallet monitoring, and persistent storage.
    • Added a command-line interface for configuring and running the client, including terminal UI status display.
    • Provided comprehensive configuration management, error handling, and flexible storage backends (memory and disk).
    • Implemented multi-peer networking, DNS peer discovery, and peer persistence.
    • Enabled wallet UTXO tracking, address watching, transaction/block processing, and detailed balance reporting.
    • Added extensive documentation, usage guides, and a test plan.
  • Bug Fixes

    • Corrected version-aware serialization/deserialization of special transaction payloads to prevent over-reading and ensure compatibility.
    • Updated Dash mainnet genesis block constants for accuracy.
    • Fixed deserialization logic for special transaction types to avoid incorrect segwit interpretation.
  • Tests

    • Added a wide suite of unit, integration, and edge case tests covering network, storage, sync, validation, wallet, and transaction logic, including real-node integration and performance benchmarks.
  • Documentation

    • Added detailed README, architecture, integration test guides, and a comprehensive test plan.
  • Chores

    • Updated workspace configuration to include the new Dash SPV package.

@coderabbitai

coderabbitai Bot commented Jun 17, 2025

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This change introduces a new "dash-spv" Rust package implementing a modular, async Dash SPV client library, including its configuration, documentation, CLI, core modules, and comprehensive test suite. It also updates Dash mainnet genesis block parameters, fixes version-aware coinbase payload decoding, and adds support for the MNHF Signal special transaction type. The update includes major new modules for networking, storage, sync, validation, wallet, and error handling.

Changes

File(s) / Path(s) Change Summary
Cargo.toml Added "dash-spv" to workspace members.
dash-spv/Cargo.toml, dash-spv/README.md, dash-spv/CLAUDE.md, dash-spv/run_integration_tests.md Added new package manifest, documentation, and integration test instructions for the Dash SPV client.
dash-spv/src/lib.rs, dash-spv/src/main.rs Introduced library and CLI entrypoint for the Dash SPV client.
dash-spv/src/client/* Implemented high-level client API, configuration, block processing, consistency checks, filter sync, message handling, status display, wallet utilities, and watch management.
dash-spv/src/error.rs Added comprehensive error handling enums and result types.
dash-spv/src/filters/mod.rs Added BIP157 filter manager placeholder.
dash-spv/src/network/* Implemented full networking stack: TCP/multi-peer, handshake, peer management, address exchange, DNS discovery, connection pooling, persistence, and message routing.
dash-spv/src/storage/* Added async, segmented disk and in-memory storage managers, with trait abstraction, UTXO/index handling, and storage types.
dash-spv/src/sync/* Added synchronization managers for headers, filters, masternodes, with state tracking and flow control.
dash-spv/src/terminal.rs Added terminal UI/status bar utilities.
dash-spv/src/types.rs Defined common types: sync progress, chain state, validation modes, peer info, filter matches, watch items, stats, balances.
dash-spv/src/validation/* Added modular validation for headers, chainlocks, instantlocks, with a central validation manager.
dash-spv/src/wallet/* Implemented wallet with watched addresses, UTXO tracking, transaction/block processing, and balance calculation.
dash-spv/examples/* Added example programs for header and filter synchronization.
dash-spv/tests/* Added extensive async/unit/integration tests for networking, storage, sync, wallet, filter headers, transactions, and overall client behavior.
block_with_pro_reg_tx.data Added raw block data file with ProRegTx for test purposes.
dash-network/src/lib.rs Updated Regtest network magic number constant.
dash/src/blockdata/constants.rs Updated Dash mainnet genesis block header fields (time, bits, nonce).
dash/src/blockdata/transaction/mod.rs Fixed deserialization logic for MnhfSignal special transaction type (non-segwit).
dash/src/blockdata/transaction/special_transaction/coinbase.rs Made merkle_root_quorums field version-aware in coinbase payload encoding/decoding; added regression and versioning tests.
dash/src/blockdata/transaction/special_transaction/mnhf_signal.rs Added new module for MNHF Signal special transaction type and payload, with encoding/decoding and tests.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI/Main
    participant DashSpvClient
    participant NetworkManager
    participant StorageManager
    participant Wallet
    participant SyncManager

    User->>CLI/Main: Run dash-spv with config
    CLI/Main->>DashSpvClient: new(config)
    DashSpvClient->>NetworkManager: connect()
    DashSpvClient->>StorageManager: load headers, state
    DashSpvClient->>Wallet: load watched addresses, UTXOs
    CLI/Main->>DashSpvClient: start()
    DashSpvClient->>SyncManager: sync_to_tip()
    SyncManager->>NetworkManager: request headers/filters
    NetworkManager->>DashSpvClient: deliver network messages
    DashSpvClient->>SyncManager: handle headers/filters
    SyncManager->>StorageManager: store headers/filters
    DashSpvClient->>Wallet: update UTXOs/balances on relevant txs
    DashSpvClient->>CLI/Main: report sync progress
    User->>CLI/Main: Ctrl-C (shutdown)
    CLI/Main->>DashSpvClient: stop()
    DashSpvClient->>NetworkManager: disconnect
    DashSpvClient->>StorageManager: shutdown
Loading

Possibly related PRs

  • dashpay/rust-dashcore#50: Adds new masternode list-related network messages (getmnlistd, mnlistdiff) to Dash core library, which are directly used by the SPV client’s masternode sync features introduced in this PR.

Poem

(\(\
( -.-) A dash of code, a hop, a leap—
o_(")(") Now SPV bunnies never sleep!

With headers, wallets, blocks in tow,
Through network fields we swiftly go.

Modular, async, tests abound—
The Dashy future’s rabbit-bound!

🥕✨


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

PastaPastaPasta and others added 23 commits June 16, 2025 19:36
- Replace placeholder filter_matches_scripts with real BIP158 GCS implementation
- Add comprehensive integration test framework with Docker support
- Implement network monitoring for ChainLocks and InstantLocks with signature verification
- Enhance masternode engine with proper block header feeding and state management
- Add watch item persistence and improved transaction discovery
- Increase filter search range from 50 to 1000 blocks for better coverage
- Enable X11 hashing and BLS signature verification in dependencies
- Add proper error handling and logging throughout the sync pipeline

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Ping and Pong Handling: Added mechanisms to send periodic pings and handle incoming pings/pongs, enhancing network reliability.
- Block Processing: Implemented functions to process new block hashes immediately and manage block headers and filters effectively.
- Filter Headers and Filters: Added logic to handle CFHeaders and CFilter network messages and check them against watch items.
- Logging Enhancements: Improved logging for better traceability, including filter matches and network message receipt.
- Error Handling: Strengthened error handling for network messages and block processing errors.

This update enhances network responsiveness and block synchronization, enabling better SPV client performance."
- Add get_header_height_by_hash() method for O(1) hash-to-height lookups
- Add get_headers_batch() method for efficient bulk header loading
- Implement reverse index in both disk and memory storage
- Add as_any_mut() trait for storage downcasting
- Leverage existing segmented file structure for batch operations

These optimizations enable efficient masternode sync by reducing
individual storage reads from millions to thousands.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Replace inefficient strategy that fed ALL 2.2+ million headers individually
with selective feeding of only required headers:

- Use reverse index for O(1) hash-to-height lookups
- Feed only target, base, and quorum block hashes
- Use batch loading for recent header ranges (~1000 headers)
- Eliminate "Feeding 2278524 block headers" bottleneck

Performance improvement: ~2.2M individual reads → ~1K batch operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Implement a status bar showing sync progress at the bottom of the terminal:
- Headers count and filter headers count
- Latest ChainLock height and peer count
- Network name (Dash/Testnet/Regtest)
- Updates every 100ms without interfering with log output

Features:
- Uses crossterm for cross-platform terminal control
- RAII cleanup with TerminalGuard
- Logs stream normally above persistent status bar
- Optional --no-terminal-ui flag to disable

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive terminal UI integration to the SPV client:
- enable_terminal_ui() and get_terminal_ui() methods
- Real-time status updates after network connections
- Status updates after header processing and ChainLock events
- update_status_display() method with storage data integration
- Proper shutdown sequence ensuring storage persistence
- Network configuration getter for UI display

The client now displays live sync progress including header counts
from storage, peer connections, and ChainLock heights.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
CLI improvements:
- Add --no-terminal-ui flag to disable status bar
- Proper terminal UI initialization timing
- Network name display integration
- Remove unused Arc import

Logging improvements:
- Fix log level handling in init_logging()
- Improve tracing-subscriber configuration
- Remove thread IDs for cleaner output

The CLI now provides a modern terminal experience with optional
real-time status display alongside streaming logs.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Small enhancements to header and filter sync:
- Improve logging and error handling
- Better progress reporting during sync operations
- Consistent formatting across sync modules

These changes support the terminal UI integration and provide
better visibility into sync progress.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
PastaPastaPasta and others added 5 commits June 16, 2025 20:44
- Add thread-safe Mutex wrapper around BufReader to prevent race conditions
- Implement sticky peer selection for sync consistency during operations
- Increase peer count limits (2-5 peers) for better network resilience
- Add single-peer message routing for sync operations requiring consistency
- Improve connection error handling and peer disconnection detection
- Add timeout-based message receiving to prevent indefinite blocking
- Reduce log verbosity for common sync messages to improve readability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive sync state management with timeout detection
- Implement overlapping header handling for improved sync reliability
- Add coordinated message routing between sync managers and main client
- Enhance filter sync with batch processing and progress tracking
- Add sync timeout detection and recovery mechanisms
- Improve masternode sync coordination and state management
- Add detailed sync progress logging and error handling
- Implement proper chain validation during sync operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add centralized network message handling to prevent race conditions
- Implement message routing between monitoring loop and sync operations
- Add comprehensive sync timeout detection and recovery mechanisms
- Enhance filter sync coordination with monitoring loop management
- Add detailed documentation for network message architecture
- Improve sync progress reporting and status updates
- Reduce debug noise from transaction input checking
- Add sync_and_check_filters_with_monitoring method for better coordination

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add filter header verification test for chain validation
- Enhance multi-peer test with better error handling and timeouts
- Add checksum utility for data integrity verification
- Improve consensus encoding with better error messages
- Add test infrastructure for sync coordination scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Remove premature finish_sync() calls that were marking header and filter
header synchronization as complete immediately after starting. The sync
should only be marked as finished when handle_*_message() returns false,
indicating actual sync completion.

- Remove finish_sync() calls after starting header sync
- Remove finish_sync() calls after starting filter header sync
- Add sync_state_mut() accessor for proper state management
- Add proper sync completion in client message handlers

This fixes the issue where sync would complete with 0 headers because
the sync state was marked as finished before any headers were processed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@QuantumExplorer
QuantumExplorer merged commit fb6ecde into v0.40-dev Jun 17, 2025
17 checks passed
@QuantumExplorer

Copy link
Copy Markdown
Member

Good job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants