Skip to content

fix(torii): atomic token registry cache to fix race condition - #3124

Merged
Larkooo merged 3 commits into
dojoengine:mainfrom
Larkooo:tokens-register
Mar 24, 2025
Merged

fix(torii): atomic token registry cache to fix race condition#3124
Larkooo merged 3 commits into
dojoengine:mainfrom
Larkooo:tokens-register

Conversation

@Larkooo

@Larkooo Larkooo commented Mar 21, 2025

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Refactor
    • Streamlined token processing by optimizing how parallel operations are handled with a new registration method.
    • Adopted an optimistic approach for token registration to reduce redundant checks and improve efficiency.
    • Simplified error feedback during token registration for clearer operation outcomes by removing contextual error handling.
  • Chores
    • Removed the configuration file dope.toml, eliminating outdated settings related to blockchain interaction.

@coderabbitai

coderabbitai Bot commented Mar 21, 2025

Copy link
Copy Markdown
Contributor

Ohayo, sensei! Below is the detailed overview of the changes:

Walkthrough

This pull request updates the token ID management in the local cache for token transfers. The LocalCache struct now uses a Mutex<HashSet<String>> instead of a RwLock, and the associated methods have been updated to use the mutex lock accordingly. A new method, try_register_token_id, is introduced for optimistic token registration. In the ERC transfer handlers, the existence check using contains_token_id has been replaced with this new registration approach, and explicit token metadata registration has been removed. Additionally, error context handling in one database query has been simplified.

Changes

File(s) Change Summary
crates/torii/sqlite/src/cache.rs Changed token_id_registry from RwLock<HashSet<String>> to Mutex<HashSet<String>>; updated contains_token_id and register_token_id to use lock().await; added new try_register_token_id for optimistic registration.
crates/torii/sqlite/src/erc.rs Replaced explicit token existence check with an optimistic registration approach using try_register_token_id in ERC20 and NFT transfer handlers; removed explicit token metadata registration calls.
crates/torii/sqlite/src/executor/mod.rs Removed the error context wrapper from the database query (fetch_one) in ERC20 token registration, resulting in simplified error reporting.

Sequence Diagram(s)

sequenceDiagram
    participant T as ERC Transfer Handler
    participant C as LocalCache
    T->>C: try_register_token_id(token_id)
    alt Token ID not registered
        C-->>T: true
        T->>T: Register token metadata
    else Already registered
        C-->>T: false
        T->>T: Skip metadata registration
    end
Loading

Possibly related PRs

  • feat(torii): erc options for max tasks & artifacts path #3061: The changes in the main PR regarding the LocalCache struct and its methods, particularly the introduction of try_register_token_id, are directly related to the modifications in the retrieved PR that replace the use of contains_token_id with try_register_token_id in the handling of token IDs during ERC transfer operations.
  • test(torii): add metadata to erc1155 and update #3054: The changes in the main PR, specifically the introduction of the try_register_token_id method and its use in managing token IDs, are directly related to the modifications in the retrieved PR, which replaces the use of contains_token_id with try_register_token_id in the context of ERC20 and NFT transfers.
  • feat(torii-indexer): task manager & parallelize erc transfers #2913: The changes in the main PR, specifically the introduction of the try_register_token_id method and modifications to the contains_token_id and register_token_id methods in the LocalCache struct, are directly related to the changes in the retrieved PR, which replaces the use of contains_token_id with try_register_token_id in the handle_erc20_transfer and handle_nft_transfer methods.

Suggested Reviewers

  • glihm

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between f19f0f7 and 73775da.

📒 Files selected for processing (3)
  • crates/torii/sqlite/src/cache.rs (3 hunks)
  • crates/torii/sqlite/src/erc.rs (2 hunks)
  • examples/spawn-and-move/dope.toml (0 hunks)
💤 Files with no reviewable changes (1)
  • examples/spawn-and-move/dope.toml
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/torii/sqlite/src/erc.rs
  • crates/torii/sqlite/src/cache.rs

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
crates/torii/sqlite/src/executor/mod.rs (1)

813-813: Ohayo! Simplified error handling is cleaner, but less informative

The removal of .with_context() makes the code cleaner but sacrifices detailed error information. This is acceptable since the SQL query is straightforward, but future developers may have slightly less context when debugging if this query fails.

If detailed error context is valuable for troubleshooting, you could consider:

-let token = query.fetch_one(&mut **tx).await?;
+let token = query.fetch_one(&mut **tx).await
+    .with_context(|| format!("Failed to register ERC20 token with ID {}", &register_erc20_token.token_id))?;
crates/torii/sqlite/src/erc.rs (1)

1-304: Overall architecture significantly improved

Ohayo, sensei! The change from checking and then registering to an atomic optimistic registration approach is an excellent architectural improvement. This resolves the race condition that could occur when multiple threads try to register the same token ID simultaneously.

This implementation follows the "optimistic concurrency control" pattern, which is a great fit for this use case. The use of a Mutex for atomic operations ensures thread safety while the optimistic approach minimizes lock contention by only acquiring the lock once instead of multiple times.

🧰 Tools
🪛 GitHub Actions: ci

[error] 40-40: Rust formatting check failed. Please run 'rustfmt' to format the code.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d2436c7 and f19f0f7.

📒 Files selected for processing (3)
  • crates/torii/sqlite/src/cache.rs (3 hunks)
  • crates/torii/sqlite/src/erc.rs (2 hunks)
  • crates/torii/sqlite/src/executor/mod.rs (1 hunks)
🧰 Additional context used
🪛 GitHub Actions: ci
crates/torii/sqlite/src/erc.rs

[error] 40-40: Rust formatting check failed. Please run 'rustfmt' to format the code.

crates/torii/sqlite/src/cache.rs

[error] 144-144: Rust formatting check failed. Please run 'rustfmt' to format the code.

🔇 Additional comments (7)
crates/torii/sqlite/src/cache.rs (5)

130-131: Good transition from RwLock to Mutex for atomic operations!

The switch from RwLock to Mutex is appropriate for this use case, sensei! The comment clearly explains the reasoning - we need atomic operations because of parallelized token transfers and registrations.


147-147: Clean initialization update for Mutex

Properly updated the initialization to use Mutex::new().


152-152: Updated method to use Mutex lock

The contains_token_id method has been correctly updated to use lock() instead of read() to match the new mutex-based implementation.


156-157: Updated register method to use Mutex lock

The register_token_id method has been correctly updated to use lock() instead of write().


159-167: Excellent atomic registration method!

This new try_register_token_id method elegantly implements an atomic check-and-set operation, which is perfect for preventing race conditions in concurrent token registration, sensei!

crates/torii/sqlite/src/erc.rs (2)

43-46: Excellent implementation of optimistic registration!

This change uses the new atomic try_register_token_id method instead of separate check-then-register calls, which effectively fixes the race condition. The logic has been inverted (using ! before the method call) to maintain the same semantic meaning as before.


98-103: Consistent implementation for NFT transfers

You've applied the same optimistic registration pattern for NFT tokens as well, maintaining consistency across the codebase. The comments are also helpful in explaining the approach.

use starknet::providers::{Provider, ProviderError};
use starknet_crypto::Felt;
use tokio::sync::RwLock;
use tokio::sync::{Mutex, RwLock};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix the formatting issue

The pipeline is failing due to formatting issues.

Please run rustfmt on this file to fix the formatting as indicated by the CI error.

Comment thread crates/torii/sqlite/src/erc.rs Outdated
Comment on lines +39 to +43
// optimistically add the token_id to cache
// this cache is used while applying the cache diff
// so we need to make sure that all RegisterErc*Token queries
// are applied before the cache diff is applied
let token_exists: bool = !self.local_cache.try_register_token_id(token_id.to_string()).await;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Great comments explaining the optimization!

These comments clearly explain the rationale behind the optimistic registration approach. Ohayo, sensei! You've made the code more understandable.

However, please run rustfmt on this file to fix the formatting issue reported by CI.

🧰 Tools
🪛 GitHub Actions: ci

[error] 40-40: Rust formatting check failed. Please run 'rustfmt' to format the code.

@codecov

codecov Bot commented Mar 21, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 56.81%. Comparing base (d2436c7) to head (73775da).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
crates/torii/sqlite/src/cache.rs 75.00% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3124   +/-   ##
=======================================
  Coverage   56.81%   56.81%           
=======================================
  Files         442      442           
  Lines       61561    61564    +3     
=======================================
+ Hits        34975    34977    +2     
- Misses      26586    26587    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Larkooo
Larkooo enabled auto-merge (squash) March 24, 2025 01:22
@glihm glihm changed the title fix(torii): atomic token registry cache to ifx race condition fix(torii): atomic token registry cache to fix race condition Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants