Skip to content

fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868#6192

Merged
PastaPastaPasta merged 2 commits into
dashpay:developfrom
kwvg:6143_fixup
Aug 29, 2024
Merged

fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868#6192
PastaPastaPasta merged 2 commits into
dashpay:developfrom
kwvg:6143_fixup

Conversation

@kwvg

@kwvg kwvg commented Aug 10, 2024

Copy link
Copy Markdown
Collaborator

Additional Information

This pull request aims to deal with regressions (build) spotted in develop after the merger of dash#6143, namely, a failure to build the multiprocess variant and two data races, one involving ChainstateManager::m_best_header and another involving CoinJoinWalletManager::m_wallet_manager_map.

  • Fixes for the build failure and the first data race (b136742 and 9e7c685), have been spun off into dash#6199 and merged

  • The second data race is being avoided by protected m_wallet_manager_map with a new RecursiveMutex, cs_wallet_manager_map (the contents of this PR). A version of these changes are available using a regular Mutex but prove far more cumbersome (source) when taking practicalities into account (comment). A Mutex version is now available courtesy of UdjinM6 (see patch), thanks!

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas (note: N/A)
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (note: N/A)
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@kwvg kwvg added this to the 21.2 milestone Aug 10, 2024
@kwvg kwvg changed the title fix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changes fix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changes, data races Aug 10, 2024
@kwvg
kwvg marked this pull request as ready for review August 10, 2024 21:22
@kwvg
kwvg force-pushed the 6143_fixup branch 3 times, most recently from d492d1c to 9e50226 Compare August 11, 2024 08:34
Comment thread src/coinjoin/client.cpp Outdated
Comment thread src/rpc/coinjoin.cpp Outdated
@kwvg
kwvg requested a review from PastaPastaPasta August 11, 2024 11:39
PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Aug 11, 2024
… run multiprocess on CI

d5a944d Revert: fix: missing changes from bitcoin#19267 - run multiprocess on CI (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  See dashpay#6192 for a discussion around this: for now simply make CI happy again, by not running for multiprocessor (since this is new, and unstable / dev feature, and not something shipped).

  ## What was done?
  Revert commit which reintroduces multiprocessor CI while we investigate proper fix

  ## How Has This Been Tested?
  CI

  ## Breaking Changes

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: d878b02efc72019e09b09e2482e63dca276824b66272135e00ff340ba60b5b58e5f5ae3feff3b894f08958ca8fc6ce71f91c926f6aaf29a459d6230f129316f6
Comment thread src/coinjoin/interfaces.cpp Outdated
Comment thread src/wallet/wallet.cpp Outdated
Comment thread src/dsnotificationinterface.cpp
Comment thread src/coinjoin/client.cpp Outdated
Comment thread src/coinjoin/client.cpp Outdated
Comment thread src/rpc/coinjoin.cpp Outdated
PastaPastaPasta added a commit that referenced this pull request Aug 12, 2024
…#22937 and ipc/process

540f687 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh)
aafded6 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh)

Pull request description:

  ## Issue being fixed or feature implemented
  CI failure for multiprocess

  ## What was done?
  It resolve compilation failure on develop (apply changes from 22937 to ipc/process)

  See alternate solutions:
   - #6192
   - #6195

  ## How Has This Been Tested?
  Run build locally:
  ```
  make MULTIPROCESS=1 -j10
  ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror    --enable-debug --enable-crash-hooks  --enable-maintainer-mode --enable-stacktraces  --enable-multi-process
  ```

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 540f687

Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
@kwvg
kwvg marked this pull request as draft August 12, 2024 12:45
@kwvg kwvg changed the title fix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changes, data races fix: protect CoinJoinWalletManager::m_wallet_manager_map with RecursiveMutex, avoid data race Aug 12, 2024
@kwvg
kwvg force-pushed the 6143_fixup branch 2 times, most recently from fb186ef to dc0fbcd Compare August 12, 2024 19:06
@UdjinM6

UdjinM6 commented Aug 12, 2024

Copy link
Copy Markdown

my bad, should've run linter locally 🙈 pls also check 2d6612a

@kwvg
kwvg force-pushed the 6143_fixup branch 2 times, most recently from e95dd63 to 1c55375 Compare August 12, 2024 19:33
@kwvg kwvg changed the title fix: protect CoinJoinWalletManager::m_wallet_manager_map with RecursiveMutex, avoid data race fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race Aug 13, 2024
@kwvg
kwvg marked this pull request as ready for review August 15, 2024 11:44
@kwvg
kwvg requested a review from PastaPastaPasta August 15, 2024 11:44
Comment thread src/coinjoin/client.h Outdated
Comment thread src/coinjoin/client.h Outdated
Comment on lines 101 to 104

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use std::for_each here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would not be able to do the below snippet because the function isn't taking in a pair but one only one half of it

std::for_each(m_wallet_manager_map.begin(), m_wallet_manager_map.end(), func);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could do something ```suggestion
ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@UdjinM6, thoughts?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't use structured bindings there, would be more like 019a780 instead

@knst knst Aug 26, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like to use std::for_each if it's not super trivial.

ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});

it looks like extra more difficult to read, I disagree with this suggestment.

For-each has been a great option when there was no range-loop in C++! But in this particular case it looks difficult to read, difficult to understand. That's a case when 2 lines better than one!

using any_of below makes sense, but not here.

@PastaPastaPasta

Copy link
Copy Markdown
Member

Can you squash these two commits?

@UdjinM6

UdjinM6 commented Aug 25, 2024

Copy link
Copy Markdown
POTENTIAL DEADLOCK DETECTED
Previous lock order was:
 (2) 'walletInstance->cs_wallet' in wallet/wallet.cpp:4952 (in thread 'qt-init')
 (1) 'cs_wallet_manager_map' in coinjoin/client.cpp:1903 (in thread 'qt-init')
Current lock order is:
 (1) 'cs_wallet_manager_map' in coinjoin/client.cpp:1913 (in thread 'scheduler')
 'cs_deqsessions' in coinjoin/client.cpp:998 (in thread 'scheduler')
 (2) 'm_wallet.cs_wallet' in coinjoin/client.cpp:804 (in thread 'scheduler')

can be fixed by backporting bitcoin#22868 or via a temporary fix

diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp
index f648fc2ced..6a33585ede 100644
--- a/src/wallet/wallet.cpp
+++ b/src/wallet/wallet.cpp
@@ -4949,8 +4949,6 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain* chain, interfaces::C
     // Try to top up keypool. No-op if the wallet is locked.
     walletInstance->TopUpKeyPool();
 
-    LOCK(walletInstance->cs_wallet);
-
     if (chain && !AttachChain(walletInstance, *chain, error, warnings)) {
         return nullptr;
     }
@@ -4959,6 +4957,8 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain* chain, interfaces::C
         coinjoin_loader->AddWallet(*walletInstance);
     }
 
+    LOCK(walletInstance->cs_wallet);
+
     {
         LOCK(cs_wallets);
         for (auto& load_wallet : g_load_wallet_fns) {

kwvg added 2 commits August 26, 2024 15:55
Avoid TSan-reported data race

```
WARNING: ThreadSanitizer: data race (pid=374820)
  Read of size 8 at 0x7b140002ce10 by thread T12:
    #0 _M_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:154:42 (dashd+0xb58e08) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #1 get /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:361:21 (dashd+0xb58e08)
    #2 operator-> /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:355:9 (dashd+0xb58e08)
    #3 CoinJoinWalletManager::DoMaintenance() /src/dash/src/coinjoin/client.cpp:1907:9 (dashd+0xb58e08)
    [...]
  Previous write of size 8 at 0x7b140002ce10 by thread T17 (mutexes: write M0):
    #0 operator new(unsigned long) <null> (dashd+0x162657) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #1 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:114:27 (dashd+0xb772b4) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #2 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:443:20 (dashd+0xb772b4)
    #3 _M_get_node /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_tree.h:580:16 (dashd+0xb772b4)
    [...]
    dashpay#8 CoinJoinWalletManager::Add(CWallet&) /src/dash/src/coinjoin/client.cpp:1898:26 (dashd+0xb58c73) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
[...]
SUMMARY: ThreadSanitizer: data race [...]
```
excludes changes in `wallet/context.h` due to `::vpwallets` (and thus,
`cs_wallets`) not being deglobalized yet
@kwvg kwvg changed the title fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868 Aug 26, 2024

@UdjinM6 UdjinM6 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (to be merged via merge commit)

light-ACK 87d775d

@PastaPastaPasta
PastaPastaPasta merged commit 6dbed2a into dashpay:develop Aug 29, 2024
@UdjinM6 UdjinM6 modified the milestones: 21.2, 22 Oct 29, 2024
ab4cus pushed a commit to ab4cus/e4Coin-core that referenced this pull request Jul 8, 2026
…#22937 and ipc/process

bdce842 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh)
42ad8d1 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh)

Pull request description:

  ## Issue being fixed or feature implemented
  CI failure for multiprocess

  ## What was done?
  It resolve compilation failure on develop (apply changes from 22937 to ipc/process)

  See alternate solutions:
   - dashpay/dash#6192
   - dashpay/dash#6195

  ## How Has This Been Tested?
  Run build locally:
  ```
  make MULTIPROCESS=1 -j10
  ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror    --enable-debug --enable-crash-hooks  --enable-maintainer-mode --enable-stacktraces  --enable-multi-process
  ```

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK bdce842

Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
ab4cus pushed a commit to ab4cus/e4Coin-core that referenced this pull request Jul 8, 2026
… with Mutex, avoid data race, partial bitcoin#22868

50547cc partial bitcoin#22868: Call load handlers without cs_wallet locked (Kittywhiskers Van Gogh)
84f1113 coinjoin: protect `m_wallet_manager_map` with `cs_wallet_manager_map` (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  This pull request aims to deal with regressions ([build](https://gitlab.com/dashpay/dash/-/jobs/7550859495)) spotted in `develop` after the merger of [dash#6143](dashpay/dash#6143), namely, a failure to build the multiprocess variant and two data races, one involving `ChainstateManager::m_best_header` and another involving `CoinJoinWalletManager::m_wallet_manager_map`.

  * Fixes for the build failure and the first data race (b136742f98c6792735232cc170123f849fd13bad and 9e7c6850139e69cb64cfe89813e6f0e4b1e626e1), have been spun off into [dash#6199](dashpay/dash#6199) and merged

  * The second data race is being avoided by protected `m_wallet_manager_map` with a new `RecursiveMutex`, `cs_wallet_manager_map` (the contents of this PR). ~~A version of these changes are available using a regular `Mutex` but prove far more cumbersome ([source](dashpay/dash@b89457d)) when taking practicalities into account ([comment](dashpay/dash#6192 (comment) A `Mutex` version is now available courtesy of UdjinM6 (see [patch](dashpay/dash@ecf5c1b)), thanks!

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    light-ACK 50547cc

Tree-SHA512: 52545a1547357a45fb6bf4d3948fc75a5e88f1d86e9809b1060007cd74dd383249691d8d9777f647c7534d458fe126ae818fa2b47f0e5b0cee78a469af1ed0c9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants