Skip to content

[Core] Remove BIP30 check#1775

Merged
furszy merged 1 commit into
PIVX-Project:masterfrom
random-zebra:202007_removeBIP30
Aug 2, 2020
Merged

[Core] Remove BIP30 check#1775
furszy merged 1 commit into
PIVX-Project:masterfrom
random-zebra:202007_removeBIP30

Conversation

@random-zebra

@random-zebra random-zebra commented Jul 31, 2020

Copy link
Copy Markdown

A bit of history

Two transactions can have the same txid if their parents are identical, since the txids of the parents are included in a transaction.
Coinbases have no parents, so it used to be possible for two of them to be identical.
Further, by building on duplicate coinbases, duplicate normal transactions were possible as well (http://r6.ca/blog/20120206T005236Z.html).

In order to remove the possibility of having duplicate transaction ids, Bitcoin introduced, with BIP30, the following consensus rule:

  • Blocks are not allowed to contain a transaction whose identifier matches that of an earlier, not-fully-spent transaction in the same chain. [1]

This rule was enforced by verifying (in ConnectBlock) that none of the transactions in the block was overwriting an already existent non-pruned CCoins entry.

BIP34 was later added in Bitcoin to enforce better transaction uniqueness, with the update of block version to 2, which introduced the following consensus rule:

  • the first serialized element in the scriptSig of coinbase transactions must be the height of the chain. [2]

After the complete activation of BIP34, there seemed to be no longer need for the check in ConnectBlock, added for BIP30, as duplicated coinbases were rendered impossible with BIP34 (bitcoin#6931).

This assumption was later revisited, when several blocks were found on Bitcoin's main chain (before BIP34 activation), having coinbase scripts with an height higher than the current chain height (and higher than the BIP34 activation height).
Thus, coinbases for blocks at these "future" heights would have given the opportunity for BIP30 to be violated even with BIP34 enforced (bitcoin#12204).

Motivation for this PR

PIVX has BIP30 and BIP34 consensus rules already implemented since the chain start.
The first block after the genesis (height=1) has version 3.
The "block.nVersion=2 rule" is enforced in ContextualCheckBlock

PIVX/src/main.cpp

Lines 3605 to 3612 in af3dd41

// Enforce block.nVersion=2 rule that the coinbase starts with serialized block height
if (pindexPrev) { // pindexPrev is only null on the first block which is a version 1 block.
CScript expect = CScript() << nHeight;
if (block.vtx[0].vin[0].scriptSig.size() < expect.size() ||
!std::equal(expect.begin(), expect.end(), block.vtx[0].vin[0].scriptSig.begin())) {
return state.DoS(100, false, REJECT_INVALID, "bad-cb-height", false, "block height mismatch in coinbase");
}
}

However the code still has the (somewhat expensive) BIP30 check in ConnectBlock, which wasn't needed at all, given the full enforcement of BIP34 since the start of the chain.

This PR removes it.

Side Note: Even without BIP34, with Proof-of-Stake, coinbase transactions have empty scriptPubKey (thus unspendable outputs), therefore there would have been no need for BIP30 checks in any case (at least after PoS activation height).

Not needed as BIP34 is enforced since the genesis
(bitcoin/bitcoin@06d81ad)
@random-zebra random-zebra added this to the 5.0.0 milestone Jul 31, 2020
@random-zebra random-zebra self-assigned this Jul 31, 2020

@furszy furszy left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find! ACK c894e8f.

@Fuzzbawls Fuzzbawls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!
ACK c894e8f

@furszy
furszy merged commit 85b5f2e into PIVX-Project:master Aug 2, 2020
random-zebra added a commit that referenced this pull request Aug 8, 2020
9d97db0 Add unit test for UpdateCoins (random-zebra)
0965d3a Make CCoinsViewTest behave like CCoinsViewDB (Alex Morcos)
7d23d5c [Core] PIVX: remove db lookup for coinbase outputs (random-zebra)
1e88b7f ModifyNewCoins saves database lookups (Alex Morcos)

Pull request description:

  Adapted version of bitcoin#6932

  > When processing a new transaction, in addition to spending the Coins of its txin's, it creates a new Coins for its outputs. The existing ModifyCoins function will first make sure this Coins does not already exist. It can not exist due to BIP 30, but because of that, the lookup can't be cached and always has to go to the database. Since we are creating the Coins to match the new tx anyway, there is no point in checking if it exists first anyway. However this should not be used for coinbase tx's in order to preserve the historical behavior of overwriting the two existing duplicate tx pairs.
  >
  > In conjunction with bitcoin#6931 this will help ConnectBlock be much more efficient with caching access to the database.

  We remove the restriction on coinbases, and thus use always `ModifyNewCoins` instead of `ModifyCoins`, since duplicated coinbases are not possible on PIVX (ref. #1775).
  Accordingly, we remove the logic for checking the effects of duplicate coinbase txs in the unit test.

ACKs for top commit:
  furszy:
    code looking good, tests and reindex went well too,  ACK 9d97db0
  Fuzzbawls:
    utACK 9d97db0

Tree-SHA512: e427f00699dfda29a9536b4e43919aecbfea90cf30195fe5a3835a49a715abf4f353a4672e6e99213b2016fb5b93f4ebfc94998a420ee8dc10626658069993af
random-zebra added a commit to random-zebra/PIVX that referenced this pull request Aug 9, 2020
@random-zebra random-zebra added the Needs Backport Placeholder tag for anything needing a backport to prior version branches label Aug 12, 2020
Fuzzbawls pushed a commit to Fuzzbawls/PIVX that referenced this pull request Aug 13, 2020
Not needed as BIP34 is enforced since the genesis
(bitcoin/bitcoin@06d81ad)
Github-Pull: PIVX-Project#1775
Rebased-From: c894e8f
furszy added a commit that referenced this pull request Aug 18, 2020
dc0ed71 [BUG][GUI] Don't append cold-stake records twice (random-zebra)
91adcd7 masternode: CalculateScore and GetBlockHash accessing to chainActive without cs_main fix. (furszy)
eeb112f GetMasternodeInputAge: Missing cs_main lock (furszy)
52ec12d refactor: decouple decompose coinstake from TransactionRecord::decomposeTransaction. (furszy)
51a8389 [BUG] Properly copy fCoinStake memeber between CTxInUndo and CCoins Github-Pull: #1796 Rebased-From: acf757b (random-zebra)
5c3caa4 lock cs_main for Misbehaving (furszy)
5c629d8 openssl.org dependency download link changed (CryptoDev-Project)
11aa63c Do not try to resend transactions if the node is importing or in IBD. (furszy)
c59b62e [Core] Remove BIP30 check (random-zebra)
3f05eba include missing atomic to make CMake linux happy. (furszy)
f0cfd88 Make the cs_sendProcessing a LOCK instead of a TRY_LOCK (Matt Corallo)
d38e28c Split CNode::cs_vSend: message processing and message sending (Matt Corallo)
7561b18 Remove double brackets in addrman (Matt Corallo)
63c629b Fix AddrMan locking (Matt Corallo)
f78cec4 Make fDisconnect an std::atomic (Matt Corallo)
ec56cf5 net: fix a few cases where messages were sent rather than dropped upon disconnection (furszy)
7697085 Acquire cs_main lock before cs_wallet during wallet initialization (random-zebra)
c796593 Remove bogus assert on number of oubound connections. (Matt Corallo)
7521065 wallet: Ignore MarkConflict if block hash is not known (random-zebra)
72042ac Move disconnectBlocks logging to use __func__ instead of hardcoded method name. (furszy)
0f66764 Simplify DisconnectBlock arguments/return value (furszy)
cca4a8c Split logic to undo txin's off DisconnectBlock (furszy)
0f883e6 Cleaner disconnectBlock flow for coinbase and zerocoin txs. (furszy)
a7cbc46 Move UndoWriteToDisk() and UndoReadFromDisk() to anon namespace (random-zebra)
de5a405 Decouple CBlockUndo from CDiskBlockPos (jtimon)
671143c Decouple miner.o and txmempool.o from CTxUndo (random-zebra)
9419228 Decouple CCoins from CTxInUndo (jtimon)

Pull request description:

  Backports the following PRs to the `4.2` branch:

  #1746
  #1735
  #1767
  #1769
  #1781
  #1780
  #1775
  #1783
  #1750
  #1785
  #1796
  #1776
  #1791

ACKs for top commit:
  furszy:
    Added #1805. utACK dc0ed71.
  random-zebra:
    utACK dc0ed71

Tree-SHA512: 0e59c9e751597b1b6f9a419e117946cec468dffdb921c882a44e5770ecb1a36b7d3d25f8c7f97d48bb3d59f136492842c08969901512d957a17bfaec6aece449
random-zebra added a commit to random-zebra/PIVX that referenced this pull request Aug 23, 2020
random-zebra added a commit that referenced this pull request Sep 5, 2020
535d8e4 scripted-diff: various renames for per-utxo consistency (random-zebra)
60c73ad Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
1166f1f Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
f25d3c6 [MOVEONLY] Move old CCoins class to txdb.cpp (random-zebra)
aab17b3 Upgrade from per-tx database to per-txout (Pieter Wuille)
a55eb98 Reduce reserved memory space for flushing (random-zebra)
7d637ea Remove unused CCoins methods (random-zebra)
4b7b1a3 Extend coins_tests (random-zebra)
8cf2dc0 Switch CCoinsView and chainstate db from per-txid to per-txout (random-zebra)
b20a662 [Cleanup] Remove unused CCoinsViewCache::IsOutputAvailable (random-zebra)
ea82855 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
0f9f2b6 [Core] Fix not-pruned-check in miner for zerocoin mint outputs (random-zebra)
3698d47 Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
11c728a Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
3c65936 Switch tests from ModifyCoins to AddCoin/SpendCoin (random-zebra)
b05fe55 Switch CScriptCheck to use Coin instead of CCoins (random-zebra)
9e5c2ae Only pass things committed to by tx's witness hash to CScriptCheck (random-zebra)
22b0b4a [Cleanup] fix warning in comparisons with nCoinbaseMaturity (random-zebra)
666081d Switch from per-tx to per-txout CCoinsViewCache methods in some places (random-zebra)
53fc2be [Core] PIVX: remove potential_overwrite: BIP34 has always been enforced (random-zebra)
16924aa Introduce new per-txout CCoinsViewCache functions (random-zebra)
2d74bc1 Optimization: Coin&& to ApplyTxInUndo (random-zebra)
3fcb092 Replace CTxInUndo with Coin (random-zebra)
42a8996 [Core] PIVX: Add fCoinStake boolean field to Coin class (random-zebra)
a01a9f2 Introduce Coin, a single unspent output (random-zebra)

Pull request description:

  The chainstate database, and its cache, rely on a per-tx model (i.e. it is a map from txids to *lists* of unspent outputs).
  This PR chages it to a per-txout model (i.e. a map from outpoints to *single* unspent outputs).

  Pros:
  - Simpler code.
  - Avoiding the CPU overhead of deserializing and serializing the unused outputs.
  - More predictable memory usage.
  - More easily adaptable to various cache flushing strategies.

  Cons:
  - Slightly larger on-disk representation, and sometimes larger in-memory representation (when there are multiple outputs for the same txid in the cache, which becomes optional).

  Adapted from bitcoin#10195 (adding a `fCoinStake` boolean memeber to the Coin class and considering BIP34 always in effect as per #1775)

  Based on top of:
  - [x] #1799
  - [x] #1800
  - [x] #1797
  - [x] #1796
  - [x] #1795
  - [x] #1793
  - [x] #1773

  The actual PR starts with "Introduce Coin, a single unspent output" (171bf0b8a55de0e3f94305b33e97bb1cf4a01b70)

  ⚠️  **Note for testers**: Do a backup of the chainstate before testing. With this PR, at startup, the database is automatically upgraded to the new format (so a reindex is not needed).

ACKs for top commit:
  furszy:
    Code review ACK 535d8e4. Need some live testing now.
  furszy:
    ACK 535d8e4
  Fuzzbawls:
    ACK 535d8e4

Tree-SHA512: 8648d31c343ff901f8568c14d0848241c9b8aca0b9a8f89f3e5dbe02d384ba35faf572757e84f979fba1f198d721a2516d0e784b5e0dc8d02b6e308b4278b484
@random-zebra random-zebra modified the milestones: 5.0.0, 4.3.0 Sep 10, 2020
@random-zebra random-zebra removed the Needs Backport Placeholder tag for anything needing a backport to prior version branches label Sep 10, 2020
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Jul 14, 2026
…IP30/BIP34

36a5509 test: remove unused scan-pre-bip34-coinbases.py once fix is merged (Konstantin Akimov)
69dd635 refactor: remove Bitcoin's specific hashes for BIP30/BIP34 (Konstantin Akimov)
e207edc test: script to scan pre-bip34 coinbases to find potential issues for futher blocks (Konstantin Akimov)
e6d6f07 feat: remove bip30 field from coinstat index (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  Bitcoin Core has workarounds for BIP30/BIP34 historical blocks; this workaround is irrelevant for Dash Core.

  ## What was done?
  Removed Bitcoin Core exceptions from validation for Dash Core.
  Removed bip30 counter from RPC output `gettxoutsetinfo` which is always 0 for dash core.

  Though, we can't fully remove checks from validation.cpp same as it is done in PIVX (PIVX-Project#1775) because we had 951 blocks at mainnet and 76 blocks on testnet before BIP34 has been activated and this check is still used for validation of old blocks.

  ## How Has This Been Tested?
  Run `scan-pre-bip34-coinbases.py` to find a blocks that violates bip34 safety measures before bip34 is activated for both main and testnet.

      $ ./scan-pre-bip34-coinbases.py --network=test --datadir=/DASH/.dashcore
      Scanning test heights 1..75 (BIP34Height=76)  @ http://127.0.0.1:19998/

      Scanned 75 blocks (heights 1..75, BIP34Height=76).
        0 candidate future-collision targets (indicated height > BIP34Height).
        1 scriptSigs not minimally encoding any integer.
          These are safe ONLY IF the decoder above is exhaustive -- please eyeball:
            h=    16  scriptSig=0110062f503253482f042c51615308f800000402000000102f7374726174756d2d7365727665722f

      RESULT: No pre-BIP34 coinbase has a leading minimal CScriptNum > BIP34Height.
              Assuming the decoder is correct (verify against the unparseable list),
              BIP30 enforcement is unreachable on this chain.

      $ ./scan-pre-bip34-coinbases.py --network=main --datadir=/DASH/.dashcore
      Scanning main heights 1..950 (BIP34Height=951)  @ http://127.0.0.1:9998/

      Scanned 950 blocks (heights 1..950, BIP34Height=951).
        0 candidate future-collision targets (indicated height > BIP34Height).
        0 scriptSigs not minimally encoding any integer.
          These are safe ONLY IF the decoder above is exhaustive -- please eyeball:

      RESULT: No pre-BIP34 coinbase has a leading minimal CScriptNum > BIP34Height.
              Assuming the decoder is correct (verify against the unparseable list),
              BIP30 enforcement is unreachable on this chain.

  Extra double check is call of `gettxoutsetinfo none` to be sure that `bip30` is 0 for main and testnets:

      $ gettxoutsetinfo none
      {
        "height": 2476080,
        "bestblock": "0000000000000010ee3a3763bed9f1b5ec25e08d0594749a968636b714d0924b",
        "txouts": 4514705,
        "bogosize": 339509165,
        "total_amount": 12710996.18871387,
        "total_unspendable_amount": -740940.11336565,
        "block_info": {
          "prevout_spent": 6.93831412,
          "coinbase": 1.27251741,
          "new_outputs_ex_coinbase": 6.93814597,
          "unspendable": 0.49787579,
          "unspendables": {
            "genesis_block": 0.00000000,
            "bip30": 0.00000000,
            "scripts": 0.49787579,
            "unclaimed_rewards": 0.00000000
          }
        }
      }

  ## Breaking Changes
  Removed rpc field `bip30` from RPC `gettxoutsetinfo`

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: 966a8e314195e9d9581b886313cb38aac35ed9f3eb3ff61eb132352c1504b3de522a1d389cf008df5b261f7e280cd1b410d2b27e1661d1e96c5679cabe5cfbb7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants