Observer Refactor Decrypted transactions handling

[fredo]

We will redesign the way, decrypted transactions and the decrypted tx table is handled.

Tx Status in decrypted tx table

• not decrypted: tx cannot be decrypted
• invalid: transaction is invalid
• pending: transaction is valid
• protected inclusion: transaction was included as per shutter protocol defined
• unprotected inclusion: transaction was included but not as per shutter protocol defined
• not included: At one point we give up waiting for inclusion

Handle decrypted transactions

Whenever we will receive decryption keys we will decrypt and create an entry in the decrypted transaction table.

1. decrypt: if it cannot be decrypted mark as not decrypted

2. send transaction to RPC: Send the transaction to the observers RPC. If it gets rejected, mark transaction as invalid.
   Note: As soon as keys are released, we consider the transaction to be publicly known, and we consider the transaction to be protected, since the keys are only released after the last block has been seen AND the next validator is registered. So it's safe to gossip the transaction.

3. If success, mark the transaction as pending. Additionally, start a routine to wait for transaction receipt.

4. Once the transaction receipt is received, check for the correct inclusion. See Check for protected inclusion. If true, mark as potected inclusion. If false, mark as unprotected inclusion

5. If time X (i.e. 1 hour) has passed , mark as not included.

Check for protected inclusion

protected inclusion is received iff:

tx_list = [tx0, tx1, tx2, tx3, ...]
valid_tx_list = filter(not_invalid_and_not_not_decrypted, tx_list)

for i, tx in valid_tx_list:
  if not get_slot_for(tx.receipt.block_number) == expected_slot:
    tx_status = "unprotected inclusion"
    continue
  if tx.receipt.transaction_index != i:
     tx_status = "unprotected inclusion"
    continue
  tx_status = "protected inclusion"

Note: In get_slot_for(blocknumber) we either check via the timestamp, that'S safe or we check against the database entry of the block. That'S unsafe as the receipt could race before the block.

Note: We are igbnoring the fact that a malicious validator could replace a preceeding transaction with a frontrunning transaction. We'll fix that later.

Receive a new block

We don't query the blocks transaction data after receiving block anymore. no need to do that.

[faheelsattar]

For FE and BE shutter-explorer

1. Not found
2. Submitted i.e seq tx for which keys arent released.(transaction_submitted_event, decrypted_tx join). color: green
3. Pending user transaction i.e tx for which keys have been released but it's not included yet(erpc, decrypted_tx ). color: yellow
4. invalid i.e tx is invalid in the decrypted_tx table. color: red
5. Cannot be decrypted i.e not decrypted in the decrypted_tx table. color: red
6. Shielded inclusion i.e shielded inclusion in the decrypted_tx table. color: green
7. Unshielded inclusion i.e unshielded inclusion in the decrypted_tx table. color: green
8. Not included i.e not included in the decrypted_tx table. color: red.