report marconi target addresses PLT-1009#796
Conversation
Update Default ProtocolParameters in Ledger.Params to current values on mainnet. This will allow to emulator to use the same ProtocolParameters values as the current mainnet following Vasil HF
Provide a JSON-RPC function to report on user provided address
Add networkId to dbConfig. This is required so that we can convert addresses to and from cardano to pluts
…input-output-hk/plutus-apps into 1009/report-marconi-target-addresses-v2
add rest endpoint to report on marconi mamba addresses user has provided. Add sql utility for examle and demo purpose
Report target addresses, user input addresses, as valid Bech32 addresses
…09/report-marconi-target-addresses-v2
Add README to db-utils
kayvank
changed the title
kayvank
changed the title
Add: - concurrent query for the in-memory utxo indexer. - eumulator for utxo indexer. This allows for the JSON-RPC-example server to function without marconi. However, the example server does have a dependency on SQLite utxo-db pre-populated as a result of previous marconi execution. This will allow the server to run without a need for marconi. In this case, we are fetching utxo data directly for cold-store, SQLite db
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
9befcdd to
9b04775
Compare
Merge branch 'main' of github.com:input-output-hk/plutus-apps into 1009/report-marconi-target-addresses-v2
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
295ee3f to
b7a429a
Compare
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
b7a429a to
7a4f8d3
Compare
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
7a4f8d3 to
9b674a6
Compare
Changed Cardano.Api From/To Filed SQL mappings to RawByteString
Add test data from preview-testnet
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
9b674a6 to
23bfc2c
Compare
provide end-point to report user-provided Shelley addresses for - REST end-point - CONSOLE - JSON-RPC
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
8f7f32e to
267ec3d
Compare
berewt
left a comment
berewt
left a comment
There was a problem hiding this comment.
Looks good to me, just a minor comment on the semaphore usage.
| (atomically (takeTMVar qreq ) ) | ||
| (atomically (takeTMVar qreq ) ) | ||
| (atomically (putTMVar utxoIndexer ix) ) |
There was a problem hiding this comment.
The synchronisation here looks expensive: when a query arrives, we wait for it to release a semaphore, to release an ix to release its action before we wait for the end of the transaction.
The first step looks unnecessary to me, but as always concurrency is complex and I may be wrong (if I'm right removing the initialisation in this bracket and the one in the query shouldn't impact the behaviour and save us a bit of time).
There was a problem hiding this comment.
Thank you for taking a detailed look at this part, as I could certainly use another pair of eyes here. Although this section is from the toy example demo, its counterpart, indexer.hs is a critical part of the flow, since the potential of locking SQLite is present.
The main things here are:
- queries take precedence over inserts.
- SQLite locks when concurrently performing inserts/queries, concurrent inserts or concurrent queries are OK, mixing them is not.
The section of the code you’re referring to is in the toy json-rpc demo and is mocking the Marconi insert operation, so the code resembles that flow from indexer.hs
When the query arrives, we need to stop the inserts and resume them only after they're completed from both cold and hot storage. The initialization section of the bracket, stops Marconi from inserting, and the closing part releases the lock only after queries have been completed.
Changing this section of the code will require changing the indexer.hs section as well. I’ll be happy to discuss that section with you further as it be great to get more performance from that section. However, since that is a more significant change, and not part of your comment, I suggest we proceed with the merge. If we find that indeed the initialization is not required, I'll do a quick patch to address it.
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
e24d2df to
dbba01c
Compare
kayvank
force-pushed
the
1009/report-marconi-target-addresses-v2
branch
from
dbba01c to
fcf25c9
Compare
* Update Default ProtocolParams to mainnet PLT-1037 Update Default ProtocolParameters in Ledger.Params to current values on mainnet. This will allow to emulator to use the same ProtocolParameters values as the current mainnet following Vasil HF * Report-marconi-target-addresses PLT-1009 Provide a JSON-RPC function to report on user provided address * report-marconi-target-addresses -PLT-1009 Add networkId to dbConfig. This is required so that we can convert addresses to and from cardano to pluts * report marconi-mamba-Target-Addresses PLT-1009 * report marconi-mamba-Target-Addresses PLT-1009 * report-marconi-target-addresses-v2 PLT-1009 Report target addresses, user input addresses, as valid Bech32 addresses * report-marconi-target-addresses PLT-1009 Add README to db-utils * report-marconi-target-addresses PLT-1009 Add: - concurrent query for the in-memory utxo indexer. - eumulator for utxo indexer. This allows for the JSON-RPC-example server to function without marconi. However, the example server does have a dependency on SQLite utxo-db pre-populated as a result of previous marconi execution. This will allow the server to run without a need for marconi. In this case, we are fetching utxo data directly for cold-store, SQLite db * report-marconi-target-addresses-v2 PLT-1009 Changed Cardano.Api From/To Filed SQL mappings to RawByteString * report-marconi-target-addresses -PLT-1009 Add test data from preview-testnet * ./marconi-mamba/examples/test-json-rpc.http
3 participants
This PR contains:
Pre-submit checklist: