You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 2, 2024. It is now read-only.
Copy file name to clipboardExpand all lines: marconi/README.md
+41-41Lines changed: 41 additions & 41 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,19 +9,19 @@ A lightweight customizable solution for indexing and querying the Cardano blockc
9
9
10
10
### Built by IOG in Haskell
11
11
12
-
* dApp developers can index whatever is important to them.
13
-
* Marconi is an indexing solution for developers who need to store on-chain data in a local database.
12
+
* dApp developers can index whatever is important to them.
13
+
* Marconi is an indexing solution for developers who need to store on-chain data in a local database.
14
14
15
15
## Introduction
16
16
17
-
Marconi is a blockchain query solution that you can configure for the types of information that you need to index and according to how you need to structure your database schema.
17
+
Marconi is a blockchain query solution that you can configure for the types of information that you need to index and according to how you need to structure your database schema.
18
18
19
-
Marconi consists of these three components:
19
+
Marconi consists of these three components:
20
20
21
-
| Component | Description |
22
-
| --------- | ----------- |
23
-
| Streamer | Provides event stream of configured data types |
24
-
| Index | Indexes and stores the data in a database |
21
+
| Component | Description |
22
+
| --------- | ----------- |
23
+
| Streamer | Provides event stream of configured data types |
24
+
| Index | Indexes and stores the data in a database |
25
25
| Query | Fetches information from the database |
26
26
27
27
The streamer attaches to a node, providing an event stream of the configured data types. Your applications can use streams directly. Or streams may be passed to the index component which indexes and stores the data in a database. You can use the query component to fetch information from the database.
@@ -34,23 +34,23 @@ The philosophy behind Marconi is to maintain scalability by customizing the solu
34
34
35
35
### Handling Blockchain Rollback Events Properly
36
36
37
-
Marconi is designed to enable dApps to handle blockchain rollback events properly. Since the head of the chain is constantly advancing and rolling back as the consensus algorithm advances the chain, this causes challenging synchronization issues for dApps that need rapid updates of the chain state.
37
+
Marconi is designed to enable dApps to handle blockchain rollback events properly. Since the head of the chain is constantly advancing and rolling back as the consensus algorithm advances the chain, this causes challenging synchronization issues for dApps that need rapid updates of the chain state.
38
38
39
39
Marconi's index component handles rollbacks by distinguishing volatile and immutable blocks. A block on the top of the chain is volatile since it is subject to some probability of being rolled back. Eventually, when the block is deeper than the security parameter, the block becomes immutable. The Marconi indexer tracks this status and keeps the volatile blocks in memory until they become immutable. Once immutable, Marconi persists the blocks to the database.
40
40
41
41
### Specifying Slots to Query
42
42
43
-
Through the query interface, you can specify a range of slots to query or indicate that the query is for blocks deeper than a specified number of slots. As a result, you can use simple state management for applications that work only with immutable blocks.
43
+
Through the query interface, you can specify a range of slots to query or indicate that the query is for blocks deeper than a specified number of slots. As a result, you can use simple state management for applications that work only with immutable blocks.
44
44
45
45
### Querying Volatile Data
46
46
47
-
If your application requires faster response times, you can query volatile data on the head of the chain; however, you must take care to invalidate cached data when rollbacks occur.
47
+
If your application requires faster response times, you can query volatile data on the head of the chain; however, you must take care to invalidate cached data when rollbacks occur.
48
48
49
49
## API Layers
50
50
51
-
The core Marconi APIs are defined in Haskell in order to support simple use cases where all the components run together in a single process.
51
+
The core Marconi APIs are defined in Haskell in order to support simple use cases where all the components run together in a single process.
52
52
53
-
If your use cases require multiple synchronized instances for load balancing or for supporting different indexers, Marconi is designed with alternative transport layers on top of the core API that support network streaming and RPC calls.
53
+
If your use cases require multiple synchronized instances for load balancing or for supporting different indexers, Marconi is designed with alternative transport layers on top of the core API that support network streaming and RPC calls.
54
54
55
55
## What Differentiates Marconi
56
56
@@ -59,9 +59,9 @@ If your use cases require multiple synchronized instances for load balancing or
59
59
60
60
### DB Sync
61
61
62
-
Cardano DB-sync is IOG's current indexing solution, but it uses lots of memory and days/week to sync. DB-sync takes an index everything approach to put all information into a database that can support any query at the cost of requiring a large DB server to run it.
62
+
Cardano DB-sync is IOG's current indexing solution, but it uses lots of memory and days/week to sync. DB-sync takes an index everything approach to put all information into a database that can support any query at the cost of requiring a large DB server to run it.
63
63
64
-
In contrast, Marconi requires the user to customize it for the specific application so that only relevant information is streamed or stored. Marconi will be a scalable solution that allows the dApp developer to index whatever is important for their unique use case.
64
+
In contrast, Marconi requires the user to customize it for the specific application so that only relevant information is streamed or stored. Marconi will be a scalable solution that allows the dApp developer to index whatever is important for their unique use case.
65
65
66
66
### Ogmios
67
67
@@ -71,31 +71,31 @@ Ogmios exposes the basic node interface as a web service. This is a great option
71
71
72
72
Oura, implemented in Rust, functions as a notification system that indexers provide. It has good connectivity with cloud infrastructure like Kafka. Marconi provides similar functionality implemented in Haskell to make it easy for Haskell programmers to customize it.
73
73
74
-
### Scrolls
74
+
### Scrolls
75
75
76
-
Scrolls is the storage and query solution that our indexers provide. Like Scrolls, we can be selective for what is to be indexed. Scrolls can store into multiple databases. Marconi is currently focused on local DBs like SQL lite.
76
+
Scrolls is the storage and query solution that our indexers provide. Like Scrolls, we can be selective for what is to be indexed. Scrolls can store into multiple databases. Marconi is currently focused on local DBs like SQL lite.
77
77
78
-
### Carp
78
+
### Carp
79
79
80
80
## Customizing Marconi
81
81
82
-
You can customize indexing in the following ways:
82
+
You can customize indexing in the following ways:
83
83
84
-
1.*This statement is a placeholder:* A function that is given events and current state outputs notifications.
84
+
1.*This statement is a placeholder:* A function that is given events and current state outputs notifications.
85
85
86
86
2. Customize how a function performs queries by determining the following aspects of the function:
87
87
88
-
* The query type. Because Marconi uses an Abstract Data Type (ADT), you need to define the types of queries that the indexer responds to.
88
+
* The query type. Because Marconi uses an Abstract Data Type (ADT), you need to define the types of queries that the indexer responds to.
89
89
* Query for certain slot numbers by determining the point in the in-memory history where you want to run the query.
90
90
* Weigh considerations for in-memory and on-disk data. The query function produces a result by merging the in-memory and on-disk data. If you are concerned only with on-disk data, then you can query the database directly.
91
91
92
-
3. Customize a function that stores buffered data. While there is no connection to any storage mechanism, there is a nice API is available, and you are encouraged to change it to suit your needs. Changing it is very simple.
92
+
3. Customize a function that stores buffered data. While there is no connection to any storage mechanism, there is a nice API is available, and you are encouraged to change it to suit your needs. Changing it is very simple.
93
93
94
-
Marconi uses the `streaming` library so you can make use of its many combinators.
94
+
Marconi uses the `streaming` library so you can make use of its many combinators.
95
95
96
96
## Documentation
97
97
98
-
When it exists, we can link to our readthedocs user documentation.
98
+
When it exists, we can link to our readthedocs user documentation.
99
99
100
100
* User Guide
101
101
* Reference Guide
@@ -104,7 +104,7 @@ When it exists, we can link to our readthedocs user documentation.
104
104
105
105
## Intended Use Cases
106
106
107
-
Description of its primary intended use cases.
107
+
Description of its primary intended use cases.
108
108
109
109
* Sync with the Cardano blockchain (private/public testnet or mainnet) by reading all blocks from the genesis block to the current tip
110
110
* Index the syncing information based on the user’s predefined indexing
@@ -116,21 +116,21 @@ Description of its primary intended use cases.
116
116
117
117
### Use Case 3
118
118
119
-
## Example Queries
119
+
## Example Queries
120
120
121
-
Can we provide example queries?
121
+
Can we provide example queries?
122
122
123
123
## Architecture
124
124
125
125
* Provide a compelling diagram
126
126
127
127
## Architecture Decision Records
128
128
129
-
* Records of decisions that were made by the team.
130
-
* Discuss alternatives.
131
-
* Why did we choose one design over another?
132
-
* Konstantinos has some for Plutus Apps.
133
-
* Radu has some for Marconi.
129
+
* Records of decisions that were made by the team.
130
+
* Discuss alternatives.
131
+
* Why did we choose one design over another?
132
+
* Konstantinos has some for Plutus Apps.
133
+
* Radu has some for Marconi.
134
134
135
135
## System Requirements
136
136
@@ -142,29 +142,29 @@ The `marconi` executable is available as a nix flake.
142
142
143
143
You may either clone the [`plutus-apps`](https://github.com/input-output-hk/plutus-apps)
0 commit comments