Skip to content

Commit 3c47764

Browse files
CmdvCmdv
committed
add documentation on en-de-coders and DbInfo class
1 parent 87d52d5 commit 3c47764

File tree

2 files changed

+324
-11
lines changed

2 files changed

+324
-11
lines changed

‎doc/Readme.md

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -26,24 +26,26 @@ This directory contains various documentation files for setting up, configuring,
2626

2727
11. [Developer Hasql Instructions](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/hasql.md) - Guide for developers working with the new Hasql implementation, covering the DbAction monad, statement construction patterns, type-safe schema operations, and migration strategies from the previous Persistent ORM to ensure efficient and maintainable database interactions.
2828

29-
11. [Schema](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/schema.md) - Overview of the database schema used by the Cardano DB Sync Node, providing a detailed description of the tables, relationships, and data types used in the database.
29+
12. [Creating Hasql Encoders, Decoders, and DbInfo Instances](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/hasql-decode-encode-dbinfo.md) - Comprehensive developer guide for implementing database schema components with Hasql, covering DbInfo instance configuration, entity and record encoders/decoders, bulk operation patterns, type mapping conventions, and field naming requirements to ensure type-safe database interactions and proper schema correspondence.
3030

31-
12. [Schema Management](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/schema-management.md) - Instructions on managing the database schema and creating migrations, covering tools and techniques for making schema changes and ensuring they are applied correctly.
31+
13. [Schema](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/schema.md) - Overview of the database schema used by the Cardano DB Sync Node, providing a detailed description of the tables, relationships, and data types used in the database.
3232

33-
13. [Syncing and Rollbacks](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/syncing-and-rollbacks.md) - Details on the syncing procedure and handling rollbacks, explaining how the node syncs with the blockchain and manages rollbacks in case of errors or inconsistencies.
33+
14. [Schema Management](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/schema-management.md) - Instructions on managing the database schema and creating migrations, covering tools and techniques for making schema changes and ensuring they are applied correctly.
3434

35-
14. [Community Tools](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/community-tools.md) - Information on various community tools like Koios and Blockfrost, providing an overview of these tools, their features, and how they can be used to interact with Cardano DB Sync.
35+
15. [Syncing and Rollbacks](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/syncing-and-rollbacks.md) - Details on the syncing procedure and handling rollbacks, explaining how the node syncs with the blockchain and manages rollbacks in case of errors or inconsistencies.
3636

37-
15. [Interesting Queries](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/interesting-queries.md) - A collection of useful SQL queries for interacting with the database, including examples of queries for retrieving data, analyzing transactions, and generating reports.
37+
16. [Community Tools](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/community-tools.md) - Information on various community tools like Koios and Blockfrost, providing an overview of these tools, their features, and how they can be used to interact with Cardano DB Sync.
3838

39-
16. [Troubleshooting](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/troubleshooting.md) - Common issues and troubleshooting steps for Cardano DB Sync, providing solutions for various problems that users may encounter while running the node.
39+
17. [Interesting Queries](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/interesting-queries.md) - A collection of useful SQL queries for interacting with the database, including examples of queries for retrieving data, analyzing transactions, and generating reports.
4040

41-
17. [Release Process](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/release-process.md) - Detailed process for releasing new versions of Cardano DB Sync, covering the steps required to prepare, test, and publish a new release.
41+
18. [Troubleshooting](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/troubleshooting.md) - Common issues and troubleshooting steps for Cardano DB Sync, providing solutions for various problems that users may encounter while running the node.
4242

43-
18. [State Snapshot](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/state-snapshot.md) - Guide to creating and restoring state snapshots, explaining how to take snapshots of the database state and restore them when needed.
43+
19. [Release Process](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/release-process.md) - Detailed process for releasing new versions of Cardano DB Sync, covering the steps required to prepare, test, and publish a new release.
4444

45-
19. [Pool OffChain Data](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/pool-offchain-data.md) - Handling off-chain data for staking pools, providing details on managing off-chain data and integrating it with the Cardano DB Sync Node.
45+
20. [State Snapshot](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/state-snapshot.md) - Guide to creating and restoring state snapshots, explaining how to take snapshots of the database state and restore them when needed.
4646

47-
20. [SMASH](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/smash.md) - Information on the Stakepool Metadata Aggregation Server (SMASH), explaining the purpose of SMASH, how it works, and how to set it up.
47+
21. [Pool OffChain Data](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/pool-offchain-data.md) - Handling off-chain data for staking pools, providing details on managing off-chain data and integrating it with the Cardano DB Sync Node.
4848

49-
21. [HLint and Stylish Haskell](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/hlint-stylish-haskell.md) - Setting up `hlint` and `stylish-haskell` for code linting and formatting, providing instructions on configuring these tools to maintain code quality and consistency.
49+
22. [SMASH](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/smash.md) - Information on the Stakepool Metadata Aggregation Server (SMASH), explaining the purpose of SMASH, how it works, and how to set it up.
50+
51+
23. [HLint and Stylish Haskell](https://github.com/IntersectMBO/cardano-db-sync/blob/master/doc/hlint-stylish-haskell.md) - Setting up `hlint` and `stylish-haskell` for code linting and formatting, providing instructions on configuring these tools to maintain code quality and consistency.

‎doc/hasql-decode-encode-dbinfo.md

Lines changed: 311 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,311 @@
1+
# Creating Hasql Encoders, Decoders, and DbInfo Instances
2+
3+
## Data Type Definition
4+
5+
```haskell
6+
-- Example data type
7+
data MaTxOutAddress = MaTxOutAddress
8+
{ maTxOutAddressIdent :: !Id.MultiAssetId
9+
, maTxOutAddressQuantity :: !DbWord64
10+
, maTxOutAddressTxOutId :: !Id.TxOutAddressId
11+
}
12+
deriving (Eq, Show, Generic)
13+
14+
-- Required: Key type instance
15+
type instance Key MaTxOutAddress = Id.MaTxOutAddressId
16+
```
17+
18+
## DbInfo Instance
19+
20+
```haskell
21+
instance DbInfo MaTxOutAddress where
22+
-- Explicit table name (overrides default snake_case conversion)
23+
tableName _ = "ma_tx_out"
24+
25+
-- Column names in database order (excludes auto-generated 'id' column)
26+
columnNames _ = NE.fromList ["quantity", "tx_out_id", "ident"]
27+
28+
-- For bulk operations: (column_name, postgres_array_type)
29+
unnestParamTypes _ =
30+
[ ("ident", "bigint[]")
31+
, ("quantity", "bigint[]")
32+
, ("tx_out_id", "bigint[]")
33+
]
34+
35+
-- Optional: Unique constraint columns
36+
uniqueFields _ = ["unique_col1", "unique_col2"]
37+
38+
-- Optional: JSONB columns
39+
jsonbFields _ = ["json_column"]
40+
```
41+
42+
### DbInfo Configuration Options
43+
44+
```haskell
45+
instance DbInfo SomeTable where
46+
-- Table name (default: snake_case of type name)
47+
tableName _ = "custom_table_name"
48+
49+
-- Column names (default: derived from field names)
50+
columnNames _ = NE.fromList ["col1", "col2", "col3"]
51+
52+
-- Unique constraints
53+
uniqueFields _ = ["col1", "col2"] -- Multi-column unique constraint
54+
55+
-- Bulk unique fields (for bulk operations only)
56+
bulkUniqueFields _ = ["bulk_unique_col"]
57+
58+
-- JSONB columns (require ::jsonb casting)
59+
jsonbFields _ = ["metadata", "config"]
60+
61+
-- Enum columns with their types
62+
enumFields _ = [("status", "status_type"), ("priority", "priority_type")]
63+
64+
-- Generated columns (excluded from inserts)
65+
generatedFields _ = ["created_at", "updated_at"]
66+
67+
-- Bulk operation parameters
68+
unnestParamTypes _ =
69+
[ ("col1", "bigint[]")
70+
, ("col2", "text[]")
71+
, ("col3", "boolean[]")
72+
]
73+
```
74+
75+
## Entity Decoder
76+
77+
```haskell
78+
entityMaTxOutAddressDecoder :: D.Row (Entity MaTxOutAddress)
79+
entityMaTxOutAddressDecoder =
80+
Entity
81+
<$> Id.idDecoder Id.MaTxOutAddressId -- Entity ID
82+
<*> maTxOutAddressDecoder -- Entity data
83+
```
84+
85+
## Record Decoder
86+
87+
```haskell
88+
maTxOutAddressDecoder :: D.Row MaTxOutAddress
89+
maTxOutAddressDecoder =
90+
MaTxOutAddress
91+
<$> Id.idDecoder Id.MultiAssetId -- Foreign key ID
92+
<*> D.column (D.nonNullable $ DbWord64 . fromIntegral <$> D.int8) -- DbWord64
93+
<*> Id.idDecoder Id.TxOutAddressId -- Another foreign key ID
94+
```
95+
96+
### Decoder Patterns
97+
98+
```haskell
99+
-- Basic types
100+
<*> D.column (D.nonNullable D.text) -- Text
101+
<*> D.column (D.nonNullable D.bool) -- Bool
102+
<*> D.column (D.nonNullable D.bytea) -- ByteString
103+
<*> D.column (D.nonNullable $ fromIntegral <$> D.int8) -- Word64/Int
104+
105+
-- Nullable types
106+
<*> D.column (D.nullable D.text) -- Maybe Text
107+
<*> D.column (D.nullable D.bytea) -- Maybe ByteString
108+
109+
-- ID types
110+
<*> Id.idDecoder Id.SomeId -- !Id.SomeId
111+
<*> Id.maybeIdDecoder Id.SomeId -- !(Maybe Id.SomeId)
112+
113+
-- Custom types with decoders
114+
<*> dbLovelaceDecoder -- DbLovelace
115+
<*> D.column (D.nonNullable utcTimeAsTimestampDecoder) -- UTCTime
116+
<*> rewardSourceDecoder -- Custom enum
117+
118+
-- Wrapped types
119+
<*> D.column (D.nonNullable $ DbWord64 . fromIntegral <$> D.int8) -- DbWord64
120+
```
121+
122+
## Entity Encoder
123+
124+
```haskell
125+
entityMaTxOutAddressEncoder :: E.Params (Entity MaTxOutAddress)
126+
entityMaTxOutAddressEncoder =
127+
mconcat
128+
[ entityKey >$< Id.idEncoder Id.getMaTxOutAddressId -- Entity ID
129+
, entityVal >$< maTxOutAddressEncoder -- Entity data
130+
]
131+
```
132+
133+
## Record Encoder
134+
135+
```haskell
136+
maTxOutAddressEncoder :: E.Params MaTxOutAddress
137+
maTxOutAddressEncoder =
138+
mconcat
139+
[ maTxOutAddressIdent >$< Id.idEncoder Id.getMultiAssetId
140+
, maTxOutAddressQuantity >$< E.param (E.nonNullable $ fromIntegral . unDbWord64 >$< E.int8)
141+
, maTxOutAddressTxOutId >$< Id.idEncoder Id.getTxOutAddressId
142+
]
143+
```
144+
145+
### Encoder Patterns
146+
147+
```haskell
148+
-- Basic types
149+
field >$< E.param (E.nonNullable E.text) -- Text
150+
field >$< E.param (E.nonNullable E.bool) -- Bool
151+
field >$< E.param (E.nonNullable E.bytea) -- ByteString
152+
field >$< E.param (E.nonNullable $ fromIntegral >$< E.int8) -- Word64/Int
153+
154+
-- Nullable types
155+
field >$< E.param (E.nullable E.text) -- Maybe Text
156+
field >$< E.param (E.nullable E.bytea) -- Maybe ByteString
157+
158+
-- ID types
159+
field >$< Id.idEncoder Id.getSomeId -- Id.SomeId
160+
field >$< Id.maybeIdEncoder Id.getSomeId -- Maybe Id.SomeId
161+
162+
-- Custom types with encoders
163+
field >$< dbLovelaceEncoder -- DbLovelace
164+
field >$< E.param (E.nonNullable utcTimeAsTimestampEncoder) -- UTCTime
165+
field >$< rewardSourceEncoder -- Custom enum
166+
167+
-- Wrapped types
168+
field >$< E.param (E.nonNullable $ fromIntegral . unDbWord64 >$< E.int8) -- DbWord64
169+
```
170+
171+
## Bulk Encoder
172+
173+
```haskell
174+
maTxOutAddressBulkEncoder :: E.Params ([Id.MultiAssetId], [DbWord64], [Id.TxOutAddressId])
175+
maTxOutAddressBulkEncoder =
176+
contrazip3
177+
(bulkEncoder $ E.nonNullable $ Id.getMultiAssetId >$< E.int8)
178+
(bulkEncoder $ E.nonNullable $ fromIntegral . unDbWord64 >$< E.int8)
179+
(bulkEncoder $ E.nonNullable $ Id.getTxOutAddressId >$< E.int8)
180+
```
181+
182+
### Bulk Encoder Utilities
183+
184+
```haskell
185+
-- For 2 fields
186+
contrazip2 encoder1 encoder2
187+
188+
-- For 3 fields
189+
contrazip3 encoder1 encoder2 encoder3
190+
191+
-- For 4 fields
192+
contrazip4 encoder1 encoder2 encoder3 encoder4
193+
194+
-- For 5 fields
195+
contrazip5 encoder1 encoder2 encoder3 encoder4 encoder5
196+
197+
-- Pattern for each field
198+
(bulkEncoder $ E.nonNullable $ transformation >$< E.baseType)
199+
(bulkEncoder $ E.nullable $ transformation >$< E.baseType) -- For nullable
200+
```
201+
202+
## Complete Example
203+
204+
```haskell
205+
-- Data type
206+
data EventInfo = EventInfo
207+
{ eventInfoTxId :: !(Maybe Id.TxId)
208+
, eventInfoEpoch :: !Word64
209+
, eventInfoType :: !Text
210+
, eventInfoExplanation :: !(Maybe Text)
211+
}
212+
deriving (Eq, Show, Generic)
213+
214+
type instance Key EventInfo = Id.EventInfoId
215+
216+
-- DbInfo instance
217+
instance DbInfo EventInfo where
218+
tableName _ = "event_info"
219+
columnNames _ = NE.fromList ["tx_id", "epoch", "type", "explanation"]
220+
unnestParamTypes _ =
221+
[ ("tx_id", "bigint[]")
222+
, ("epoch", "bigint[]")
223+
, ("type", "text[]")
224+
, ("explanation", "text[]")
225+
]
226+
227+
-- Entity decoder
228+
entityEventInfoDecoder :: D.Row (Entity EventInfo)
229+
entityEventInfoDecoder =
230+
Entity
231+
<$> Id.idDecoder Id.EventInfoId
232+
<*> eventInfoDecoder
233+
234+
-- Record decoder
235+
eventInfoDecoder :: D.Row EventInfo
236+
eventInfoDecoder =
237+
EventInfo
238+
<$> Id.maybeIdDecoder Id.TxId
239+
<*> D.column (D.nonNullable $ fromIntegral <$> D.int8)
240+
<*> D.column (D.nonNullable D.text)
241+
<*> D.column (D.nullable D.text)
242+
243+
-- Entity encoder
244+
entityEventInfoEncoder :: E.Params (Entity EventInfo)
245+
entityEventInfoEncoder =
246+
mconcat
247+
[ entityKey >$< Id.idEncoder Id.getEventInfoId
248+
, entityVal >$< eventInfoEncoder
249+
]
250+
251+
-- Record encoder
252+
eventInfoEncoder :: E.Params EventInfo
253+
eventInfoEncoder =
254+
mconcat
255+
[ eventInfoTxId >$< Id.maybeIdEncoder Id.getTxId
256+
, eventInfoEpoch >$< E.param (E.nonNullable $ fromIntegral >$< E.int8)
257+
, eventInfoType >$< E.param (E.nonNullable E.text)
258+
, eventInfoExplanation >$< E.param (E.nullable E.text)
259+
]
260+
261+
-- Bulk encoder
262+
eventInfoBulkEncoder :: E.Params ([Maybe Id.TxId], [Word64], [Text], [Maybe Text])
263+
eventInfoBulkEncoder =
264+
contrazip4
265+
(bulkEncoder $ E.nullable $ Id.getTxId >$< E.int8)
266+
(bulkEncoder $ E.nonNullable $ fromIntegral >$< E.int8)
267+
(bulkEncoder $ E.nonNullable E.text)
268+
(bulkEncoder $ E.nullable E.text)
269+
```
270+
271+
## Field Naming Convention
272+
273+
- Fields must start with the lowercased type name
274+
- Follow with uppercase letter for the actual field name
275+
- Example: `MaTxOutAddress` → `maTxOutAddressFieldName`
276+
277+
## Type Mapping Reference
278+
279+
| Haskell Type | Decoder | Encoder |
280+
|-------------|---------|---------|
281+
| `Text` | `D.text` | `E.text` |
282+
| `Bool` | `D.bool` | `E.bool` |
283+
| `ByteString` | `D.bytea` | `E.bytea` |
284+
| `Word64` | `fromIntegral <$> D.int8` | `fromIntegral >$< E.int8` |
285+
| `UTCTime` | `utcTimeAsTimestampDecoder` | `utcTimeAsTimestampEncoder` |
286+
| `DbLovelace` | `dbLovelaceDecoder` | `dbLovelaceEncoder` |
287+
| `DbWord64` | `DbWord64 . fromIntegral <$> D.int8` | `fromIntegral . unDbWord64 >$< E.int8` |
288+
| `Id.SomeId` | `Id.idDecoder Id.SomeId` | `Id.idEncoder Id.getSomeId` |
289+
| `Maybe Id.SomeId` | `Id.maybeIdDecoder Id.SomeId` | `Id.maybeIdEncoder Id.getSomeId` |
290+
291+
## Common Patterns
292+
293+
### JSON Fields
294+
```haskell
295+
instance DbInfo MyTable where
296+
jsonbFields _ = ["metadata"]
297+
298+
-- In decoder/encoder, treat as Text with special handling
299+
```
300+
301+
### Unique Constraints
302+
```haskell
303+
instance DbInfo MyTable where
304+
uniqueFields _ = ["field1", "field2"] -- Composite unique constraint
305+
```
306+
307+
### Generated Fields
308+
```haskell
309+
instance DbInfo MyTable where
310+
generatedFields _ = ["created_at"] -- Excluded from inserts
311+
```

0 commit comments

Comments
 (0)