Skip to content
This repository was archived by the owner on Oct 10, 2025. It is now read-only.

Add doc for duckdb/sqlite/postgres's type conversion #348

Merged
prrao87 merged 10 commits into from
Feb 3, 2025
Merged

Add doc for duckdb/sqlite/postgres's type conversion #348

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
63e1871
Add doc for duckdb's type conversion
acquamarin Jan 27, 2025
b57a0b7
Update rdbms.mdx
acquamarin Jan 27, 2025
495edbe
Update rdbms.mdx
acquamarin Jan 27, 2025
bd17f6d
Update rdbms.mdx
acquamarin Jan 30, 2025
110c0ca
Update rdbms.mdx
acquamarin Jan 30, 2025
b7ce423
Update src/content/docs/extensions/attach/rdbms.mdx
acquamarin Feb 3, 2025
4b129e9
Update src/content/docs/extensions/attach/rdbms.mdx
acquamarin Feb 3, 2025
81e5647
Update src/content/docs/extensions/attach/rdbms.mdx
acquamarin Feb 3, 2025
60f4c4a
Update src/content/docs/extensions/attach/rdbms.mdx
acquamarin Feb 3, 2025
ea4cd94
Update rdbms.mdx
acquamarin Feb 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 123 additions & 18 deletions src/content/docs/extensions/attach/rdbms.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,42 @@ Result:
└──────────────┘
```


#### 3. Scan from DuckDB tables
#### 3. Data type mapping from DuckDB to Kùzu

The table below shows the mapping from duckdb's type to Kùzu's type:
| Data type in DuckDB | Corresponding data type in Kùzu |
|-----------------------------|----------------------------------|
| BIGINT | INT64 |
| BIT | UNSUPPORTED |
| BLOB | BLOB |
| BOOLEAN | BOOL |
| DATE | DATE |
| DECIMAL(prec, scale) | DECIMAL(prec, scale) |
| DOUBLE | DOUBLE |
| FLOAT | FLOAT |
| HUGEINT | INT128 |
| INTEGER | INT32 |
| INTERVAL | INTERVAL |
| SMALLINT | INT16 |
| TIME | UNSUPPORTED |
| TIMESTAMP WITH TIME ZONE | UNSUPPORTED |
| TIMESTAMP | TIMESTAMP |
| TINYINT | INT8 |
| UBIGINT | UINT64 |
| UHUGEINT | UNSUPPORTED |
| UINTEGER | UINT32 |
| USMALLINT | UINT16 |
| UTINYINT | UINT8 |
| UUID | UUID |
| VARCHAR | STRING |
| ENUM | UNSUPPORTED |
| ARRAY | ARRAY |
| LIST | LIST |
| MAP | MAP |
| STRUCT | STRUCT |
| UNION | UNION |

#### 4. Scan from DuckDB tables

Finally, we can utilize the `LOAD FROM` statement to scan the `person` table. Note that you need to prefix the
external `person` table with the database alias (in our example `uw`). See the `USE` statement which allows you to
Expand All @@ -137,7 +171,7 @@ Result:
---------------
```

#### 4. USE: Reference database without alias
#### 5. USE: Reference database without alias

You can use the `USE` statement for attached databases to use a default database name for future operations.
This can be used when reading from an attached database to avoid specifying the full database name
Expand All @@ -164,7 +198,7 @@ LOAD FROM person
RETURN *
```

#### 5. Copy data from DuckDB tables
#### 6. Copy data from DuckDB tables

One important use case of the external RDBMS extensions is to facilitate seamless data transfer from the external RDBMS to Kùzu.
In this example, we continue using the `university.db` database created in the last step, but this time,
Expand All @@ -187,7 +221,7 @@ If the schemas are not the same, e.g., `Person` contains only `name` property wh
COPY Person FROM (LOAD FROM uw.person RETURN name);
```

#### 6. Query the data in Kùzu
#### 7. Query the data in Kùzu

Finally, we can verify the data in the `Person` table in Kùzu.

Expand All @@ -210,7 +244,7 @@ Result:
------------------
```

#### 7. Clear attached database schema cache
#### 8. Clear attached database schema cache

To avoid redundantly retrieving schema information from attached databases, Kùzu maintains a schema cache
including table names and their respective columns and types. Should modifications occur in the schema
Expand Down Expand Up @@ -319,7 +353,56 @@ The below table lists some common connection string parameters:
| `password` | Postgres password | [empty] |
| `port` | Port number | 5432 |

#### 3. Scan from PostgreSQL tables
#### 3. Data type mapping from PostgreSQL to Kùzu

The table below shows the mapping from PostgreSQL's type to Kùzu's type:
| PostgreSQL Data Type | Corresponding Data Type in Kùzu |
|-------------------------------------------|----------------------------------|
| bigint (int8) | INT64 |
| bigserial (serial8) | INT64 |
| bit [ (n) ] | STRING |
| bit varying [ (n) ] (varbit [ (n) ]) | STRING |
| boolean (bool) | BOOL |
| box | DOUBLE[] |
| bytea | BLOB |
| character [ (n) ] (char [ (n) ]) | STRING |
| character varying [ (n) ] (varchar [ (n)])| STRING |
| cidr | STRING |
| circle | DOUBLE[] |
| date | DATE |
| double precision (float8) | DOUBLE |
| inet | STRING |
| integer (int, int4) | INT32 |
| interval [ fields ] [ (p) ] | INTERVAL |
| json | JSON |
| line | DOUBLE[] |
| lseg | DOUBLE[] |
| macaddr | STRING |
| macaddr8 | STRING |
| money | STRING |
| numeric [ (p, s) ] (decimal [ (p, s) ]) | DECIMAL |
| path | DOUBLE[] |
| pg_lsn | STRING |
| pg_snapshot | STRING |
| point | STRUCT(x DOUBLE, y DOUBLE) |
| polygon | DOUBLE[] |
| real (float4) | FLOAT |
| smallint (int2) | INT16 |
| smallserial (serial2) | INT16 |
| serial (serial4) | INT32 |
| text | STRING |
| time [ (p) ] [ without time zone ] | UNSUPPORTED |
| time [ (p) ] with time zone (timetz) | UNSUPPORTED |
| timestamp [ (p) ] [ without time zone ] | TIMESTAMP |
| timestamp [ (p) ] with time zone (timestamptz) | UNSUPPORTED |
| tsquery | STRING |
| tsvector | STRING |
| txid_snapshot | STRING |
| uuid | UUID |
| xml | STRING |


#### 4. Scan from PostgreSQL tables

Finally, we can utilize the `LOAD FROM` statement to scan the `Person` table.

Expand All @@ -344,7 +427,7 @@ Result:
---------------
```

#### 4. USE: Reference database without alias
#### 5. USE: Reference database without alias

You can use the `USE` statement for attached databases to use a default database name for future operations.
This can be used when reading from an attached database to avoid specifying the full database name
Expand All @@ -371,7 +454,7 @@ LOAD FROM person
RETURN *
```

#### 5. Copy data from PostgreSQL tables
#### 6. Copy data from PostgreSQL tables

One important use case of the external RDBMS extensions is to facilitate seamless data transfer from the external RDBMS to Kùzu.
In this example, we continue using the `university.db` database created in the last step, but this time,
Expand All @@ -394,7 +477,7 @@ If the schemas are not the same, e.g., `Person` contains only `name` property wh
COPY Person FROM (LOAD FROM uw.person RETURN name);
```

#### 6. Query the data in Kùzu
#### 7. Query the data in Kùzu

Finally, we can verify the data in the `Person` table in Kùzu.

Expand All @@ -417,7 +500,7 @@ Result:
------------------
```

#### 7. Clear attached database schema cache
#### 8. Clear attached database schema cache

To avoid redundantly retrieving schema information from attached databases, Kùzu maintains a schema cache
including table names and their respective columns and types. Should modifications occur in the schema
Expand All @@ -431,7 +514,7 @@ CALL clear_attached_db_cache() RETURN *
Note: If you have attached to databases from different
RDBMSs, say Postgres, DuckDB, and Sqlite, this call will clear the cache for all of them.

#### 8. Detach database
#### 9. Detach database

To detach a database, use `DETACH [ALIAS]` as follows:

Expand Down Expand Up @@ -489,7 +572,29 @@ the alias `uw`:
ATTACH 'university.db' AS uw (dbtype sqlite);
```

#### 3. Scan from SQLite tables
#### 3. Data type mapping from SQLite to Kùzu

The table below shows the mapping from SQLite's type to Kùzu's type:
| SQLite Storage Class / Datatype | Corresponding Data Type in Kùzu |
|--------------------------------------------|----------------------------------|
| NULL | BLOB |
| INTEGER | INT64 |
| REAL | DOUBLE |
| TEXT | STRING |
| BLOB | BLOB |
| BOOLEAN | INT64 |
| DATE | DATE |
| TIME | TIMESTAMP |

Note: Sqlite uses a [dynamic type system](https://www.sqlite.org/datatype3.html), meaning that a column in sqlite can store values with different types. The option: `sqlite_all_varchar_option` is provided to scan such columns in Kùzu.
Usage:
```
`CALL sqlite_all_varchar_option=<OPTION>`
```
If `sqlite_all_varchar_option` is set to true, all sqlite columns will be treated and scanned as `VAR_CHAR` columns.
If `sqlite_all_varchar_option` is set to false, trying to scan a column with values incompatible with the specified data type will result in a runtime exception.

#### 4. Scan from SQLite tables

Finally, we can utilize the `LOAD FROM` statement to scan the `person` table.

Expand All @@ -514,7 +619,7 @@ Result:
---------------
```

#### 4. USE: Reference database without alias
#### 5. USE: Reference database without alias

You can use the `USE` statement for attached databases to use a default database name for future operations.
This can be used when reading from an attached database to avoid specifying the full database name
Expand All @@ -541,7 +646,7 @@ LOAD FROM person
RETURN *
```

#### 5. Copy data from SQLite tables
#### 6. Copy data from SQLite tables

One important use case of the external RDBMS extensions is to facilitate seamless data transfer from the external RDBMS to Kùzu.
In this example, we continue using the `university.db` database created in the last step, but this time,
Expand All @@ -564,7 +669,7 @@ If the schemas are not the same, e.g., `Person` contains only `name` property wh
COPY Person FROM (LOAD FROM uw.person RETURN name);
```

#### 6. Query the data in Kùzu
#### 7. Query the data in Kùzu

Finally, we can verify the data in the `Person` table in Kùzu.

Expand All @@ -587,7 +692,7 @@ Result:
------------------
```

#### 7. Clear attached database schema cache
#### 8. Clear attached database schema cache

To avoid redundantly retrieving schema information from attached databases, Kùzu maintains a schema cache
including table names and their respective columns and types. Should modifications occur in the schema
Expand All @@ -601,7 +706,7 @@ CALL clear_attached_db_cache() RETURN *
Note: If you have attached to databases from different
RDBMSs, say Postgres, DuckDB, and Sqlite, this call will clear the cache for all of them.

#### 8. Detach database
#### 9. Detach database

To detach a database, use `DETACH [ALIAS]` as follows:

Expand Down