Skip to content

Commit ee3ad5e

Browse files
committed
docs: clarify schema merging note
1 parent 3fd5be9 commit ee3ad5e

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

docs/hub/datasets-fenic.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,8 @@ df = session.read.csv("hf://datasets/username/dataset_name/*.csv", merge_schemas
7171
df = session.read.parquet("hf://datasets/username/dataset_name/*.parquet", merge_schemas=True)
7272
```
7373

74+
> **Note:** In fenic, a schema is the set of column names and their data types. When you enable `merge_schemas`, fenic tries to reconcile differences across files by filling missing columns with nulls and widening types where it can. Some layouts still cannot be merged—consult the fenic docs for [CSV schema merging limitations](https://docs.fenic.ai/latest/reference/fenic/?h=parquet#fenic.DataFrameReader.csv) and [Parquet schema merging limitations](https://docs.fenic.ai/latest/reference/fenic/?h=parquet#fenic.DataFrameReader.parquet).
75+
7476
### Authentication
7577

7678
To read private datasets, you need to set your Hugging Face token as an environment variable:

0 commit comments

Comments
 (0)