You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update of the documentation for the 0.12.1 release (#556)
* Add SQL schema generation guide to gradleReference.md
Extended the gradleReference documentation by adding a section on how to generate schema for existing SQL tables. This includes JDBC connection establishment and usage limitations and support for particular databases.
* Finished gradleReference.md changes
* Removed OpenAPI schemas section from schemasGradle.md file because it is a duplication of the information
* Add SQL schema import tutorial and update documentation
Added a new document providing a step-by-step guide on importing SQL metadata as a schema in Gradle project. In addition, minor documentation enhancements were made across multiple files to ensure clarity and precision. An unnecessary OpenAPI schemas section was removed from the schemasGradle.md file due to duplication.
* Update SQL schema import guide and enhance documentation.
* The modification of the guide in the SQL schema import document provides a clearer instruction on how to import data. The change improves the language use in the guide, ensuring better understandability for users following the instructions for importing schema data from an SQL table or query.
Copy file name to clipboardExpand all lines: docs/StardustDocs/topics/gradleReference.md
+86-14Lines changed: 86 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ dataframes {
10
10
}
11
11
}
12
12
```
13
-
Note than name of the file and the interface are normalized: split by '_' and ' ' and joined to camel case.
13
+
Note that the name of the file and the interface are normalized: split by '_' and ' ' and joined to CamelCase.
14
14
You can set parsing options for CSV:
15
15
```kotlin
16
16
dataframes {
@@ -23,24 +23,36 @@ dataframes {
23
23
}
24
24
}
25
25
```
26
-
In this case output path will depend on your directory structure. For project with package `org.example` path will be `build/generated/dataframe/main/kotlin/org/example/dataframe/JetbrainsRepositories.Generated.kt
27
-
`. Note that name of the Kotlin file is derived from the name of the data file with the suffix `.Generated` and the package
28
-
is derived from the directory structure with child directory `dataframe`. The name of the **data schema** itself is `JetbrainsRepositories`. You could specify it explicitly:
26
+
In this case, the output path will depend on your directory structure.
27
+
For project with package `org.example` path will be `build/generated/dataframe/main/kotlin/org/example/dataframe/JetbrainsRepositories.Generated.kt
28
+
`.
29
+
30
+
Note that the name of the Kotlin file is derived from the name of the data file with the suffix
31
+
`.Generated` and the package
32
+
is derived from the directory structure with child directory `dataframe`.
33
+
34
+
The name of the **data schema** itself is `JetbrainsRepositories`.
you need to define a few parameters to establish a JDBC connection:
113
+
URL (passing to `data` field), username, and password.
114
+
115
+
Also, the `tableName` parameter should be specified to convert the data from the table with that name to the dataframe.
116
+
117
+
```kotlin
118
+
dataframes {
119
+
schema {
120
+
data ="jdbc:mariadb://localhost:3306/imdb"
121
+
name ="org.example.imdb.Actors"
122
+
jdbcOptions {
123
+
user ="root"
124
+
password ="pass"
125
+
tableName ="actors"
126
+
}
127
+
}
128
+
}
129
+
```
130
+
131
+
To generate a schema for the result of an SQL query,
132
+
you need to define the same parameters as before together with the SQL query to establish connection.
133
+
134
+
```kotlin
135
+
dataframes {
136
+
schema {
137
+
data ="jdbc:mariadb://localhost:3306/imdb"
138
+
name ="org.example.imdb.TarantinoFilms"
139
+
jdbcOptions {
140
+
user ="root"
141
+
password ="pass"
142
+
sqlQuery ="""
143
+
SELECT name, year, rank,
144
+
GROUP_CONCAT (genre) as "genres"
145
+
FROM movies JOIN movies_directors ON movie_id = movies.id
146
+
JOIN directors ON directors.id=director_id LEFT JOIN movies_genres ON movies.id = movies_genres.movie_id
147
+
WHERE directors.first_name = "Quentin" AND directors.last_name = "Tarantino"
148
+
GROUP BY name, year, rank
149
+
ORDER BY year
150
+
"""
151
+
}
152
+
}
153
+
}
154
+
```
155
+
156
+
**NOTE:** This is an experimental functionality and, for now,
157
+
we only support four databases: MariaDB, MySQL, PostgreSQL, and SQLite.
158
+
159
+
Additionally, support for JSON and date-time types is limited.
160
+
Please take this into consideration when using these functions.
90
161
91
162
## DSL reference
92
-
Inside `dataframes` you can configure parameters that will apply to all schemas. Configuration inside `schema` will override these defaults for specific schema.
93
-
Here is full DSL for declaring data schemas:
163
+
Inside `dataframes` you can configure parameters that will apply to all schemas.
164
+
Configuration inside `schema` will override these defaults for a specific schema.
0 commit comments