You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/StardustDocs/topics/gradleReference.md
+86-14Lines changed: 86 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ dataframes {
10
10
}
11
11
}
12
12
```
13
-
Note than name of the file and the interface are normalized: split by '_' and ' ' and joined to camel case.
13
+
Note that the name of the file and the interface are normalized: split by '_' and ' ' and joined to CamelCase.
14
14
You can set parsing options for CSV:
15
15
```kotlin
16
16
dataframes {
@@ -23,24 +23,36 @@ dataframes {
23
23
}
24
24
}
25
25
```
26
-
In this case output path will depend on your directory structure. For project with package `org.example` path will be `build/generated/dataframe/main/kotlin/org/example/dataframe/JetbrainsRepositories.Generated.kt
27
-
`. Note that name of the Kotlin file is derived from the name of the data file with the suffix `.Generated` and the package
28
-
is derived from the directory structure with child directory `dataframe`. The name of the **data schema** itself is `JetbrainsRepositories`. You could specify it explicitly:
26
+
In this case, the output path will depend on your directory structure.
27
+
For project with package `org.example` path will be `build/generated/dataframe/main/kotlin/org/example/dataframe/JetbrainsRepositories.Generated.kt
28
+
`.
29
+
30
+
Note that the name of the Kotlin file is derived from the name of the data file with the suffix
31
+
`.Generated` and the package
32
+
is derived from the directory structure with child directory `dataframe`.
33
+
34
+
The name of the **data schema** itself is `JetbrainsRepositories`.
you need to define a few parameters to establish a JDBC connection:
113
+
URL (passing to `data` field), username, and password.
114
+
115
+
Also, the `tableName` parameter should be specified to convert the data from the table with that name to the dataframe.
116
+
117
+
```kotlin
118
+
dataframes {
119
+
schema {
120
+
data ="jdbc:mariadb://localhost:3306/imdb"
121
+
name ="org.example.imdb.Actors"
122
+
jdbcOptions {
123
+
user ="root"
124
+
password ="pass"
125
+
tableName ="actors"
126
+
}
127
+
}
128
+
}
129
+
```
130
+
131
+
To generate a schema for the result of an SQL query,
132
+
you need to define the same parameters as before together with the SQL query to establish connection.
133
+
134
+
```kotlin
135
+
dataframes {
136
+
schema {
137
+
data ="jdbc:mariadb://localhost:3306/imdb"
138
+
name ="org.example.imdb.TarantinoFilms"
139
+
jdbcOptions {
140
+
user ="root"
141
+
password ="pass"
142
+
sqlQuery ="""
143
+
SELECT name, year, rank,
144
+
GROUP_CONCAT (genre) as "genres"
145
+
FROM movies JOIN movies_directors ON movie_id = movies.id
146
+
JOIN directors ON directors.id=director_id LEFT JOIN movies_genres ON movies.id = movies_genres.movie_id
147
+
WHERE directors.first_name = "Quentin" AND directors.last_name = "Tarantino"
148
+
GROUP BY name, year, rank
149
+
ORDER BY year
150
+
"""
151
+
}
152
+
}
153
+
}
154
+
```
155
+
156
+
**NOTE:** This is an experimental functionality and, for now,
157
+
we only support four databases: MariaDB, MySQL, PostgreSQL, and SQLite.
158
+
159
+
Additionally, support for JSON and date-time types is limited.
160
+
Please take this into consideration when using these functions.
90
161
91
162
## DSL reference
92
-
Inside `dataframes` you can configure parameters that will apply to all schemas. Configuration inside `schema` will override these defaults for specific schema.
93
-
Here is full DSL for declaring data schemas:
163
+
Inside `dataframes` you can configure parameters that will apply to all schemas.
164
+
Configuration inside `schema` will override these defaults for a specific schema.
0 commit comments