Skip to content

Commit dd825a4

Browse files
quinnjquinnj
committed
Update the docs according to the main UI name changes as well as the DataStreams implementation
1 parent 7ab4f6d commit dd825a4

File tree

1 file changed

+89
-53
lines changed

1 file changed

+89
-53
lines changed

‎README.md

Lines changed: 89 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -13,31 +13,25 @@ A Julia interface to the SQLite library and support for the `DataStreams` data p
1313
## Package Documentation
1414

1515
#### Types/Functions
16-
* `SQLiteDB(file::String; UTF16::Bool=false)`
16+
* `SQLite.DB(file::AbstractString)`
1717

18-
`SQLiteDB` requires the `file` string argument as the name of either a pre-defined SQLite database to be opened, or if the file doesn't exist, a database will be created.
18+
`SQLite.DB` requires the `file` string argument as the name of either a pre-defined SQLite database to be opened, or if the file doesn't exist, a database will be created.
1919

20-
The `SQLiteDB` object represents a single connection to an SQLite database. All other SQLite.jl functions take an `SQLiteDB` as the first argument as context.
20+
The `SQLite.DB` object represents a single connection to an SQLite database. All other SQLite.jl functions take an `SQLite.DB` as the first argument as context.
2121

22-
The keyword argument `UTF16` can be set to true to force the creation of a database with UTF16-encoded strings.
22+
To create an in-memory temporary database, call `SQLite.DB()`.
2323

24-
To create an in-memory temporary database, one can also call `SQLiteDB(":memory:")`.
24+
The `SQLite.DB` will automatically closed/shutdown when it goes out of scope (i.e. the end of the Julia session, end of a function call wherein it was created, etc.)
2525

26-
* `close(db::SQLiteDB)`
26+
* `SQLite.Stmt(db::SQLite.DB, sql::String)`
2727

28-
Closes an open database connection.
28+
Constructs and prepares (compiled by the SQLite library) an SQL statement in the context of the provided `db`. Note the SQL statement is not actually executed, but only compiled (mainly for usage where the same statement is repeated with different parameters bound as values. See `bind!` below).
2929

30-
* `SQLiteStmt(db::SQLiteDB, sql::String)`
30+
The `SQLite.Stmt` will automatically closed/shutdown when it goes out of scope (i.e. the end of the Julia session, end of a function call wherein it was created, etc.)
3131

32-
Constructs and prepares (compiled by SQLite library) an SQL statement in the context of the provided `db`. Note the SQL statement is not actually executed, but only compiled (mainly for usage where the same statement is repeated with different parameters bound as values. See `bind` below).
32+
* `SQLite.bind!(stmt::SQLite.Stmt,index,value)`
3333

34-
* `close(stmt::SQLiteStmt)`
35-
36-
Closes or finalizes an SQLiteStmt. A closed `SQLiteStmt` can no longer be executed.
37-
38-
* `bind(stmt::SQLiteStmt,index,value)`
39-
40-
Used to bind values to parameter placeholders in an prepared `SQLiteStmt`. From the SQLite documentation:
34+
Used to bind values to parameter placeholders in an prepared `SQLite.Stmt`. From the SQLite documentation:
4135

4236
> Usually, though, it is not useful to evaluate exactly the same SQL statement more than once. More often, one wants to evaluate similar statements. For example, you might want to evaluate an INSERT statement multiple times though with different values to insert. To accommodate this kind of flexibility, SQLite allows SQL statements to contain parameters which are "bound" to values prior to being evaluated. These values can later be changed and the same prepared statement can be evaluated a second time using the new values.
4337
@@ -51,46 +45,88 @@ A Julia interface to the SQLite library and support for the `DataStreams` data p
5145
5246
> In the examples above, NNN is an integer value and AAA is an identifier. A parameter initially has a value of NULL. Prior to calling sqlite3_step() for the first time or immediately after sqlite3_reset(), the application can invoke one of the sqlite3_bind() interfaces to attach values to the parameters. Each call to sqlite3_bind() overrides prior bindings on the same parameter.
5347
54-
* `execute(stmt::SQLiteStmt)`
55-
`execute(db::SQLiteDB, sql::String)`
48+
* `SQLite.execute!(stmt::SQLite.Stmt)`
49+
`SQLite.execute!(db::SQLite.DB, sql::String)`
50+
51+
Used to execute a prepared `SQLite.Stmt`. The 2nd method is a convenience method to pass in an SQL statement as a string which gets prepared and executed in one call. This method does not check for or return any results, hence it is only useful for database manipulation methods (i.e. ALTER, CREATE, UPDATE, DROP). To return results, see `SQLite.query` below.
52+
53+
* `SQLite.query(db::SQLite.DB, sql::String, values=[])`
54+
55+
An SQL statement `sql` is prepared, executed in the context of `db`, and results, if any, are returned. The return value is a `Data.Table` by default from the `DataStreams.jl` package. The `Data.Table` has a field `.data` which is a `Vector{NullableVector}` which holds the columns of data returned from the `sql` statement.
56+
57+
The values in `values` are used in parameter binding (see `bind!` above). If your statement uses nameless parameters `values` must be a `Vector` of the values you wish to bind to your statment. If your statement uses named parameters `values` must be a Dict where the keys are of type `Symbol`. The key must match an identifier name in the statement (the name **should not** include the ':', '@' or '$' prefix).
58+
59+
* `SQLite.drop!(db::SQLite.DB,table::String;ifexists::Bool=false)`
60+
`SQLite.dropindex!(db::SQLite.DB,index::String;ifexists::Bool=false)`
61+
62+
These are pretty self-explanatory. They're really just a convenience methods to execute DROP TABLE/DROP INDEX commands, while also calling "VACUUM" to clean out freed memory from the database.
63+
64+
* `SQLite.createindex!(db::DB,table::AbstractString,index::AbstractString,cols;unique::Bool=true,ifnotexists::Bool=false)`
65+
66+
Create a new index named `index` for `table` with the columns in `cols`, which should be a comma delimited list of column names. `unique` indicates whether the index will have unique values or not. `ifnotexists` will not throw an error if the index already exists.
67+
68+
* `SQLite.removeduplicates!(db,table::AbstractString,cols::AbstractString)`
69+
70+
A convenience method for the common task of removing duplicate rows in a dataset according to some subset of columns that make up a "primary key".
71+
72+
* `SQLite.tables(db::SQLite.DB)`
73+
74+
List the tables in an SQLite database `db`
75+
76+
* `SQLite.columns(db::SQLite.DB,table::AbstractString)`
77+
78+
List the columns in an SQLite table
79+
80+
* `SQLite.indices(db::SQLite.DB)`
81+
82+
List the indices that have been created in `db`
83+
84+
* `SQLite.Source(db::DB, sql; rows=0, stricttypes::Bool=true)`
85+
86+
Create an `SQLite.Source` type in `db` with the SQL statement `sql`. This prepares and executes the statement, but does not return any data. The source is ready to be streamed to any sink type with an appropriate `Data.stream!` method defined. `rows` can be used to fetch a specific number of rows if known before hand and can help in pre-allocating for sinks. `stricttypes=false` can be used to relax type requirements on returned columns; typically, each value in a column must be of the same type, but SQLite itself uses a fairly weak typing scheme which allows any value to be in a column, regardless of type. Working with data in Julia is much more useful in strongly-typed structures, so by default we try to return strongly-typed columns, but this can be relaxed.
87+
88+
* `SQLite.Source(sink::SQLite.Sink,sql)`
89+
90+
Creates an `SQLite.Source` type according the `sql` command executed in the same `db` as `sink`. By default, the `sql` command is `select * from $(sink.tablename)`.
91+
92+
* `Data.stream!(source::SQLite.Source,::Type{Data.Table})`
5693

57-
Used to execute prepared `SQLiteStmt`. The 2nd method is a convenience method to pass in an SQL statement as a string which gets prepared and executed in one call. This method does not check for or return any results, hence it is only useful for database manipulation methods (i.e. ALTER, CREATE, UPDATE, DROP). To return results, see `query` below. Also consider the `create`, `droptable`, and `append` methods for manipulation statements as further SQLite performance tricks are incorporated automatically.
94+
Create a `Data.Table` and stream the data from an `SQLite.Source` into it.
5895

59-
* `query(db::SQLiteDB, sql::String, values=[])`
96+
* `Data.stream!(source::SQLite.Source,sink::CSV.Sink;header::Bool=true)`
6097

61-
An SQL statement `sql` is prepared, executed in the context of `db`, and results, if any, are returned. The return values are a `(String[],Any[])` tuple representing `(column names, result values)`.
98+
stream the data from `source` to a `CSV.Sink`. `header` indicates whether the column names will be written first to `sink`.
6299

63-
The values in `values` are used in parameter binding (see `bind` above). If your statement uses nameless parameters `values` must be a `Vector` of the values you wish to bind to your statment. If your statement uses named parameters `values` must be a Dict where the keys are of type `Symbol`. The key must match an identifier name in the statement (the name **does not** include the ':', '@' or '$' prefix).
100+
* `SQLite.Sink(schema::Data.Schema,db::DB,tablename;temp::Bool=false,ifnotexists::Bool=true)`
64101

65-
* `create(db::SQLiteDB,name::String,table::AbstractMatrix,
66-
colnames=String[],coltypes=DataType[];temp::Bool=false)`
102+
Create a new SQLite table as a sink to stream data to in `db`. `schema` is used to create the column names and types. If no `tablename` is supplied, one will be generated. `temp` indicates whether the table will be temporary, i.e. if it will be automatically destroyed when the `db` is closed.
67103

68-
Convenience method for "CREATE TABLE" and "INSERT" statements to insert `table` as an SQLite table in the `db` database. `name` will be the name of the SQLite table. `table` can be any AbstractMatrix that supports the `table[i,j]` getindex method. `colnames` is an optional vector to be used as the names of the columns for the SQLite table. `coltypes` is also an optional vector to specify the Julia types of the columns in `table`. The optional keyword `temp` can be set to `true` to specify the creation of a temporary table that will be destroyed when the database connection is closed.
104+
* `SQLite.Sink(source::Data.Source, db::DB, tablename)`
69105

70-
This method automatically takes care of SQLite transaction handling and other performance enhancements.
106+
Create a new SQLite table as a sink in `db` according to the `Data.schema(source)`.
71107

72-
* `append(db::SQLiteDB,name::String,table::AbstractMatrix)`
108+
* `Data.stream!(dt::Data.Table,sink::SQLite.Sink)`
73109

74-
Takes the values in `table` and appends (by repeated inserts) to the SQLite table `name`. No column checking is done to ensure correct types, so care should be taken as SQLite is "typeless" in that it allows items of any type to be stored in columns. Transaction handling is automatic as well as performance enhancements.
110+
stream the data from a `Data.Table` to `sink`
75111

76-
* `droptable(db::SQLiteDB,table::String)`
112+
* `Data.stream!(source::CSV.Source,sink::SQLite.Sink)`
77113

78-
`droptable` is pretty self-explanatory. It's really just a convenience wrapper around `query` to execute a DROP TABLE command, while also calling "VACUUM" to clean out freed memory from the database.
114+
stream the data from a `CSV.Source` to `sink`
79115

80-
* `register(db::SQLiteDB, func::Function; nargs::Int=-1, name::AbstractString=string(func), isdeterm::Bool=true)`
81-
* `register(db::SQLiteDB, init, step::Function, final::Function=identity; nargs::Int=-1, name::AbstractString=string(final), isdeterm::Bool=true)`
116+
* `SQLite.register(db::SQLite.DB, func::Function; nargs::Int=-1, name::AbstractString=string(func), isdeterm::Bool=true)`
117+
* `SQLite.register(db::SQLite.DB, init, step::Function, final::Function=identity; nargs::Int=-1, name::AbstractString=string(final), isdeterm::Bool=true)`
82118

83-
Register a scalar (first method) or aggregate (second method) function with a `SQLiteDB`.
119+
Register a scalar (first method) or aggregate (second method) function with a `SQLite.DB`.
84120

85121
* `@register db function`
86122

87-
Automatically define then register `function` with a `SQLiteDB`.
123+
Automatically define then register `function` with a `SQLite.DB`.
88124

89125
* `sr"..."`
90126

91127
This string literal is used to escape all special characters in the string, useful for using regex in a query.
92128

93-
* `sqlreturn(contex, val)`
129+
* `SQLite.sqlreturn(contex, val)`
94130

95131
This function should never be called explicitly. Instead it is exported so that it can be overloaded when necessary, see below.
96132

@@ -103,19 +139,19 @@ SQLite provides syntax for calling the [`regexp` function](http://sqlite.org/lan
103139
```julia
104140
julia> using SQLite
105141

106-
julia> db = SQLiteDB("Chinook_Sqlite.sqlite")
142+
julia> db = SQLite.DB("Chinook_Sqlite.sqlite")
107143

108144
julia> # using SQLite's in-built syntax
109145

110-
julia> query(db, "SELECT FirstName, LastName FROM Employee WHERE LastName REGEXP 'e(?=a)'")
146+
julia> SQLite.query(db, "SELECT FirstName, LastName FROM Employee WHERE LastName REGEXP 'e(?=a)'")
111147
1x2 ResultSet
112148
| Row | "FirstName" | "LastName" |
113149
|-----|-------------|------------|
114150
| 1 | "Jane" | "Peacock" |
115151

116152
julia> # explicitly calling the regexp() function
117153

118-
julia> query(db, "SELECT * FROM Genre WHERE regexp('e[trs]', Name)")
154+
julia> SQLite.query(db, "SELECT * FROM Genre WHERE regexp('e[trs]', Name)")
119155
6x2 ResultSet
120156
| Row | "GenreId" | "Name" |
121157
|-----|-----------|----------------------|
@@ -128,37 +164,37 @@ julia> query(db, "SELECT * FROM Genre WHERE regexp('e[trs]', Name)")
128164

129165
julia> # you can even do strange things like this if you really want
130166

131-
julia> query(db, "SELECT * FROM Genre ORDER BY GenreId LIMIT 2")
167+
julia> SQLite.query(db, "SELECT * FROM Genre ORDER BY GenreId LIMIT 2")
132168
2x2 ResultSet
133169
| Row | "GenreId" | "Name" |
134170
|-----|-----------|--------|
135171
| 1 | 1 | "Rock" |
136172
| 2 | 2 | "Jazz" |
137173

138-
julia> query(db, "INSERT INTO Genre VALUES (regexp('^word', 'this is a string'), 'My Genre')")
174+
julia> SQLite.query(db, "INSERT INTO Genre VALUES (regexp('^word', 'this is a string'), 'My Genre')")
139175
1x1 ResultSet
140176
| Row | "Rows Affected" |
141177
|-----|-----------------|
142178
| 1 | 0 |
143179

144-
julia> query(db, "SELECT * FROM Genre ORDER BY GenreId LIMIT 2")
180+
julia> SQLite.query(db, "SELECT * FROM Genre ORDER BY GenreId LIMIT 2")
145181
2x2 ResultSet
146182
| Row | "GenreId" | "Name" |
147183
|-----|-----------|------------|
148184
| 1 | 0 | "My Genre" |
149185
| 2 | 1 | "Rock" |
150186
```
151187

152-
Due to the heavy use of escape characters you may run into problems where julia parses out some backslashes in your query, for example `"\y"` simlpy becomes `"y"`. For example the following two queries are identical
188+
Due to the heavy use of escape characters you may run into problems where julia parses out some backslashes in your query, for example `"\y"` simply becomes `"y"`. For example the following two queries are identical
153189

154190
```julia
155-
julia> query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-\d'")
191+
julia> SQLite.query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-\d'")
156192
1x1 ResultSet
157193
| Row | "Rows Affected" |
158194
|-----|-----------------|
159195
| 1 | 0 |
160196

161-
julia> query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-d'")
197+
julia> SQLite.query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-d'")
162198
1x1 ResultSet
163199
| Row | "Rows Affected" |
164200
|-----|-----------------|
@@ -170,15 +206,15 @@ This can be avoided in two ways. You can either escape each backslash yourself o
170206
```julia
171207
julia> # manually escaping backslashes
172208

173-
julia> query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-\\d'")
209+
julia> SQLite.query(db, "SELECT * FROM MediaType WHERE Name REGEXP '-\\d'")
174210
1x2 ResultSet
175211
| Row | "MediaTypeId" | "Name" |
176212
|-----|---------------|-------------------------------|
177213
| 1 | 3 | "Protected MPEG-4 video file" |
178214

179215
julia> # using sr"..."
180216

181-
julia> query(db, sr"SELECT * FROM MediaType WHERE Name REGEXP '-\d'")
217+
julia> SQLite.query(db, sr"SELECT * FROM MediaType WHERE Name REGEXP '-\d'")
182218
1x2 ResultSet
183219
| Row | "MediaTypeId" | "Name" |
184220
|-----|---------------|-------------------------------|
@@ -191,7 +227,7 @@ The sr"..." currently escapes all special characters in a string but it may be c
191227

192228
SQLite.jl also provides a way that you can implement your own [Scalar Functions](https://www.sqlite.org/lang_corefunc.html). This is done using the `register` function and macro.
193229

194-
`@register` takes a `SQLiteDB` and a function. The function can be in block syntax
230+
`@register` takes a `SQLite.DB` and a function. The function can be in block syntax
195231

196232
```julia
197233
julia> @register db function add3(x)
@@ -211,9 +247,9 @@ and previously defined functions
211247
julia> @register db sin
212248
```
213249

214-
The `register` function takes optional arguments; `nargs` which defaults to `-1`, `name` which defaults to the name of the function, `isdeterm` which defaults to `true`. In practice these rarely need to be used.
250+
The `SQLite.register` function takes optional arguments; `nargs` which defaults to `-1`, `name` which defaults to the name of the function, `isdeterm` which defaults to `true`. In practice these rarely need to be used.
215251

216-
The `register` function uses the `sqlreturn` function to return your function's return value to SQLite. By default, `sqlreturn` maps the returned value to a [native SQLite type](http://sqlite.org/c3ref/result_blob.html) or, failing that, serializes the julia value and stores it as a `BLOB`. To change this behaviour simply define a new method for `sqlreturn` which then calls a previously defined method for `sqlreturn`. Methods which map to native SQLite types are
252+
The `SQLite.register` function uses the `sqlreturn` function to return your function's return value to SQLite. By default, `sqlreturn` maps the returned value to a [native SQLite type](http://sqlite.org/c3ref/result_blob.html) or, failing that, serializes the julia value and stores it as a `BLOB`. To change this behaviour simply define a new method for `sqlreturn` which then calls a previously defined method for `sqlreturn`. Methods which map to native SQLite types are
217253

218254
```julia
219255
sqlreturn(context, ::NullType)
@@ -241,20 +277,20 @@ Any new method defined for `sqlreturn` must take two arguments and must pass the
241277

242278
#### Custom Aggregate Functions
243279

244-
Using the `register` function, you can also define your own aggregate functions with largely the same semantics.
280+
Using the `SQLite.register` function, you can also define your own aggregate functions with largely the same semantics.
245281

246-
The `register` function for aggregates must take a `SQLiteDB`, an initial value, a step function and a final function. The first argument to the step function will be the return value of the previous function (or the initial value if it is the first iteration). The final function must take a single argument which will be the return value of the last step function.
282+
The `SQLite.register` function for aggregates must take a `SQLite.DB`, an initial value, a step function and a final function. The first argument to the step function will be the return value of the previous function (or the initial value if it is the first iteration). The final function must take a single argument which will be the return value of the last step function.
247283

248284
```julia
249285
julia> dsum(prev, cur) = prev + cur
250286

251287
julia> dsum(prev) = 2 * prev
252288

253-
julia> register(db, 0, dsum, dsum)
289+
julia> SQLite.register(db, 0, dsum, dsum)
254290
```
255291

256292
If no name is given the name of the first (step) function is used (in this case "dsum"). You can also use lambdas, the following does the same as the previous code snippet
257293

258294
```julia
259-
julia> register(db, 0, (p,c) -> p+c, p -> 2p, name="dsum")
295+
julia> SQLite.register(db, 0, (p,c) -> p+c, p -> 2p, name="dsum")
260296
```

0 commit comments

Comments
 (0)