|
1 | 1 | # DBFTables |
2 | 2 |
|
3 | 3 | [](https://travis-ci.org/JuliaData/DBFTables.jl) |
4 | | - |
5 | 4 | [](https://coveralls.io/github/JuliaData/DBFTables.jl?branch=master) |
6 | | - |
7 | 5 | [](http://codecov.io/github/JuliaData/DBFTables.jl?branch=master) |
8 | 6 |
|
9 | | -For reading [.dbf](https://en.wikipedia.org/wiki/.dbf) files in Julia. |
| 7 | +Read xBase / dBASE III+ [.dbf](https://en.wikipedia.org/wiki/.dbf) files in Julia. Supports the [Tables.jl](https://github.com/JuliaData/Tables.jl) interface. |
| 8 | + |
| 9 | +[Shapefile.jl](https://github.com/JuliaGeo/Shapefile.jl) uses this package to read the information associated to the geometries of the `.shp` file. |
10 | 10 |
|
11 | | -#### Usage |
| 11 | +## Usage |
12 | 12 |
|
13 | 13 | ```julia |
14 | 14 | using DBFTables |
15 | | -io = open("test.dbf") |
16 | | -df = DBFTables.read_dbf(io) |
17 | | -``` |
| 15 | +dbf = DBFTables.Table("test.dbf") |
| 16 | + |
| 17 | +# whole columns can be retrieved by their name |
| 18 | +# note that this creates a copy, so instead of repeated `dbf.field` calls, |
| 19 | +# it is faster to once do `field = dbf.field` and then use `field` instead |
| 20 | +dbf.INTEGER # => Union{Missing, Int64}[100, 101, 102, 0, 2222222222, 4444444444, missing] |
| 21 | + |
| 22 | +# example function that iterates over the rows and uses two columns |
| 23 | +function sumif(dbf) |
| 24 | + total = 0.0 |
| 25 | + for row in dbf |
| 26 | + if row.BOOLEAN && !ismissing(row.NUMERIC) |
| 27 | + value += row.NUMERIC |
| 28 | + end |
| 29 | + end |
| 30 | + return total |
| 31 | +end |
| 32 | + |
| 33 | +# for other functionality, convert to other Tables such as DataFrame |
| 34 | +using DataFrames |
| 35 | +df = DataFrame(dbf) |
| 36 | +``` |
| 37 | + |
| 38 | +## Format description resources |
| 39 | +- https://en.wikipedia.org/wiki/.dbf |
| 40 | +- https://www.clicketyclick.dk/databases/xbase/format/dbf.html |
| 41 | +- http://www.independent-software.com/dbase-dbf-dbt-file-format.html |
| 42 | + |
| 43 | +## Implementation details |
| 44 | + |
| 45 | +The DBF header contains information on the amount of rows, which columns are present, what type they are, and how many bytes the entries are. Based on this we can create a `Tables.Schema`. Each row is a fixed amount of bytes. All data is represented as strings, with different conventions based on the specified type. There are no delimiters between the entries, but since we know the sizes from the header, it is not needed. |
| 46 | + |
| 47 | +The `DBFTables.Table` struct holds both the header and data. All data is read into memory in one go as a `Vector{UInt8}`. To provide efficient access into the individual entries, we use [WeakRefStrings](https://github.com/JuliaData/WeakRefStrings.jl/). WeakRefStrings' `StringArray` only holds the offsets and lengths into the `Vector{UInt8}` with all the data. Then we still need to convert from the string to the julia type. This is done on demand with `dbf_value`. |
| 48 | + |
| 49 | +Note that the format also contains a "record deleted" flag, which is represented by a `'*'` at the start of the row. When this is encountered the record should be treated as if it doesn't exist. Since normally writers strip these records when writing, they are rarely encountered. For that reason this package ignores these flags by default right now. To check for the flags yourself, there is the `isdeleted` function. A sample file with deleted record flags is available [here](https://issues.qgis.org/issues/11007#note-30). |
0 commit comments