Skip to content

Commit d058b5f

Browse files
authored
Merge pull request #26 from ws-garcia/CSV-Interfacev4.1.0
Csv interfacev4.1.0 * __Improvements__: integration with [`VBAexpressions`](https://github.com/ws-garcia/VBA-Expressions) core modules.
2 parents f9b0b10 + 8629751 commit d058b5f

22 files changed

+3891
-742
lines changed

README.md

Lines changed: 13 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ The most powerful and comprehensive CSV/[TSV](https://www.iana.org/assignments/m
1818
* __Automatic delimiter sniffer__. Don't worry if you forgot the file configuration. The interface has a solid strategy to sniff delimiters!
1919
* __Highly Configurable__. User can configure the parser to work with a wide range of CSV files.
2020
* __CSV data subsetting__. Split CSV data into a set of files with related data.
21-
* __Like SQL queries on CSV files__. Add your own logic to mimic SQL queries and filter data by criteria (=, <>, >=, <=, AND, OR).
21+
* __Like SQL queries on CSV files__. Use complex patterns to mimic SQL queries and filter data by criteria (=, <>, >=, <=, & (AND), |(OR)).
2222
* __Flexible__. Import only certain range of records from the given file, import fields (columns) by indexes or names, read records in sequential mode.
2323
* __Dynamic Typing support__. Turn CSV data field to a desired VBA data type.
2424
* __Multi-level data sorting__. Sort CSV imported data over multiple columns using the hyper-fast(100k records per second) [Yaroslavskiy Dual-Pivot Quicksort](https://web.archive.org/web/20151002230717/http://iaroslavski.narod.ru/quicksort/DualPivotQuicksort.pdf) like Java and also other methods like: IntroSort, HeapSort and Merge sort.
@@ -283,26 +283,20 @@ End Sub
283283
So far, it has been outlined the way in which you can import the records from a CSV file sequentially, the following example shows how to filter the records, in a like SQL way, according to whether they meet a criterion set by the user:
284284

285285
```
286-
Sub QueryCSV(path As String, ByVal keyIndex As Long, queryFilters As Variant)
286+
Sub QueryCSV()
287287
Dim CSVint As CSVinterface
288-
Dim CSVrecords As ECPArrayList
288+
Dim path As String
289+
Dim FilteredData As CSVArrayList
289290
290291
Set CSVint = New CSVinterface
291-
With CSVint.parseConfig
292-
.path = "C:\Sample.csv"
293-
.dialect.fieldsDelimiter = "," ' Columns delimiter
294-
.dialect.recordsDelimiter = vbCrLf ' Rows delimiter
295-
End With
292+
path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
293+
CSVint.parseConfig.Headers = False 'The file has no header record/row
294+
CSVint.parseConfig.path = path
296295
If path <> vbNullString Then
297-
'@-----------------------------------------------
298-
' The following instruction will filter the data
299-
' on the keyIndex(th) field.
300-
Set CSVrecords = CSVint.GetCSVsubset(path, _
301-
queryFilters, _
302-
keyIndex)
303-
CSVint.DumpToSheet DataSource:=CSVrecords ' Dump result to new WorkSheet
296+
Set FilteredData = CSVint.Filter("f1='Asia' & f9>20 & f9<=50", path) 'Select "Units sold" greater than 20 and less or
297+
'equal to 50 from Asian customers
304298
Set CSVint = Nothing
305-
Set CSVrecords = Nothing
299+
Set FilteredData = Nothing
306300
End If
307301
End Sub
308302
```
@@ -383,11 +377,14 @@ The benchmark results for VBA-CSV Interface are available at [this site](https:/
383377

384378
The library is composed of the following class modules:
385379
* `CSVArrayList`
380+
* [`CSVcallBack`](https://github.com/ws-garcia/VBA-Expressions)
386381
* `CSVdialect`
382+
* [`CSVexpressions`](https://github.com/ws-garcia/VBA-Expressions)
387383
* `CSVinterface`
388384
* `CSVparserConfig`
389385
* `CSVSniffer`
390386
* `CSVTextStream`
387+
* [`CSVudFunctions`](https://github.com/ws-garcia/VBA-Expressions)
391388

392389
All dependencies are written in pure VBA.
393390

docs/api/csvarraylist.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,11 @@ Class module developed to emulate some functionalities from the `ArrayList` pres
5252
<td style="text-align: left;">Concatenates the values from the current instance with the specified values and returns a <code>CSVArraylist</code> object as result. The <code>AValues</code> parameter is a <code>Variant</code> data type containing the array, <code>CSVArraylist</code> or value to concatenate.</td>
5353
</tr>
5454
<tr>
55+
<td style="text-align: left; color:blue;"><em>Concat2</em></td>
56+
<td style="text-align: left;">Method</td>
57+
<td style="text-align: left;">Concatenates the values from the current instance with the specified values and returns a <code>CSVArraylist</code> object as result. The <code>AValues</code> parameter is a <code>CSVArraylist</code> with the values to concatenate.</td>
58+
</tr>
59+
<tr>
5560
<td style="text-align: left; color:blue;"><em>Copy</em></td>
5661
<td style="text-align: left;">Method</td>
5762
<td style="text-align: left;">Returns a <code>CSVArraylist</code> object with a copy of the current instance from and to a given index. The <code>StartIndex</code> parameter indicates where the copy will start and the <code>EndIndex</code> determines where the operation will end. If the <code>EndIndex</code> parameter is set to <code>-1</code>, the operation will end at the maximum index available for the current instance.</td>
@@ -72,6 +77,11 @@ Class module developed to emulate some functionalities from the `ArrayList` pres
7277
<td style="text-align: left;">Creates an empty jagged array. The operation will turns the array <code>ArrVar</code> into an jagged array with <code>ArraySize + 1</code> rows and each row with <code>VectorSize</code> columns. To access to an individual element user must use something like <code>expression(i)(j)</code>, where <code>i</code> denotes an index in the main array and <code>j</code> denotes an index in the child array.</td>
7378
</tr>
7479
<tr>
80+
<td style="text-align: left; color:blue;"><em>Filter</em></td>
81+
<td style="text-align: left;">Method</td>
82+
<td style="text-align: left;">Returns a filtered array list using the <code>CSVexpressions</code> class module.</td>
83+
</tr>
84+
<tr>
7585
<td style="text-align: left; color:blue;"><em>Insert</em></td>
7686
<td style="text-align: left;">Method</td>
7787
<td style="text-align: left;">Inserts an Item, at the given Index, in the current instance of the class.</td>

docs/api/errors/index.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,6 @@ This section describes the custom errors whose components are returned by the pa
5555
<td style="text-align: left;">CSVinterface Class.</td>
5656
</tr>
5757
<tr>
58-
<td style="text-align: left;"><em>-2147212490</em></td>
59-
<td style="text-align: left;">[CSV file subset]: The given path name is an empty string or the specified CSV file does not exist in the supplied path.</td>
60-
<td style="text-align: left;">CSVinterface Class.</td>
61-
</tr>
62-
<tr>
6358
<td style="text-align: left;"><em>-2147212489</em></td>
6459
<td style="text-align: left;">[CSV Field Insert]: Cannot insert a field in the current instance. This is because there is no imported data or the records do not have the same number of fields.</td>
6560
<td style="text-align: left;">CSVinterface Class.</td>

docs/api/methods/filter.md

Lines changed: 26 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,14 @@ nav_order: 10
1111
New
1212
{: .label .label-purple }
1313

14-
Returns a list of records as a result of applying filters on the imported CSV data.
14+
Returns a list of records as a result of applying filters on the target CSV file or imported CSV data using expression evaluation.
1515
{: .fs-4 .fw-300 }
1616

1717
---
1818

1919
## Syntax
2020

21-
*expression*.`Filter`*(fieldIndex, patterns)*
21+
*expression*.`Filter`*(Pattern, [FilePath], [ExcludeFirstRecord])*
2222

2323
### Parameters
2424

@@ -31,12 +31,16 @@ Returns a list of records as a result of applying filters on the imported CSV da
3131
</thead>
3232
<tbody>
3333
<tr>
34-
<td style="text-align: left;"><em>fieldIndex</em></td>
35-
<td style="text-align: left;">Required. Identifier specifying a <code>Long</code> Type variable. Represents the index of the field used for data filtering.</td>
34+
<td style="text-align: left;"><em>Pattern</em></td>
35+
<td style="text-align: left;">Required. Identifier specifying a <code>String</code> Type variable. Represents a valid string expression to evaluate when filtering records</td>
3636
</tr>
3737
<tr>
38-
<td style="text-align: left;"><em>patterns</em></td>
39-
<td style="text-align: left;">Optional. Identifier specifying a list of <code>Strings</code> Type variables.</td>
38+
<td style="text-align: left;"><em>FilePath</em></td>
39+
<td style="text-align: left;">Optional. Identifier specifying a <code>String</code> Type variable. Represents the full file path, including file extension, of the CSV file used for data filtering.</td>
40+
</tr>
41+
<tr>
42+
<td style="text-align: left;"><em>ExcludeFirstRecord</em></td>
43+
<td style="text-align: left;">Optional. Identifier specifying a <code>Boolean</code> Type variable. When <code>True</code>, the file headers will be excluded.</td>
4044
</tr>
4145
</tbody>
4246
</table>
@@ -52,42 +56,31 @@ See also
5256

5357
## Behavior
5458

55-
If the `patterns` parameter is omitted the complete set of stored data will be returned. The rules that apply to the `patterns` parameter are listed below:
56-
* The comparison is influenced by the `Option Compare` statement (one of: `Option Compare Binary` or `Option Compare Text`). The binary compare is case sensitive, the text compare is not.
57-
* The following table describes the special characters to be used when creating patterns; all other characters match themselves:
58-
*
59-
|Character|Meaning|
60-
|:------:|:-----|
61-
|?|Any single character|
62-
|\*|Zero or more characters|
63-
|#|Any single digit (0-9)|
64-
|\[list\]|Any single character in list|
65-
|\[!list\]|Any single character not in list|
66-
|\[\]|A zero-length string ("")|
67-
* 'list' matches a group of characters in `patterns` to a single character in the string and can contain almost all available characters, including digits.
68-
* Use a hyphen (-) in 'list' to create a range of characters that matches a character in the string: e.g. [A-D] matches A,B,C, or D at that character position in the string. Multiple ranges of characters can be included in 'list' without the use of a delimiter: e.g. \[A-DJ-L\].
69-
* Use the hyphen at the start or end of 'list' to match to itself. For example, \[-A-G\] matches a hyphen or any character from A to G.
70-
* The exclamation mark in the "pattern" match is similar to the negation operator. For example, [!A-G] matches all characters except characters A through G.
71-
* The exclamation mark outside the bracket matches itself.
72-
* To use any special character as a matching character, enclose the special character in brackets. For example, to match a question mark, use \[?\].
59+
The `Pattern` parameter is evaluated according to the number of records in the CSV file, when the evaluation returns `True`, the current record is saved. The rules that apply to the `Pattern` parameter are listed below:
60+
* To reference a field value, the user must type something like `f#` where `f` is a required identifier and `#` is the numeric position of the desired field. For example, `f1>5` indicates the selection of records whose first field value is greater than `5`.
61+
* If the user needs to compare literal strings, the values must be enclosed in apostrophes. Example, `Region = 'Central America'` is a valid string assigned to the variable `Region`.
62+
* User can use functions in the `Pattern` definition, including custom UDFs (refer to [VBAexpressions documentation](https://github.com/ws-garcia/VBA-Expressions)). I.e.: `min(f5;f2)>=100`
63+
64+
When the `FilePath` argument is omitted, the method will proceed to filter the data stored in the current instance, otherwise it will filter the content of the CSV file specified with the referred argument.
7365

7466
### ☕Example
7567

7668
```vb
7769
Sub FilterCSV()
7870
Dim CSVint As CSVinterface
71+
Dim path As String
7972
Dim FilteredData As CSVArrayList
8073

8174
Set CSVint = New CSVinterface
82-
With CSVint.parseConfig
83-
.path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
84-
End With
85-
With CSVint
86-
.ImportFromCSV .parseConfig
87-
Set FilteredData = .Filter(11, "###.##", "####.##") 'Filter data between hundreds and thousands
88-
End With
89-
Set CSVint = Nothing
90-
Set FilteredData = Nothing
75+
path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
76+
CSVint.parseConfig.Headers = False 'The file has no header record/row
77+
CSVint.parseConfig.path = path
78+
If path <> vbNullString Then
79+
Set FilteredData = CSVint.Filter("f1='Asia' & f9>20 & f9<=50", path) 'Select "Units sold" greater than 20 and less or
80+
'equal to 50 from Asian customers
81+
Set CSVint = Nothing
82+
Set FilteredData = Nothing
83+
End If
9184
End Sub
9285
```
9386

docs/api/methods/filter2.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
title: Filter2
3+
parent: Methods
4+
grand_parent: API
5+
nav_order: 11
6+
---
7+
8+
# Filter2
9+
10+
Returns a list of records as a result of applying filters on the imported CSV data.
11+
{: .fs-4 .fw-300 }
12+
13+
---
14+
15+
## Syntax
16+
17+
*expression*.`Filter2`*(fieldIndex, patterns)*
18+
19+
### Parameters
20+
21+
<table>
22+
<thead>
23+
<tr>
24+
<th style="text-align: left;">Part</th>
25+
<th style="text-align: left;">Description</th>
26+
</tr>
27+
</thead>
28+
<tbody>
29+
<tr>
30+
<td style="text-align: left;"><em>fieldIndex</em></td>
31+
<td style="text-align: left;">Required. Identifier specifying a <code>Long</code> Type variable. Represents the index of the field used for data filtering.</td>
32+
</tr>
33+
<tr>
34+
<td style="text-align: left;"><em>patterns</em></td>
35+
<td style="text-align: left;">Optional. Identifier specifying a list of <code>Strings</code> Type variables.</td>
36+
</tr>
37+
</tbody>
38+
</table>
39+
40+
### Returns value
41+
42+
*Type*: `CSVArrayList`
43+
44+
---
45+
46+
See also
47+
: [ImportFromCSV method](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/importfromcsv.html), [ImportFromCSVstring method](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/importfromcsvstring.html), [CSVArrayList class](https://ws-garcia.github.io/VBA-CSV-interface/api/csvarraylist.html).
48+
49+
## Behavior
50+
51+
If the `patterns` parameter is omitted the complete set of stored data will be returned. The rules that apply to the `patterns` parameter are listed below:
52+
* The comparison is influenced by the `Option Compare` statement (one of: `Option Compare Binary` or `Option Compare Text`). The binary compare is case sensitive, the text compare is not.
53+
* The following table describes the special characters to be used when creating patterns; all other characters match themselves:
54+
*
55+
|Character|Meaning|
56+
|:------:|:-----|
57+
|?|Any single character|
58+
|\*|Zero or more characters|
59+
|#|Any single digit (0-9)|
60+
|\[list\]|Any single character in list|
61+
|\[!list\]|Any single character not in list|
62+
|\[\]|A zero-length string ("")|
63+
* 'list' matches a group of characters in `patterns` to a single character in the string and can contain almost all available characters, including digits.
64+
* Use a hyphen (-) in 'list' to create a range of characters that matches a character in the string: e.g. [A-D] matches A,B,C, or D at that character position in the string. Multiple ranges of characters can be included in 'list' without the use of a delimiter: e.g. \[A-DJ-L\].
65+
* Use the hyphen at the start or end of 'list' to match to itself. For example, \[-A-G\] matches a hyphen or any character from A to G.
66+
* The exclamation mark in the "pattern" match is similar to the negation operator. For example, [!A-G] matches all characters except characters A through G.
67+
* The exclamation mark outside the bracket matches itself.
68+
* To use any special character as a matching character, enclose the special character in brackets. For example, to match a question mark, use \[?\].
69+
70+
### ☕Example
71+
72+
```vb
73+
Sub FilterCSV()
74+
Dim CSVint As CSVinterface
75+
Dim FilteredData As CSVArrayList
76+
77+
Set CSVint = New CSVinterface
78+
With CSVint.parseConfig
79+
.path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
80+
End With
81+
With CSVint
82+
.ImportFromCSV .parseConfig
83+
Set FilteredData = .Filter2(11, "###.##", "####.##") 'Filter data between hundreds and thousands
84+
End With
85+
Set CSVint = Nothing
86+
Set FilteredData = Nothing
87+
End Sub
88+
```
89+
90+
[Back to Methods overview](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/)

docs/api/methods/getcsvsubset.md

Lines changed: 0 additions & 92 deletions
This file was deleted.

docs/examples/advanced-examples.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,14 @@ The \[EXAMPLE1\] shows how you can execute a like SQL simple query over a CSV fi
99

1010
#### [EXAMPLE1]
1111
```vb
12-
Private Sub Query_CSV(path As String, ByVal keyIndex As Long, queryFilters As Variant)
12+
Private Sub Query_CSV(path As String)
1313
Dim CSVint As CSVinterface
1414
Dim CSVrecords As CSVArrayList
1515

1616
Set CSVint = New CSVinterface
1717
If path <> vbNullString Then
18-
Set CSVrecords = CSVint.GetCSVsubset(path, queryFilters, keyIndex) 'data filtered on keyIndex th field
18+
Set CSVrecords = CSVint.Filter("f1='Asia' & f9>20 & f9<=50", path) 'Select "Units sold" greater than 20 and less or
19+
'equal to 50 from Asian customers
1920
CSVint.DumpToSheet DataSource:=CSVrecords 'dump result
2021
Set CSVint = Nothing
2122
Set CSVrecords = Nothing

0 commit comments

Comments
 (0)