Improvements: integration with VBAexpressions core modules.

ws-garcia · ws-garcia · commit 056ef69b7897 · 2022-02-27T16:32:08.000-04:00
diff --git a/README.md b/README.md
@@ -18,7 +18,7 @@ The most powerful and comprehensive CSV/[TSV](https://www.iana.org/assignments/m
 * __Automatic delimiter sniffer__. Don't worry if you forgot the file configuration. The interface has a solid strategy to sniff delimiters!
 * __Highly Configurable__. User can configure the parser to work with a wide range of CSV files.
 * __CSV data subsetting__. Split CSV data into a set of files with related data.
-* __Like SQL queries on CSV files__. Add your own logic to mimic SQL queries and filter data by criteria (=, <>, >=, <=, AND, OR).
+* __Like SQL queries on CSV files__. Use complex patterns to mimic SQL queries and filter data by criteria (=, <>, >=, <=, & (AND), |(OR)).
 * __Flexible__. Import only certain range of records from the given file, import fields (columns) by indexes or names, read records in sequential mode. 
 * __Dynamic Typing support__. Turn CSV data field to a desired VBA data type.
 * __Multi-level data sorting__. Sort CSV imported data over multiple columns using the hyper-fast(100k records per second) [Yaroslavskiy Dual-Pivot Quicksort](https://web.archive.org/web/20151002230717/http://iaroslavski.narod.ru/quicksort/DualPivotQuicksort.pdf) like Java and also other methods like: IntroSort, HeapSort and Merge sort.
@@ -283,26 +283,21 @@ End Sub
 So far, it has been outlined the way in which you can import the records from a CSV file sequentially, the following example shows how to filter the records, in a like SQL way, according to whether they meet a criterion set by the user:
 
 ```
-Sub QueryCSV(path As String, ByVal keyIndex As Long, queryFilters As Variant)
-    Dim CSVint As CSVinterface
-    Dim CSVrecords As ECPArrayList
+Sub QueryCSV()
+    Dim CSVint As CSVinterfac
+    Dim path As String
+    Dim FilteredData As CSVArrayList
     
     Set CSVint = New CSVinterface
-    With CSVint.parseConfig
-        .path = "C:\Sample.csv"
-        .dialect.fieldsDelimiter = ","                      ' Columns delimiter
-        .dialect.recordsDelimiter = vbCrLf                  ' Rows delimiter
-    End With
+    path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
+    CSVint.parseConfig.Headers = False                                      		'The file has no header record/row
+    CSVint.parseConfig.path = path
     If path <> vbNullString Then
-        '@-----------------------------------------------
-        ' The following instruction will filter the data
-        ' on the keyIndex(th) field.
-        Set CSVrecords = CSVint.GetCSVsubset(path, _
-                                            queryFilters, _
-                                            keyIndex)
-        CSVint.DumpToSheet DataSource:=CSVrecords           ' Dump result to new WorkSheet
+        Set CSVrecords = CSVint.ImportFromCSV(CSVint.parseConfig).items
+        Set FilteredData = CSVint.Filter(path, "f1='Asia' & f9>20 & f9<=50") 		'Select "Units sold" greater than 20 and less or 
+																											'equal to 50 from Asian customers
         Set CSVint = Nothing
-        Set CSVrecords = Nothing
+        Set FilteredData = Nothing
     End If
 End Sub
 ```
@@ -383,11 +378,14 @@ The benchmark results for VBA-CSV Interface are available at [this site](https:/
 
 The library is composed of the following class modules:
 * `CSVArrayList`
+* [`CSVcallBack`](https://github.com/ws-garcia/VBA-Expressions)
 * `CSVdialect`
+* [`CSVexpressions`](https://github.com/ws-garcia/VBA-Expressions)
 * `CSVinterface`
 * `CSVparserConfig`
 * `CSVSniffer`
 * `CSVTextStream`
+* [`CSVudFunctions`](https://github.com/ws-garcia/VBA-Expressions)
 
 All dependencies are written in pure VBA.
 
diff --git a/docs/api/errors/index.md b/docs/api/errors/index.md
@@ -55,11 +55,6 @@ This section describes the custom errors whose components are returned by the pa
 <td style="text-align: left;">CSVinterface Class.</td>
 </tr>
 <tr>
-<td style="text-align: left;"><em>-2147212490</em></td>
-<td style="text-align: left;">[CSV file subset]: The given path name is an empty string or the specified CSV file does not exist in the supplied path.</td>
-<td style="text-align: left;">CSVinterface Class.</td>
-</tr>
-<tr>
 <td style="text-align: left;"><em>-2147212489</em></td>
 <td style="text-align: left;">[CSV Field Insert]: Cannot insert a field in the current instance. This is because there is no imported data or the records do not have the same number of fields.</td>
 <td style="text-align: left;">CSVinterface Class.</td>
diff --git a/docs/api/methods/filter.md b/docs/api/methods/filter.md
@@ -11,14 +11,14 @@ nav_order: 10
 New
 {: .label .label-purple }
 
-Returns a list of records as a result of applying filters on the imported CSV data.
+Returns a list of records as a result of applying filters on the target CSV file using expression evaluation.
 {: .fs-4 .fw-300 }
 
 ---
 
 ## Syntax
 
-*expression*.`Filter`*(fieldIndex, patterns)*
+*expression*.`Filter`*(FilePath, Pattern, ExcludeFirstRecord)*
 
 ### Parameters
 
@@ -31,12 +31,16 @@ Returns a list of records as a result of applying filters on the imported CSV da
 </thead>
 <tbody>
 <tr>
-<td style="text-align: left;"><em>fieldIndex</em></td>
-<td style="text-align: left;">Required. Identifier specifying a <code>Long</code> Type variable. Represents the index of the field used for data filtering.</td>
+<td style="text-align: left;"><em>FilePath</em></td>
+<td style="text-align: left;">Required. Identifier specifying a <code>String</code> Type variable. Represents the full file path, including file extension, of the CSV file used for data filtering.</td>
 </tr>
 <tr>
-<td style="text-align: left;"><em>patterns</em></td>
-<td style="text-align: left;">Optional. Identifier specifying a list of <code>Strings</code> Type variables.</td>
+<td style="text-align: left;"><em>Pattern</em></td>
+<td style="text-align: left;">Required. Identifier specifying a <code>String</code> Type variable. Represents a valid string expression to evaluate when filtering records</td>
+</tr>
+<tr>
+<td style="text-align: left;"><em>ExcludeFirstRecord</em></td>
+<td style="text-align: left;">Optional. Identifier specifying a <code>Boolean</code> Type variable. When <code>True</code>, the file headers will be excluded.</td>
 </tr>
 </tbody>
 </table>
@@ -52,42 +56,30 @@ See also
 
 ## Behavior
 
-If the `patterns` parameter is omitted the complete set of stored data will be returned. The rules that apply to the `patterns` parameter are listed below:
-* The comparison is influenced by the `Option Compare` statement (one of: `Option Compare Binary` or `Option Compare Text`). The binary compare is case sensitive, the text compare is not.
-* The following table describes the special characters to be used when creating patterns; all other characters match themselves:
-	* 
-	|Character|Meaning|
-	|:------:|:-----|
-	|?|Any single character|
-	|\*|Zero or more characters|
-	|#|Any single digit (0-9)|
-	|\[list\]|Any single character in list|
-	|\[!list\]|Any single character not in list|
-	|\[\]|A zero-length string ("")|
-* 'list' matches a group of characters in `patterns` to a single character in the string and can contain almost all available characters, including digits.
-* Use a hyphen (-) in 'list' to create a range of characters that matches a character in the string: e.g. [A-D] matches A,B,C, or D at that character position in the string. Multiple ranges of characters can be included in 'list' without the use of a delimiter: e.g. \[A-DJ-L\].
-* Use the hyphen at the start or end of 'list' to match to itself. For example, \[-A-G\] matches a hyphen or any character from A to G.
-* The exclamation mark in the "pattern" match is similar to the negation operator. For example, [!A-G] matches all characters except characters A through G.
-* The exclamation mark outside the bracket  matches itself.
-* To use any special character as a matching character, enclose the special character in brackets. For example, to match a question mark, use \[?\].
+The `Pattern` parameter is evaluated according to the number of records in the CSV file, when the evaluation returns `True`, the current record is saved. The rules that apply to the `Pattern` parameter are listed below:
+* To reference a field value, the user must type something like `f#` where `f` is a required identifier and `#` is the numeric position of the desired field. For example, `f1>5` indicates the selection of records whose first field value is greater than `5`.
+* If the user needs to compare literal strings, the values must be enclosed in apostrophes. Example, `Region = 'Central America'` is a valid string assigned to the variable `Region`.
+* User can use functions in the `Pattern` definition, including custom UDFs (refer to [VBAexpressions documentation](https://github.com/ws-garcia/VBA-Expressions)). I.e.: `min(f5;f2)>=100` 
 
 ### ☕Example
 
 ```vb
 Sub FilterCSV()
-    Dim CSVint As CSVinterface
+    Dim CSVint As CSVinterfac
+    Dim path As String
     Dim FilteredData As CSVArrayList
     
     Set CSVint = New CSVinterface
-    With CSVint.parseConfig
-        .path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
-    End With
-    With CSVint
-        .ImportFromCSV .parseConfig
-        Set FilteredData = .Filter(11, "###.##", "####.##")   'Filter data between hundreds and thousands
-    End With
-    Set CSVint = Nothing
-    Set FilteredData = Nothing
+    path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
+    CSVint.parseConfig.Headers = False                                      		'The file has no header record/row
+    CSVint.parseConfig.path = path
+    If path <> vbNullString Then
+        Set CSVrecords = CSVint.ImportFromCSV(CSVint.parseConfig).items
+        Set FilteredData = CSVint.Filter(path, "f1='Asia' & f9>20 & f9<=50") 		'Select "Units sold" greater than 20 and less or 
+																											'equal to 50 from Asian customers
+        Set CSVint = Nothing
+        Set FilteredData = Nothing
+    End If
 End Sub
 ```
 
diff --git a/docs/api/methods/filter2.md b/docs/api/methods/filter2.md
@@ -0,0 +1,90 @@
+---
+title: Filter2
+parent: Methods
+grand_parent: API
+nav_order: 11
+---
+
+# Filter2
+
+Returns a list of records as a result of applying filters on the imported CSV data.
+{: .fs-4 .fw-300 }
+
+---
+
+## Syntax
+
+*expression*.`Filter2`*(fieldIndex, patterns)*
+
+### Parameters
+
+<table>
+<thead>
+<tr>
+<th style="text-align: left;">Part</th>
+<th style="text-align: left;">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td style="text-align: left;"><em>fieldIndex</em></td>
+<td style="text-align: left;">Required. Identifier specifying a <code>Long</code> Type variable. Represents the index of the field used for data filtering.</td>
+</tr>
+<tr>
+<td style="text-align: left;"><em>patterns</em></td>
+<td style="text-align: left;">Optional. Identifier specifying a list of <code>Strings</code> Type variables.</td>
+</tr>
+</tbody>
+</table>
+
+### Returns value
+
+*Type*: `CSVArrayList`
+
+---
+
+See also
+: [ImportFromCSV method](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/importfromcsv.html), [ImportFromCSVstring method](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/importfromcsvstring.html), [CSVArrayList class](https://ws-garcia.github.io/VBA-CSV-interface/api/csvarraylist.html).
+
+## Behavior
+
+If the `patterns` parameter is omitted the complete set of stored data will be returned. The rules that apply to the `patterns` parameter are listed below:
+* The comparison is influenced by the `Option Compare` statement (one of: `Option Compare Binary` or `Option Compare Text`). The binary compare is case sensitive, the text compare is not.
+* The following table describes the special characters to be used when creating patterns; all other characters match themselves:
+	* 
+	|Character|Meaning|
+	|:------:|:-----|
+	|?|Any single character|
+	|\*|Zero or more characters|
+	|#|Any single digit (0-9)|
+	|\[list\]|Any single character in list|
+	|\[!list\]|Any single character not in list|
+	|\[\]|A zero-length string ("")|
+* 'list' matches a group of characters in `patterns` to a single character in the string and can contain almost all available characters, including digits.
+* Use a hyphen (-) in 'list' to create a range of characters that matches a character in the string: e.g. [A-D] matches A,B,C, or D at that character position in the string. Multiple ranges of characters can be included in 'list' without the use of a delimiter: e.g. \[A-DJ-L\].
+* Use the hyphen at the start or end of 'list' to match to itself. For example, \[-A-G\] matches a hyphen or any character from A to G.
+* The exclamation mark in the "pattern" match is similar to the negation operator. For example, [!A-G] matches all characters except characters A through G.
+* The exclamation mark outside the bracket  matches itself.
+* To use any special character as a matching character, enclose the special character in brackets. For example, to match a question mark, use \[?\].
+
+### ☕Example
+
+```vb
+Sub FilterCSV()
+    Dim CSVint As CSVinterface
+    Dim FilteredData As CSVArrayList
+    
+    Set CSVint = New CSVinterface
+    With CSVint.parseConfig
+        .path = Environ("USERPROFILE") & "\Desktop\Demo_100000records.csv"
+    End With
+    With CSVint
+        .ImportFromCSV .parseConfig
+        Set FilteredData = .Filter2(11, "###.##", "####.##")   'Filter data between hundreds and thousands
+    End With
+    Set CSVint = Nothing
+    Set FilteredData = Nothing
+End Sub
+```
+
+[Back to Methods overview](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/)
diff --git a/docs/api/methods/getcsvsubset.md b/docs/api/methods/getcsvsubset.md
diff --git a/docs/home/dependencies.md b/docs/home/dependencies.md
@@ -11,8 +11,11 @@ nav_order: 3
 The CSV interface library is composed of the following class modules:
 
 * `CSVArrayList`. Developed to emulate some ArrayList functionalities present in other languages and to optimize operations on imported data.
+* [`CSVcallBack`](https://github.com/ws-garcia/VBA-Expressions). Developed to offer a filtering path using Custom Functions.
 * `CSVdialect`. To share the file dialect (delimiters and escape behavior) in a compact way.
+* [`CSVexpressions`](https://github.com/ws-garcia/VBA-Expressions). The core filtering methods module, provides users with advanced filtering capabilities.
 * `CSVinterface`. The main module for dealing with CSV file operations. 
 * `CSVparserConfig`. To easily share all parser options between methods.
 * `CSVSniffer`. CSV dialect guessing helper.
-* `CSVTextStream`. To work with text files through streams, saving RAM memory.
+* `CSVTextStream`. To work with text files through streams, saving RAM memory.
+* [`CSVudFunctions`](https://github.com/ws-garcia/VBA-Expressions). A place where users can write custom functions for use at runtime.
diff --git a/docs/home/installation.md b/docs/home/installation.md
@@ -14,11 +14,14 @@ description: "Install instructions for the VBA CSV interface class."
 In order to use the latest release of the CSV interface library within your VBA code, you must to download the provides `.zip` archive. Once downloaded, import the following files from the VBA IDE:
 
 * `CSVArrayList`
+* [`CSVcallBack`](https://github.com/ws-garcia/VBA-Expressions)
 * `CSVdialect`
+* [`CSVexpressions`](https://github.com/ws-garcia/VBA-Expressions)
 * `CSVinterface`
 * `CSVparserConfig`
 * `CSVSniffer`
 * `CSVTextStream`
+* [`CSVudFunctions`](https://github.com/ws-garcia/VBA-Expressions)
 
 >⚠️**Caution**
 >{: .text-grey-lt-000 .bg-green-000 }
diff --git a/src/Access_version.zip b/src/Access_version.zip
diff --git a/src/CSVinterface.cls b/src/CSVinterface.cls
