Merge pull request #19 from ws-garcia/unix+standard-escape-mechanism

Unix+standard escape mechanism

Author: ws-garcia

README.md

@@ -6,9 +6,10 @@
 VBA CSV interface is the most complete, and open source, CSV/TSV VBA parser library nowadays. The library is RFC-4180 compliant and enables users to manipulate CSV content at the highest speed. All the modules were developed to accomplish the data exchange task with the greatest performance and to grant an easy use.
 
 ## Advantages
-* __Stable__. Fully Test Driven Developed (TDD) library, ([48/48 test passed](https://github.com/ws-garcia/VBA-CSV-interface/blob/master/testing/tests/results/)), that includes 500+ line of code for testing. See [VBA test library by Tim Hall](https://github.com/ws-garcia/vba-test).
+* __Stable__. Fully Test Driven Developed (TDD) library, ([49/49 test passed](https://github.com/ws-garcia/VBA-CSV-interface/blob/master/testing/tests/results/)), that includes 500+ line of code for testing. See [VBA test library by Tim Hall](https://github.com/ws-garcia/vba-test).
 * __Fast__. Writes and reads files at the highest speed.
 * __Memory-friendly__. CSV/[TSV](https://www.iana.org/assignments/media-types/text/tab-separated-values) files are processed using a custom stream technique, only 0.5MB are in memory at a time.
+* __Robust__. Parser and writer accept [Unix-style escape sequences](https://www.loc.gov/preservation/digital/formats/fdd/fdd000323.shtml#useful).
 * __Easy to use__. A few lines of code can do the work!
 * __Highly Configurable__. User can configure the parser to work with a wide range of CSV files.
 * __CSV data subsetting__. Split CSV data into a set of files with related data.

csv-data/assets.zip

@@ -17,9 +17,9 @@ Provides a list of constants to configure the char used as escape character.
 
 |**_Constant_**|**_Member name_**|
 |:----------|:----------|
-|0|*NullChar*|
 |1|*Apostrophe*|
 |2|*DoubleQuotes*|
+|3|*Tilde*|
 
 ---
 
@@ -29,9 +29,7 @@ Provides a list of constants to configure the char used as escape character.
 
 >📝**Note**
 >{: .text-grey-lt-000 .bg-green-000 }
->The `EscapeTokens.NullChar` value is used to indicates the CSV/TSV file does not use any escape char in its whole length. This value induces the program to write the file assuming the `parserConfig.fieldsDelimiter` property is enough for the export operation.
->
->In the case the `parserConfig.fieldsDelimiter` property is not enough for successfully done the export operation, the `EscapeTokens.DoubleQuotes` value would be used for parse/write an CSV/TSV having fields to be escaped with double quote and the `EscapeTokens.Apostrophe` values for parse/write a file having fields to be escaped with the apostrophe.
+>The `EscapeTokens` enumeration is used to "escape" some CSV/TSV fields with embedded escape characters. The `parserConfig.unixEscapeMechanism` option will tell the parser to escape in the unix style, preceding the backslash (`\`), or in the classic way, preceding another escape character.
 {: .text-grey-dk-300 .bg-grey-lt-000 }
 
 See also

docs/api/enumerations/escapetokens.md

@@ -15,7 +15,7 @@ Exports an array's content to a CSV/TSV file.
 
 ## Syntax
 
-*expression*.`ExportToCSV`*(csvArray, \[pconfig:= `Nothing`\], \[PassControlToOS:= `True`\], \[enableDelimiterGuessing:= `True`\])*
+*expression*.`ExportToCSV`*(csvArray, \[pconfig:= `Nothing`\], \[PassControlToOS:= `True`\], \[enableDelimiterGuessing:= `True`\], \[EnforcedQuotation:= `False`\])*
 
 ### Parameters
 
@@ -43,6 +43,10 @@ Exports an array's content to a CSV/TSV file.
 <td style="text-align: left;"><em>enableDelimiterGuessing</em></td>
 <td style="text-align: left;">Optional. Identifier specifying a <code>Boolean</code> Type variable.</td>
 </tr>
+<tr>
+<td style="text-align: left;"><em>EnforcedQuotation</em></td>
+<td style="text-align: left;">Optional. Identifier specifying a <code>Boolean</code> Type variable.</td>
+</tr>
 </tbody>
 </table>
 
@@ -62,11 +66,11 @@ See also
 
 ## Behavior
 
-If the `pconfig` parameter is omited, the parser will use the `ParseConfig` property as configuration object. If the file specified in the `.path` configuration property all ready exist and have some content on it the parser will try to guess delimiters and the data will be append to the file.
+When the `pconfig` parameter is omitted, the parser will use the `ParseConfig` property as the configuration object. If the file specified in the `.path` configuration property already exists and has some content, the parser will try to guess the delimiters and the data will be added to the file. Setting the `EnforcedQuotation` property to `True` will force to quote all fields in the created CSV file. 
 
 >⚠️**Caution**
 >{: .text-grey-lt-000 .bg-green-000 }
->If there is an unescaped field, the `ExportToCSV` method will append the specified escape character to the end of the field. This is because if the data is written with this problem, the parser will not be able to import it on future occasions.
+>Fields containing literal escape characters will be escaped using the classic escape sequence (duplicating each escape character) or using the Unix escape sequence. The behavior will be controlled by the `unixEscapeMechanism` property of the given configuration object.
 {: .text-grey-dk-300 .bg-yellow-000 }
 
 [Back to Methods overview](https://ws-garcia.github.io/VBA-CSV-interface/api/methods/)

docs/api/methods/exporttocsv.md

@@ -118,7 +118,7 @@ _Yes_
 <tr>
 <td style="text-align: left; color:blue;"><em>escapeToken</em></td>
 <td style="text-align: left;">Property</td>
-<td style="text-align: left;">Gets or sets the char that will be used for escape those fields containing some CSV/TSV syntax special char. The user must use the <code>EscapeTokens</code> enumeration to define this property.</td>
+<td style="text-align: left;">Gets or sets the char that will be used for quote those fields containing some CSV/TSV syntax special char. The user must use the <code>EscapeTokens</code> enumeration to define this property. The user can choose between double quotes ("), single quotes (') and tilde (~) as escapeToken.</td>
 </tr>
 <tr>
 <td style="text-align: left; color:blue;"><em>fieldsDelimiter</em></td>
@@ -165,6 +165,11 @@ _Yes_
 <td style="text-align: left;">Property</td>
 <td style="text-align: left;">Gets or sets the behavior of the parser when reading streams from text files. By default, this property is set to <code>False</code>. If the value is set to <code>True</code>, all line break characters in the loaded stream will be converted to <code>vbLf</code>. This option will affect performance, but may be useful when faced with CSV files with <code>vbCrLf</code>, <code>vbCr</code> and <code>vbLf</code> mixed in as line endings.</td>
 </tr>
+<tr>
+<td style="text-align: left; color:blue;"><em>unixEscapeMechanism</em></td>
+<td style="text-align: left;">Property</td>
+<td style="text-align: left;">Gets or sets the behavior of the parser when escaping or unescaping quotes. By default, this property is set to <code>False</code>. If the value is set to <code>True</code>, the escape character must be escaped by preceding it with a unix-style backslash (<code>\"</code>).</td>
+</tr>
 </tbody>
 </table>
 

docs/api/properties/parseconf.md

@@ -13,9 +13,10 @@ VBA CSV interface is the most complete, and open source, CSV/TSV VBA parser libr
 {: .fs-6 .fw-300 }
 
 ## Advantages
-* __Stable__. Fully Test Driven Developed (TDD) library, ([48/48 test passed](https://github.com/ws-garcia/VBA-CSV-interface/blob/master/testing/tests/results/)), that includes 500+ line of code for testing. See [VBA test library by Tim Hall](https://github.com/ws-garcia/vba-test).
+* __Stable__. Fully Test Driven Developed (TDD) library, ([49/49 test passed](https://github.com/ws-garcia/VBA-CSV-interface/blob/master/testing/tests/results/)), that includes 500+ line of code for testing. See [VBA test library by Tim Hall](https://github.com/ws-garcia/vba-test).
 * __Fast__. Writes and reads files at the highest speed.
 * __Memory-friendly__. CSV/[TSV](https://www.iana.org/assignments/media-types/text/tab-separated-values) files are processed using a custom stream technique, only 0.5MB are in memory at a time.
+* __Robust__. Parser and writer accept [Unix-style escape sequences](https://www.loc.gov/preservation/digital/formats/fdd/fdd000323.shtml#useful).
 * __Easy to use__. A few lines of code can do the work!
 * __Highly Configurable__. User can configure the parser to work with a wide range of CSV files.
 * __CSV data subsetting__. Split CSV data into a set of files with related data.