-
Notifications
You must be signed in to change notification settings - Fork 168
Description
version: 1.29.1-dev132.0
I am using DuckDB WASM to read CSV files. The file may contain multiple formats for Time/Date/TimeStamps, as such I am not able to rely on passing a custom format to the read_csv operation (It only supports a single format for the entire file).
To work around this:
- I have so far been loading the CSV, with Dates/Time/Timestamps out of the auto-candidates.
- I then loop over each varchar column. For each column, I then loop over, a variety of custom time/date formats using
strptime. - If it throws an exception I move onto the next format etc. If there is no exception thrown, I note the date/time format that was successful.
- I then later apply a final
SELECTstatement with the adjustments for each column, (including a CAST)
This appears to work well. As strptime will throw an exception for the format that fails to parse, I have wrapped that statement in a try/catch. However the exception is still being printed to the console. I do not wish for the error to be printed. I have also attempted to use VoidLogger etc.
I prefer strptime over try_strptime as it fails fast.
It appears this issue is being caused by the following console.log.
| console.log(e); |
As its within the WebWorker, my ability to patch it out is limited.
What would be the thoughts on removing the console.log. I feel it should use the existing DuckDB logging framework instead.
const timeCasts = {};
for (const col of stringTypes) {
for (const fmt of ADDITIONAL_STRP_TIME_FORMATS) {
try {
await this.conn.query(`SELECT strptime("${col.name}", '${fmt.str}') FROM temp_csv LIMIT 20480`);
timeCasts[col.name] = fmt;
break;
} catch (e) {}
}
}