Issue with non-ASCII characters in JSON() import #2112
Description
When I create a JSON file with non-ASCII like:
[
{
"property": "惺"
}
]...and import it through the browser into a database.
await alasql.promise([
'CREATE DATABASE IF NOT EXISTS test1',
'USE test1',
'DROP TABLE IF EXISTS testtable',
'CREATE table testtable',
'SELECT * INTO testtable FROM JSON("json/test.json")'
]);
const results = await alasql.promise(
'SELECT * FROM testtable'
);
console.log('results', results[0]);I get {property: 'æ\x83º'} as the result instead of what I would expect: {property: '惺'}
The same problem does not occur in Node.
The issue is apparently with this function used by fetchData which is used by loadFile which is used by alasql.from.JSON:
function getData(path, success, error) {
return _fetch(path)
.then(response => response.arrayBuffer())
.then(buf => {
var a = new Uint8Array(buf);
var b = [...a].map(e => String.fromCharCode(e)).join('');
success(b);
})
.catch(e => {
if (error) return error(e);
console.error(e);
throw e;
});
}I'm not sure why this is being loaded as an array buffer and then processed in this manner.
When I change the code to the following, the content of my UTF8 JSON file is processed as I would expect:
function getData(path, success, error) {
return _fetch(path)
.then(response => response.text())
.then(txt => {
success(txt);
})
.catch(e => {
if (error) return error(e);
console.error(e);
throw e;
});
}Can you just change the code as above, or do you need some other handling?
Thanks!