Text encode decode

[hhugo]

No description provided.

[hhugo]

I'd like to benchmark this change

[vouillon]

One should create the encoder and the decoder only once and reuse it, and use a fixed buffer for caml_jsstring_of_string.

[hhugo]

One should create the encoder and the decoder only once and reuse it, and use a fixed buffer for caml_jsstring_of_string.

Done

[hhugo]

@vouillon, any idea on how to benchmark this other that micro benchmarks ?

[hhugo]

A quick micro benchmark show a significant slowdown.
master

n: 3, 0.167000
n: 7, 0.268000
n: 10, 0.371000
n: 20, 0.811000
n: 50, 1.989000
n: 100, 3.618000

this PR

n: 3, 1.209000
n: 7, 1.269000
n: 10, 1.354000
n: 20, 1.660000
n: 50, 2.632000
n: 100, 4.578000

[Copilot]

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

runtime/js/mlBytes.js:642

• The function caml_jsstring_of_string should handle non-ASCII strings correctly. The previous implementation used caml_utf16_of_utf8, which may handle edge cases differently than TextDecoder.

if (jsoo_is_ascii(s)) return s;

runtime/js/mlBytes.js:659

• The function caml_string_of_jsstring should handle non-ASCII strings correctly. The previous implementation used caml_utf8_of_utf16, which may handle edge cases differently than TextEncoder.

if (jsoo_is_ascii(s)) return caml_string_of_jsbytes(s);

[hhugo]

@vouillon, do you have any opinion on this ?

[OlivierNicole]

A quick micro benchmark show a significant slowdown. master

n: 3, 0.167000
n: 7, 0.268000
n: 10, 0.371000
n: 20, 0.811000
n: 50, 1.989000
n: 100, 3.618000

this PR

n: 3, 1.209000
n: 7, 1.269000
n: 10, 1.354000
n: 20, 1.660000
n: 50, 2.632000
n: 100, 4.578000

Was this slowdown mitigated, or was the end decision to go with it?

[hhugo]

Was this slowdown mitigated, or was the end decision to go with it?

The slowdown should only affect non-ascii strings.
Also, my assumption was that conversions between js and ml strings should be negligible compared to other computations.
Recent bugs found in our previous implementation made me decide to ignore the performance impact and merge this PR

[OlivierNicole]

I see. Thanks!