Skip to content

[WIP] add support for TypedArrays/arrayBuffer/nodejs' Buffer#716

Draft
dotnetCarpenter wants to merge 1 commit intosanctuary-js:mainfrom
dotnetCarpenter:feature-support-buffers
Draft

[WIP] add support for TypedArrays/arrayBuffer/nodejs' Buffer#716
dotnetCarpenter wants to merge 1 commit intosanctuary-js:mainfrom
dotnetCarpenter:feature-support-buffers

Conversation

@dotnetCarpenter
Copy link

Currently failing to implement tests for TypedArray.

In assert.strictEqual (equals (actual) (expected), true); the
Setoid.test test fails unconditionaly for a TypedArray (Uint8Array).

Need help ;)

Reproduce:

const S = require ('./internal/sanctuary');
const eq = require ('./internal/eq');

eq (S.unchecked.drop (1) (new Uint8Array([10, 20, 30, 40, 50]))) (S.unchecked.Just (new Uint8Array([20, 30, 40, 50])));

PS. For now my goal is to support various Buffers for unchecked functions, since it seems like this should be opt-in and I don't know how to implement support in https://github.com/sanctuary-js/sanctuary-def yet.

Currently failing to implement tests for TypedArray.

In `assert.strictEqual (equals (actual) (expected), true);` the
`Setoid.test` test fails unconditionaly for a TypedArray (`Uint8Array`).

Need help ;)
@dotnetCarpenter dotnetCarpenter changed the title [WIP] add support for TypedArrays/arraBuffer/nodejs' Buffer [WIP] add support for TypedArrays/arrayBuffer/nodejs' Buffer Aug 16, 2021
@davidchambers
Copy link
Member

I don't think this makes sense.

Here is the type of take:

take :: (Applicative f, Foldable f, Monoid (f a)) => Integer -> f a -> Maybe (f a)

We could specialize this by replacing f with Array and a with String:

take :: Integer -> Array String -> Maybe (Array String)

Doing the same with Uint8Array results in a nonsensical type:

take :: Integer -> Uint8Array String -> Maybe (Uint8Array String)

Uint8Array is a type rather than a type constructor: these arrays only ever contain 8-bit unsigned integers.

One could define a specialized function:

takeUint8 :: Integer -> Uint8Array -> Maybe Uint8Array

@dotnetCarpenter
Copy link
Author

dotnetCarpenter commented Aug 16, 2021

@davidchambers How would I go about implementing specialised functions for Sanctuary?

Would TypedArray encompass all of Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array, Float64Array, BigInt64Array and BigUint64Array?

takeTypedArray :: Integer -> TypedArray -> Maybe TypedArray

@davidchambers
Copy link
Member

The hypothetical TypedArray type is problematic because it would allow takeTypedArray to take a Uint8Array but return a Maybe Float64Array.

The simplest solution would be to define a specialized function for each type.

@dotnetCarpenter
Copy link
Author

So that would be 11 specialized functions, just for take?

takeInt8Array :: Integer -> Int8Array -> Maybe Int8Array
takeUint8Array :: NonNegativeInteger -> Uint8Array -> Maybe Uint8Array
takeUint8ClampedArray :: NonNegativeInteger -> Uint8ClampedArray -> Maybe Uint8ClampedArray
takeInt16Array :: Integer -> Int16Array -> Maybe Int16Array
takeUint16Array :: NonNegativeInteger -> Uint16Array -> Maybe Uint16Array
takeInt32Array :: Integer -> Int32Array -> Maybe Int32Array
takeUint32Array :: NonNegativeInteger -> Uint32Array -> Maybe Uint32Array
takeFloat32Array :: ValidNumber -> Float32Array -> Maybe Float32Array
takeFloat64Array :: ValidNumber -> Float64Array -> Maybe Float64Array

Does Sanctuary support BigInt?

takeBigInt64Array :: BigInt -> BigInt64Array -> Maybe BigInt64Array
takeBigUint64Array :: NonNegativeBigInt -> BigUint64Array -> Maybe BigUint64Array

But actually, before going down this rabbit hole. I'm much more interested in support for Nodejs' Buffer since I deal with a database that returns Buffers.

takeBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

dropBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

takeLastBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

dropLastBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

takeWhileBuffer :: (a -> Boolean) -> Buffer -> Buffer

dropWhileBuffer :: (a -> Boolean) -> Buffer -> Buffer

Ref: NonNegativeInteger

@dotnetCarpenter
Copy link
Author

I tried to look through some code a few months back to find context for this. I did not find anything.

But as I recall, I got a Buffer from the database and wanted to use S.unchecked.drop but the Array,isArray check in _takeDrop would not let me, so I used .slice instead without a Maybe.

Changing that one line, made it all work in Sanctuary but .... here I am.

@Avaq
Copy link
Member

Avaq commented Aug 16, 2021

If I'd want to take from a Buffer, I'd probably do something like:

// firstFiveBytes :: Buffer -> Maybe Buffer
const firstFiveBytes = S.pipe ([
  Array.from,
  S.take (5),
  S.map (Buffer.from),
])

@dotnetCarpenter
Copy link
Author

@Avaq I'm imagining that working with Buffers without converting them to Array would be faster overall. But I should write some code to compare with.

There is only one line that need to change in Sanctuary in order to support Buffers, so it should be fairly easy to compare.

But it won't be today :)

@imcotton
Copy link

imcotton commented Aug 26, 2021

Quick note: slice creates a copy of internal memory v.s. subarray keeps its ArrayBuffer untouched (shared).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants