Skip to content

Commit a072c38

Browse files
committed
Set copy to true by default (#71)
1 parent 6aa0ac1 commit a072c38

File tree

4 files changed

+93
-24
lines changed

4 files changed

+93
-24
lines changed

README.md

Lines changed: 39 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,20 +24,40 @@ const WASM_MEMORY: WebAssembly.Memory = ...
2424
const field = parseField(WASM_MEMORY.buffer, fieldPtr);
2525
```
2626

27+
### `parseData`
28+
29+
Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) C FFI struct into an [`arrow.Data`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Data.html) instance. Multiple `Data` instances can be joined to make an [`arrow.Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).
30+
31+
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
32+
- `ptr` (`number`): The numeric pointer in `buffer` where the C struct is located.
33+
- `dataType` (`arrow.DataType`): The type of the vector to parse. This is retrieved from `field.type` on the result of `parseField`.
34+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Data` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
35+
36+
#### Example
37+
38+
```ts
39+
const WASM_MEMORY: WebAssembly.Memory = ...
40+
const copiedData = parseData(WASM_MEMORY.buffer, arrayPtr, field.type);
41+
// Make zero-copy views instead of copying array contents
42+
const viewedData = parseData(WASM_MEMORY.buffer, arrayPtr, field.type, false);
43+
```
44+
2745
### `parseVector`
2846

2947
Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) C FFI struct into an [`arrow.Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html) instance. Multiple `Vector` instances can be joined to make an [`arrow.Table`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html).
3048

3149
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
3250
- `ptr` (`number`): The numeric pointer in `buffer` where the C struct is located.
3351
- `dataType` (`arrow.DataType`): The type of the vector to parse. This is retrieved from `field.type` on the result of `parseField`.
34-
- `copy` (`boolean`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
52+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
53+
54+
#### Example
3555

3656
```ts
3757
const WASM_MEMORY: WebAssembly.Memory = ...
38-
const wasmVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type);
39-
// Copy arrays into JS instead of creating views
40-
const wasmVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type, true);
58+
const copiedVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type);
59+
// Make zero-copy views instead of copying array contents
60+
const viewedVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type, false);
4161
```
4262

4363
### `parseRecordBatch`
@@ -47,12 +67,24 @@ Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html
4767
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
4868
- `arrayPtr` (`number`): The numeric pointer in `buffer` where the _array_ C struct is located.
4969
- `schemaPtr` (`number`): The numeric pointer in `buffer` where the _field_ C struct is located.
50-
- `copy` (`boolean`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
70+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
71+
72+
#### Example
5173

5274
```ts
5375
const WASM_MEMORY: WebAssembly.Memory = ...
54-
// Pass `true` to copy arrays across the boundary instead of creating views.
55-
const recordBatch = parseRecordBatch(WASM_MEMORY.buffer, arrayPtr, fieldPtr, true);
76+
const copiedRecordBatch = parseRecordBatch(
77+
WASM_MEMORY.buffer,
78+
arrayPtr,
79+
fieldPtr
80+
);
81+
// Pass `false` to view arrays across the boundary instead of creating copies.
82+
const viewedRecordBatch = parseRecordBatch(
83+
WASM_MEMORY.buffer,
84+
arrayPtr,
85+
fieldPtr,
86+
false
87+
);
5688
```
5789

5890
## Type Support

src/index.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
export { parseVector } from "./vector";
1+
export { parseVector, parseData } from "./vector";
22
export { parseField } from "./field";
33
export { parseRecordBatch } from "./record-batch";

src/record-batch.ts

Lines changed: 25 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,24 +3,36 @@ import { parseField } from "./field";
33
import { parseData } from "./vector";
44

55
/**
6-
* Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) C FFI struct _plus_ an [`ArrowSchema`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowschema-structure) C FFI struct into an `arrow.RecordBatch` instance. Note that the underlying array and field **must** be a `Struct` type. In essence a `Struct` array is used to mimic a `RecordBatch` while only being one array.
7-
*
8-
* - `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
9-
* - `arrayPtr` (`number`): The numeric pointer in `buffer` where the _array_ C struct is located.
10-
* - `schemaPtr` (`number`): The numeric pointer in `buffer` where the _field_ C struct is located.
11-
* - `copy` (`boolean`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
12-
*
13-
* ```ts
14-
* const WASM_MEMORY: WebAssembly.Memory = ...
15-
* // Pass `true` to copy arrays across the boundary instead of creating views.
16-
* const recordBatch = parseRecordBatch(WASM_MEMORY.buffer, arrayPtr, fieldPtr, true);
17-
* ```
6+
Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) C FFI struct _plus_ an [`ArrowSchema`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowschema-structure) C FFI struct into an `arrow.RecordBatch` instance. Note that the underlying array and field **must** be a `Struct` type. In essence a `Struct` array is used to mimic a `RecordBatch` while only being one array.
7+
8+
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
9+
- `arrayPtr` (`number`): The numeric pointer in `buffer` where the _array_ C struct is located.
10+
- `schemaPtr` (`number`): The numeric pointer in `buffer` where the _field_ C struct is located.
11+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
12+
13+
#### Example
14+
15+
```ts
16+
const WASM_MEMORY: WebAssembly.Memory = ...
17+
const copiedRecordBatch = parseRecordBatch(
18+
WASM_MEMORY.buffer,
19+
arrayPtr,
20+
fieldPtr
21+
);
22+
// Pass `false` to view arrays across the boundary instead of creating copies.
23+
const viewedRecordBatch = parseRecordBatch(
24+
WASM_MEMORY.buffer,
25+
arrayPtr,
26+
fieldPtr,
27+
false
28+
);
29+
```
1830
*/
1931
export function parseRecordBatch(
2032
buffer: ArrayBuffer,
2133
arrayPtr: number,
2234
schemaPtr: number,
23-
copy: boolean = false,
35+
copy: boolean = true,
2436
): arrow.RecordBatch {
2537
const field = parseField(buffer, schemaPtr);
2638
if (!isStructField(field)) {

src/vector.ts

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,23 +10,48 @@ Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html
1010
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
1111
- `ptr` (`number`): The numeric pointer in `buffer` where the C struct is located.
1212
- `dataType` (`arrow.DataType`): The type of the vector to parse. This is retrieved from `field.type` on the result of `parseField`.
13-
- `copy` (`boolean`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
13+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Vector` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
14+
15+
#### Example
16+
17+
```ts
18+
const WASM_MEMORY: WebAssembly.Memory = ...
19+
const copiedVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type);
20+
// Make zero-copy views instead of copying array contents
21+
const viewedVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type, false);
1422
*/
1523
export function parseVector<T extends DataType>(
1624
buffer: ArrayBuffer,
1725
ptr: number,
1826
dataType: T,
19-
copy: boolean = false,
27+
copy: boolean = true,
2028
): arrow.Vector<T> {
2129
const data = parseData(buffer, ptr, dataType, copy);
2230
return arrow.makeVector(data);
2331
}
2432

33+
/**
34+
Parse an [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) C FFI struct into an [`arrow.Data`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Data.html) instance. Multiple `Data` instances can be joined to make an [`arrow.Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).
35+
36+
- `buffer` (`ArrayBuffer`): The [`WebAssembly.Memory`](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory) instance to read from.
37+
- `ptr` (`number`): The numeric pointer in `buffer` where the C struct is located.
38+
- `dataType` (`arrow.DataType`): The type of the vector to parse. This is retrieved from `field.type` on the result of `parseField`.
39+
- `copy` (`boolean`, default: `true`): If `true`, will _copy_ data across the Wasm boundary, allowing you to delete the copy on the Wasm side. If `false`, the resulting `arrow.Data` objects will be _views_ on Wasm memory. This requires careful usage as the arrays will become invalid if the memory region in Wasm changes.
40+
41+
#### Example
42+
43+
```ts
44+
const WASM_MEMORY: WebAssembly.Memory = ...
45+
const copiedData = parseData(WASM_MEMORY.buffer, arrayPtr, field.type);
46+
// Make zero-copy views instead of copying array contents
47+
const viewedData = parseData(WASM_MEMORY.buffer, arrayPtr, field.type, false);
48+
```
49+
*/
2550
export function parseData<T extends DataType>(
2651
buffer: ArrayBuffer,
2752
ptr: number,
2853
dataType: T,
29-
copy: boolean = false,
54+
copy: boolean = true,
3055
): arrow.Data<T> {
3156
const dataView = new DataView(buffer);
3257

0 commit comments

Comments
 (0)