-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Problem
The web-sdk does not provide a mechanism for streaming TDF creation or writing data in chunks. In a browser or Node.js environment, processing large files is only feasible through streaming APIs (e.g., ReadableStream
, WritableStream
). The current approach of buffering an entire file for encryption is impractical and limits the SDKs applicability for modern web applications handling significant data volumes.
User Story
As a JavaScript/TypeScript developer, I need a primitive writer interface to create a ZTDF by streaming data. The interface should let me write encrypted data chunks as they become available and then call a final method to seal the TDF by creating the manifest and adding the key information. This will enable me to build web applications that can encrypt and upload large files efficiently.
Misc
This feature should introduce a low-level writer for ZTDFs, enabling chunked and out-of-order data writing. The design must prioritize safety and prevent developers from misusing sensitive cryptographic or policy controls.
We must ensure that this powerful interface doesnt become a "footgun." Core security features should be enforced by default. If any sensitive controls are exposed, they must be fire-walled from the public API and left undocumented for external consumers.
Acceptance Criteria
Functional Requirements
- A primitive writer interface is added to the web-sdk for creating ZTDF files.
- The interface is compatible with modern streaming APIs (e.g.,
ReadableStream
,WritableStream
). - The interface supports writing data segments in a non-sequential manner.
- A
.close()
or.finish()
method is available to finalize the TDF, which includes writing the manifest and key information. - The output TDF is valid and decryptable by other OpenTDF clients.
Non-Functional Requirements
- Performance: The implementation must efficiently handle streams without requiring the entire file to be loaded into memory.
- Memory: Memory usage must be minimal and independent of the source files total size.
- Security: The interface must not expose knobs that could weaken the TDFs cryptographic guarantees or policy enforcement.
- Documentation: Public documentation should only cover the safe, high-level usage of the interface. Any internal-only functions should be omitted.