Skip to content

v2.0.0
Compare
Choose a tag to compare
@toptobes toptobes released this 14 Apr 21:06
· 16 commits to master since this release
v2.0.0
2069720

v2.0.0 - Major client overhaul

Warning

Release notes are still under construction

Table of contents

Note

Asterisks (*) by section names indicate those that include note of any breaking changes from v1.x to v2.x.

What's new?

Prerequisites*

  • Node.js v18 or higher
  • TypeScript v5.0 or higher

Data API tables support

Adds complete support for Data API tables via the Table class.

Dual types*

Classes/types such as UpdateOneOptions, FindCursor, InsertManyError, etc. have been split into two different variants of each:

  • CollectionUpdateOneOptions and TableUpdateOneOptions,
  • CollectionFindand CursorTableFindCursor,
  • CollectionInsertManyError and TableInsertManyError,
  • etc.

Collection methods on the Db class (such as db.collection(), db.createCollection(), db.listCollections(), etc.) now have sister methods for tables:

  • db.table(),
  • db.createTable(),
  • db.listTables(),
  • etc.

Indexes

Furthermore, tables necessitate indexes, which may be created through table.createIndex() (and its sister methods), and dropped through db.dropTableIndex().

Utility types

New utility types to infer the table's TS-type from its schema has been added as well:

  • InferTableSchema
  • InferTablePrimaryKey

Datatypes

Lastly, of the major additions, new datatypes have been added as well to support the new tables:

  • DataAPIBlob
  • DataAPIDate
  • DataAPITime
  • DataAPIDuration
  • DataAPIInet
  • DataAPIVector
  • BigNumber (re-exported from bignumber.js)

See DATATYPES.md for more information on the new datatypes.

findAndRerank support

Note

This is tied to the Overhaul of cursors section; please read that section as well for more information.

Preview hybrid search support has dropped via the collection.findAndRerank() method, which uses a new FindAndRerankCursor to iterate over its results.

New LexicalDoc type added as well as a sister type to VectorDoc and VectorizeDoc.

Overhaul of the logging system*

The logging system has evolved beyond a simple, immutable monitorCommands flag which would emit events only on the DataAPIClient class.

Now, logging is configured through the logging option anywhere along the options hierarchy, and adds a variety of new features:

  • Events may log to the console as well now, as opposed to just being emitted as an event
  • All classes in the client hierarchy are now event listeners
    • These all extend the HierarchicalLogger class, which implements a minimal event emitter
    • Events are bubbled up from the class they were emitted from, all the way up to the root DataAPIClient
    • Event bubbling may be cancelled through event.stop[Immediate]Propagation() like the DOM

Logging may also be enabled/disabled/reconfigured on the fly, via the .updateLoggingConfig(<config>) method on any HierarchicalLogger instance (e.g. DataAPIClient, Db, Collection, etc.).

See the logging examples for more information.

Overhaul of the timeout system*

The timeout system has been overhauled to be more flexible and descriptive than the current singular maxTimeMS.

Timeouts are now configured through the timeoutDefaults parameter on class options interfaces, and the timeout parameter on method calls.

They are represented by the TimeoutDescriptor class, which contains the six following keys:

Timeout Option Description Default
requestTimeoutMs Maximum time the client waits for a response from the server 15 seconds
generalMethodTimeoutMs Timeout for general methods without a specific override (mostly doc/row-level ops) 30 seconds
collectionAdminTimeoutMs Timeout for collection admin operations (create, drop, list, etc.) 1 minute
tableAdminTimeoutMs Timeout for table admin operations (create, drop, list, alter, create/dropIndex, etc.) 30 seconds
databaseAdminTimeoutMs Timeout for database admin operations (create, drop, list, info, findEmbeddingProviders, etc.) 10 minutes
keyspaceAdminTimeoutMs Timeout for keyspace admin operations (create, drop, list) 30 seconds

In general, the requestTimeoutMs always corresponds to a single request from the server, but the rest of the timeouts account for the entire duration of the method, which is relevant in the case of multi-call methods.

A few important notes:

  • requestTimeoutMs applies per individual HTTP request.
  • The other timeouts account for the entire duration of the method, which is relevant in the case of multi-call methods.
  • Passing a timeout value as a plain number will use the most appropriate category for the method being called.
  • Setting a timeout to 0 disables it completely.
  • The legacy maxTimeMS option has been removed—simply replace it with timeout for a quick migration path.

Overhaul of cursors*

Cursors are no longer a single FindCursor class, but are now a hierarchy of classes (click the link for a visualization of the class hierarchy).

  • The classes now include: AbstractCursor, FindCursor, FindAndRerankCursor, CollectionFindCursor, TableFindCursor, and CollectionFindAndRerankCursor.

Cursor methods have also been overhauled.

  • Most importantly, builder methods no longer mutate the cursor. Instead, they return a fresh copy of the cursor with the new option set.
    • For example, cursor.limit(10) will return a new cursor with the limit set to 10, and the original cursor will remain unchanged.
  • There have been a few method/property changes as well:
    • .bufferedCount() => .buffered()
    • .readBufferedDocuments() => .consumeBuffer()
    • added the .consumed(): number property
      • returns the number of documents actually consumed from the cursor
    • added the .dataSource: Table | Collection property
      • returns the table or collection the cursor is associated with
    • added the .state: CursorState property
      • returns the current state of the cursor; may be 'idle', 'started', or 'closed'
    • removed the .keyspace property
      • use dataSource.keyspace instead
    • removed the .closed property
      • use .state === 'closed' instead
    • un-deprecated .forEach()

Cursors are also now typed as *Cursor<T, TRaw extends SomeDoc = SomeDoc> rather than just *Cursor<T>.

  • T represents the type of doc after any mappings have been applied (via cursor.map())
  • TRaw represents the original type of the doc, before any mappings have been applied

However, for most intents and purposes, you may continue to treat cursors as if they were still typed as *Cursor<T>.

Lastly, of the important changes, .clone() no longer strips the mapping from the cursor.

Overhauls of certain admin functionality*

The astraAdmin.dbInfo(), astraAdmin.listDatabases(), dbAdmin.info(), and db.info() methods have been updated to return a curated set of information about the database, rather than the raw response from the server.

The raw response may still be found in the info.raw property, but the info object itself now contains a more user-friendly set of properties.

The info objects have been renamed to AstraPartialDatabaseInfo and AstraFullDatabaseInfo to reflect this change.

Increased client compatibility across all environments*

General client compatability has been improved in many ways, the most significant of which is the new seamless dual support for both CommonJS and ES modules.

Beyond that, the client has also:

  • Removed its hard dependency on the events package
    • It did this by implementing its own HierarchicalLogger class, which contains a minimal event emitter implementation
    • Any events polyfill may be safely removed from your project if you are not using it anywhere else
  • Removed its hard dependency on the fetch-h2 package
    • It did this by using the native fetch implementation as the default fetcher
    • The fetch-h2 package now must be imported separately and passed to the client manually
  • Ensures complete support across both server and browser environments
    • This is guaranteed by dogfooding the client in the DataStax Astra portal
  • Uses tslib now to help reduce the library size

New fully-customizable ser/des system

Section still under construction; more details to come

Errors improvements*

Section still under construction; more details to come

Various minor changes and improvements

Section still under construction; more details to come

db.createCollection changes*

The checkExists parameter has been removed; the method now does not check for existence before attempting to create the collection.

  • This is because collection created is already idempotent (if the definition is the same), and thus, the check was unnecessary

Collection typing

Note

Unless you are using custom ser/des, you may continue to treat Collection as if it were still typed as simply Collection<Schema>.

Collection<Schema extends SomeDoc = SomeDoc> was updated to be typed as the following:

class Collection<WSchema extends SomeDoc = SomeDoc, RSchema extends WithId<SomeDoc> = FoundDoc<WSchema>> {}

where:

  • WSchema is the type of the row as it's written to the table (the "write" schema)
  • RSchema is the type of the row as it's read from the table (the "read" schema)
  • FoundDoc is a generic type which represents the type of the document as it's returned from the Data API (i.e. in read operations).

In astra-db-ts version 1.x, FoundDoc was used directly in the return type of find, findOne, and other find-ing types.

async findOne(filter: Filter, options: FindOneOptions): Promise<FoundDoc<Schema> | null>;

However, now that FoundDoc and FoundRow are type parameters of the Collection and Table types themselves, the read-types may explicitly override for advanced use cases.

DataAPIVector

The DataAPIVector class is now generally recommended to be used over raw number[]s as it offers a more performant and compact representation of vectors.

While you may still write vectors as a number[], the DataAPIVector class is now what's returned from reading from a collection or table, which will cause breaking changes for code expecting number[]s to be read back.

Other datatype updates

UUID.v1() and UUID.v6() are now available as static methods on the UUID class, alongside the existing UUID.v4() and UUID.v7() members.

Also, new uuid and oid shorthand functions/objects have been added alongside the other shorthands for the new table datatypes:

  • uuid(...) => new UUID(...)
  • uuid.v4() => UUID.v4()
  • oid(...) => new ObjectId(...)
  • oid() => new ObjectId()

escapeFieldNames / unescapeFieldPath

New utility functions have been introduced for safely escaping and un-escaping field paths when working with the Data API.

These functions ensure that special characters such as . and & in field names don't interfere with query path parsing, while preserving accurate round-tripping of deeply nested fields.

// 'shows.tom&&jerry.views'
escapeFieldNames('shows', 'tom&jerry', 'views')

// ['users', 'john.doe', 'profile']
unescapeFieldPath('users.john&.doe.profile')

Documentation overhauls

The TSDoc is in the process of being heavily overhauled to be much clearer, consistent, and easier to read, through the use of section headers, separators, and admonitions.

  • This is still a work in progress; not all areas of the client have been updated yet
  • See the Collection class for an example of the new documentation style

Better type validation errors

astra-db-ts validates options and arguments at runtime when creating/spawning new classes, and has been improved to provide better parse error messages when invalid options are passed, via the decoders library.

Testing

  • Test coverage has been heavily improved across the board
  • Property-based testing via fast-check has been integrated into the test suite

Migration tips

Warning

Migration notes are still under construction

Helpful migration type-errors

In many cases, to help you migrate to the new version, the client will throw helpful type errors when you try to use deprecated or removed options.

const coll = await db.createCollection('my_coll', {
  // TS2322: Type false is not assignable to type
  // 'ERROR: `checkExists` has been removed. It is equivalent to being always `false` now.'
  checkExists: false
});

// TS2322: Type number is not assignable to type
// 'ERROR: The `maxTimeMS` option is no longer available; the timeouts system has been overhauled, and timeouts should now be set using `timeout`'
await coll.findOne({}, { maxTimeMS: 1000 });

Splitting types

Many types have been split into two different variants of each:

  • Collection variants (e.g. CollectionUpdateOneOptions, CollectionFindCursor, etc.)
  • Table variants (e.g. TableUpdateOneOptions, TableFindCursor, etc.)

If you run into a type no longer being found, check to see if it has been split into one of these variants.

Cursors

Cursors must most importantly now be mindful that they are no longer mutated by builder methods.

In general though, due to the not insignificant number of changes to cursors, please read and accommodate the changes outlined in the overhaul of cursors section, and read the TSDoc of the cursors classes for any more information.

Vectors

It is recommended to migrate to using the DataAPIVector class rather than raw number[]s, as it is now the default type returned from reading vectors from a collection or table.

If you really need to use number[]s everywhere, you may write a simple codec to change this ser/des behavior.

maxTimeMS

You may generally just replace maxTimeMS with timeout, and it will work as expected.

Re-enabling HTTP2

If you are using HTTP2, you will need to import the fetch-h2 package separately and pass it to the client manually.

See the using-http2 example to see how to do this in just a couple easy steps.

Assets 2
Loading