Skip to content

Conversation

@va3093
Copy link
Collaborator

@va3093 va3093 commented Nov 14, 2025

Context

The Dune API has introduced new /v1/datasets and /v1/uploads endpoints to replace the legacy /v1/table endpoints. This update aligns the TypeScript client with the latest API specification and provides users with access to new functionality while maintaining backward compatibility.

Currently, the client only supports the legacy /v1/table endpoints which are being phased out in favor of the more comprehensive and better-organized /v1/uploads endpoints. Additionally, there was no support for the new /v1/datasets endpoints which provide read access to dataset information.

What Changed

New APIs Added

DatasetAPI (client.dataset):

  • list(args?) - List datasets with filtering by owner and type
  • getBySlug(slug) - Get detailed dataset information by slug

UploadsAPI (client.uploads):

  • list(args?) - List all uploaded tables for the account
  • create(args) - Create an empty table with defined schema
  • uploadCsv(args) - Upload CSV data as a new table
  • insert(args) - Insert data into an existing table (CSV/JSON)
  • delete(args) - Delete an uploaded table
  • clear(args) - New! Clear all data from a table while preserving schema

Deprecation Changes

All four methods in TableAPI are now deprecated with console warnings:

  • table.uploadCsv() → Use uploads.uploadCsv()
  • table.create() → Use uploads.create()
  • table.insert() → Use uploads.insert()
  • table.delete() → Use uploads.delete()

The deprecated methods continue to work exactly as before, ensuring full backward compatibility. Users will see helpful deprecation warnings guiding them to the new API.

Type Definitions

Added comprehensive type definitions matching the OpenAPI specification:

  • Request types: ListDatasetsArgs, ListUploadsArgs, ClearTableArgs
  • Response types: DatasetResponse, ListDatasetsResponse, TableListResponse, TableClearResponse
  • Supporting types for dataset columns, owners, and metadata

Implementation Details

Both new API classes follow established patterns:

  • Extend Router base class for consistent HTTP handling
  • Use camelCase method names for JavaScript conventions
  • Implement proper error handling and logging
  • Include JSDoc comments for IDE support
  • Export through central index.ts for clean public API

Migration Example:

// Old (still works, shows deprecation warning)
await client.table.uploadCsv({ table_name: "data", data: csv });

// New (recommended)
await client.uploads.uploadCsv({ table_name: "data", data: csv });
await client.dataset.list({ owner_handle: "dune", limit: 50 });

This adds support for the new /v1/datasets and /v1/uploads endpoints
while deprecating the legacy /v1/table endpoints in favor of the new
uploads API.

Changes:
- Add DatasetAPI class with list() and getBySlug() methods
- Add UploadsAPI class with full CRUD operations including new clear() method
- Deprecate all TableAPI methods with migration guidance
- Add comprehensive type definitions for new APIs
- Update DuneClient to expose new dataset and uploads interfaces
- Maintain full backward compatibility with existing TableAPI
@va3093 va3093 force-pushed the add-datasets-uploads-apis branch 2 times, most recently from d2aaeeb to ab1ad1d Compare November 14, 2025 12:41
Copy link
Collaborator

@bh2smith bh2smith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems great, just some failing tests

log.setLevel("silent", true);

const API_KEY = process.env.DUNE_API_KEY!;
const USER_NAME = process.env.DUNE_USER_NAME || "your_username";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May need to add an env var here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bh2smith do you know what the team handle that owns the API_KEY we use here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its probably my account bh2smith. I can try locally and see.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So locally (using a different API KEY) the error actually said the associated namespaces I could use with that key:

    Response Error: HTTP - Status: 401, Message: {"error":"You are not authorized to create a table under the 'your_username' namespace. Your accessible namespaces: 'bh2smith'"}

This leads me to believe that the API key used in the test here is so old that it may not have a name to link it to?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I discovered that I am unable to call the usage endpoint:

https://api.dune.com/api/v1/usage

{"error":"An internal error occurred"}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I will create a new team and generate a new API key.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I discovered that I am unable to call the usage endpoint:

https://api.dune.com/api/v1/usage

{"error":"An internal error occurred"}

does it still not work for you @bh2smith . I just tried it and it worked for me.

@va3093 va3093 force-pushed the add-datasets-uploads-apis branch 2 times, most recently from 87af975 to cca12a0 Compare November 14, 2025 14:45
Adds comprehensive end-to-end tests for the new dataset and uploads APIs:

DatasetAPI Tests:
- List datasets with filters (requires owner_handle or type filter)
- Get dataset by slug (tested with dex.trades)

UploadsAPI Tests:
- List uploaded tables
- Upload CSV (both public and private)
- Create, insert, clear, and delete operations (skipped by default, require DUNE_USER_NAME)

All tests pass against production API.
@va3093 va3093 force-pushed the add-datasets-uploads-apis branch 2 times, most recently from c0a1748 to e824634 Compare November 14, 2025 15:00
@va3093 va3093 merged commit caf33f3 into main Nov 14, 2025
3 of 4 checks passed
@va3093 va3093 deleted the add-datasets-uploads-apis branch November 14, 2025 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants