-
Notifications
You must be signed in to change notification settings - Fork 16
Add Datasets and Uploads APIs, deprecate Tables API #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This adds support for the new /v1/datasets and /v1/uploads endpoints while deprecating the legacy /v1/table endpoints in favor of the new uploads API. Changes: - Add DatasetAPI class with list() and getBySlug() methods - Add UploadsAPI class with full CRUD operations including new clear() method - Deprecate all TableAPI methods with migration guidance - Add comprehensive type definitions for new APIs - Update DuneClient to expose new dataset and uploads interfaces - Maintain full backward compatibility with existing TableAPI
d2aaeeb to
ab1ad1d
Compare
bh2smith
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems great, just some failing tests
tests/e2e/uploads.spec.ts
Outdated
| log.setLevel("silent", true); | ||
|
|
||
| const API_KEY = process.env.DUNE_API_KEY!; | ||
| const USER_NAME = process.env.DUNE_USER_NAME || "your_username"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May need to add an env var here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bh2smith do you know what the team handle that owns the API_KEY we use here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its probably my account bh2smith. I can try locally and see.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So locally (using a different API KEY) the error actually said the associated namespaces I could use with that key:
Response Error: HTTP - Status: 401, Message: {"error":"You are not authorized to create a table under the 'your_username' namespace. Your accessible namespaces: 'bh2smith'"}
This leads me to believe that the API key used in the test here is so old that it may not have a name to link it to?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I discovered that I am unable to call the usage endpoint:
https://api.dune.com/api/v1/usage
{"error":"An internal error occurred"}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. I will create a new team and generate a new API key.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I discovered that I am unable to call the usage endpoint:
https://api.dune.com/api/v1/usage
{"error":"An internal error occurred"}
does it still not work for you @bh2smith . I just tried it and it worked for me.
87af975 to
cca12a0
Compare
Adds comprehensive end-to-end tests for the new dataset and uploads APIs: DatasetAPI Tests: - List datasets with filters (requires owner_handle or type filter) - Get dataset by slug (tested with dex.trades) UploadsAPI Tests: - List uploaded tables - Upload CSV (both public and private) - Create, insert, clear, and delete operations (skipped by default, require DUNE_USER_NAME) All tests pass against production API.
c0a1748 to
e824634
Compare
Context
The Dune API has introduced new
/v1/datasetsand/v1/uploadsendpoints to replace the legacy/v1/tableendpoints. This update aligns the TypeScript client with the latest API specification and provides users with access to new functionality while maintaining backward compatibility.Currently, the client only supports the legacy
/v1/tableendpoints which are being phased out in favor of the more comprehensive and better-organized/v1/uploadsendpoints. Additionally, there was no support for the new/v1/datasetsendpoints which provide read access to dataset information.What Changed
New APIs Added
DatasetAPI (
client.dataset):list(args?)- List datasets with filtering by owner and typegetBySlug(slug)- Get detailed dataset information by slugUploadsAPI (
client.uploads):list(args?)- List all uploaded tables for the accountcreate(args)- Create an empty table with defined schemauploadCsv(args)- Upload CSV data as a new tableinsert(args)- Insert data into an existing table (CSV/JSON)delete(args)- Delete an uploaded tableclear(args)- New! Clear all data from a table while preserving schemaDeprecation Changes
All four methods in TableAPI are now deprecated with console warnings:
table.uploadCsv()→ Useuploads.uploadCsv()table.create()→ Useuploads.create()table.insert()→ Useuploads.insert()table.delete()→ Useuploads.delete()The deprecated methods continue to work exactly as before, ensuring full backward compatibility. Users will see helpful deprecation warnings guiding them to the new API.
Type Definitions
Added comprehensive type definitions matching the OpenAPI specification:
ListDatasetsArgs,ListUploadsArgs,ClearTableArgsDatasetResponse,ListDatasetsResponse,TableListResponse,TableClearResponseImplementation Details
Both new API classes follow established patterns:
Routerbase class for consistent HTTP handlingindex.tsfor clean public APIMigration Example: