-
Notifications
You must be signed in to change notification settings - Fork 54
Open
Description
Description
Databricks endpoints that accept a max_results parameter, such as listing catalogs, have the following advice:
when set to 0, the page length is set to a server configured value (recommended);
If not set, all valid catalogs are returned (not recommended).
However, as far as I can tell, all definitions for the max_results parameter (such as this one) have url:"max_results,omitempty" set on them.
This means that MaxResults: 0 is never sent, even if 0 is explicitly passed, and by default all results are always returned.
Reproduction
package main
import (
"context"
"errors"
"fmt"
"net/http"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/listing"
"github.com/databricks/databricks-sdk-go/service/catalog"
)
type logRoundTripper struct {
tripper http.RoundTripper
}
func (l *logRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
fmt.Printf("Request sent: %s\n", req.URL)
return l.tripper.RoundTrip(req)
}
func main() {
transport := http.DefaultTransport.(*http.Transport).Clone()
c, err := databricks.NewWorkspaceClient(&databricks.Config{
Host: "https://xxx.cloud.databricks.com/",
HTTPTransport: &logRoundTripper{
tripper: transport,
},
})
if err != nil {
panic(err)
}
ctx := context.Background()
iter := c.Tables.List(ctx, catalog.ListTablesRequest{
CatalogName: "system",
SchemaName: "information_schema",
MaxResults: 0,
})
for {
table, err := iter.Next(ctx)
if err != nil {
if errors.Is(err, listing.ErrNoMoreItems) {
break
}
panic(err)
}
println(table.Name)
}
}Expected behavior
I would expect to see a request with max_results=0 in the query. Instead, I see this:
https://xxx.cloud.databricks.com/api/2.1/unity-catalog/tables?catalog_name=system&schema_name=information_schema
Passing ForceSendFields: []string{"max_results", "MaxResults"}, does not work either.
Metadata
Metadata
Assignees
Labels
No labels