Skip to content

[ISSUE] maxResults: 0 is never sent in list requests #1325

@orf

Description

@orf

Description
Databricks endpoints that accept a max_results parameter, such as listing catalogs, have the following advice:

when set to 0, the page length is set to a server configured value (recommended);
If not set, all valid catalogs are returned (not recommended).

However, as far as I can tell, all definitions for the max_results parameter (such as this one) have url:"max_results,omitempty" set on them.

This means that MaxResults: 0 is never sent, even if 0 is explicitly passed, and by default all results are always returned.

Reproduction

package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"

	"github.com/databricks/databricks-sdk-go"
	"github.com/databricks/databricks-sdk-go/listing"
	"github.com/databricks/databricks-sdk-go/service/catalog"
)

type logRoundTripper struct {
	tripper http.RoundTripper
}

func (l *logRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
	fmt.Printf("Request sent: %s\n", req.URL)
	return l.tripper.RoundTrip(req)
}

func main() {
	transport := http.DefaultTransport.(*http.Transport).Clone()
	c, err := databricks.NewWorkspaceClient(&databricks.Config{
		Host: "https://xxx.cloud.databricks.com/",
		HTTPTransport: &logRoundTripper{
			tripper: transport,
		},
	})
	if err != nil {
		panic(err)
	}
	ctx := context.Background()
	iter := c.Tables.List(ctx, catalog.ListTablesRequest{
		CatalogName: "system",
		SchemaName:  "information_schema",
		MaxResults:  0,
	})
	for {
		table, err := iter.Next(ctx)
		if err != nil {
			if errors.Is(err, listing.ErrNoMoreItems) {
				break
			}
			panic(err)
		}
		println(table.Name)
	}
}

Expected behavior

I would expect to see a request with max_results=0 in the query. Instead, I see this:

https://xxx.cloud.databricks.com/api/2.1/unity-catalog/tables?catalog_name=system&schema_name=information_schema

Passing ForceSendFields: []string{"max_results", "MaxResults"}, does not work either.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions