Skip to content

Commit 53c4647

Browse files
Implement 2025-10 Features: DRN, BYOC, Index Schema / Metadata Update & Filter, CreateNamespace (#125)
## Problem There are a number of new features which are available for the `2025-10` API release, and the SDK needs to be updated to support them. Features: - BYOC - Dedicated Read Nodes - Index Schema - Fetch and Update vectors by metadata filter - Create Namespace ## Solution Implement `2025-10` features. ### BYOC (Bring Your Own Cloud) - `BYOCSpec` has been added to `models.go`, and `IndexSpec` has been updated to include this as a possible union type. Code has been updated to deserialize the proper specification when receiving results from operations. - `CreateBYOCIndex` method has been added to `Client`, and `CreateBYOCIndexRequest` type has been added to `client.go`. - `PrivateHost` is already available on `IndexModel`. ### DRN (Dedicated Read Nodes) - `ReadCapacityParams` and `ReadCapacityDedicatedConfig` types added to `client.go`. These can now be used in `CreateServerlessIndexRequest`, `CreateIndexForModelRequest`, and `ConfigureIndexParams` for creating and configuring indexes for DRN. - `ReadCapacity`, `ReadCapacityOnDemand`, `ReadCapacityDedicated`, `ReadCapacityScaling`, `ReadCapacityManualScaling`, and `ReadCapacityStatus` have been added to `models.go`. These are used when deserializing the `ReadCapacity` on `ServerlessSpec`. `ReadCapacityScaling` is also used in `ReadCapacityDedicatedConfig`. ### Index Schema - `MetadataSchema` and `MetadataSchemaField` have been added to `models.go`. `MetadataSchema` has been added to `CreateServerlessIndexRequest`, `CreateIndexForModelRequest, `CreateBYOCIndexRequest`, `ServerlessSpec`, and `Backup`. - The `Schema` can be applied to an index when creating a new index, or for individual namespaces when calling the new `IndexConnection.CreateNamespace` method. ### Fetch and Update vectors by metadata - `FetchVectorsByMetadata` has been added to `IndexConnection`, and `FetchVectorsByMetadataRequest` and `FetchVectorsByMetadataResponse` have been added to `index_connection.go`. - `UpdateVectorsByMetadata` has been added to `IndexConnection`, and `UpdateVectorsByMetadataRequest` and `UpdateVectorsByMetadataResponse` have been added to `index_connection.go`. ### Namespaces - `CreateNamespace` has been added to `IndexConnection`, and `CreateNamespaceParams` has been added to `index_connection.go`. - `Prefix` has been added to `ListNamespacesParams`, and `TotalCount` has been added to `ListNamespacesResponse`. ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) ## Test Plan New unit tests for utility functions added in `client_test.go` and `index_connection_test.go`. Integration tests have been updated and added to cover new functionality. There should ultimately be no breaking changes to the existing interface, so integrations should run as expected. You'll want to test the new functionality around creating and configuring and index schema, updating and filtering with metadata, and creating a new namespace. You can find code snippets in the integration tests and doc comments. Here's some example code: ### Create serverless index with dedicated ReadCapacity and Schema: ```go ctx := context.Background() clientParams := pinecone.NewClientParams{ ApiKey: "YOUR_API_KEY", } pc, err := pinecone.NewClient(clientParams) if err != nil { panic(fmt.Errorf("Failed to create Client: %v", err)) } dimension := int32(128) metric := pinecone.Cosine // Define a schema for metadata indexing schema := &pinecone.MetadataSchema{ Fields: map[string]pinecone.MetadataSchemaField{ "category": {Filterable: true}, "rating": {Filterable: true}, "year": {Filterable: true}, }, } // Configure dedicated read capacity readCapacity := &pinecone.ReadCapacityParams{ Dedicated: &pinecone.ReadCapacityDedicatedConfig{ NodeType: "t1", Scaling: &pinecone.ReadCapacityScaling{ Manual: &pinecone.ReadCapacityManualScaling{ Replicas: 1, Shards: 1, }, }, }, } // Create the serverless index idx, err := pc.CreateServerlessIndex(ctx, &pinecone.CreateServerlessIndexRequest{ Name: "my-serverless-index", Dimension: &dimension, Metric: &metric, Cloud: pinecone.Aws, Region: "us-east-1", ReadCapacity: readCapacity, Schema: schema, }) if err != nil { log.Fatalf("Failed to create serverless index: %v", err) } fmt.Printf("Successfully created index: %s\n", idx.Name) ``` ### Configuring an index to update ReadCapacity ```go ctx := context.Background() // Update the read capacity configuration updatedReadCapacity := &pinecone.ReadCapacityParams{ Dedicated: &pinecone.ReadCapacityDedicatedConfig{ NodeType: "t1", Scaling: &pinecone.ReadCapacityScaling{ Manual: &pinecone.ReadCapacityManualScaling{ Replicas: 2, // Scale up replicas Shards: 2, // Scale up shards }, }, }, } // Configure the index with updated read capacity updatedIdx, err := pc.ConfigureIndex(ctx, "my-serverless-index", pinecone.ConfigureIndexParams{ ReadCapacity: updatedReadCapacity, }) if err != nil { log.Fatalf("Failed to configure index: %v", err) } fmt.Printf("Successfully updated index read capacity. Status: %s\n", updatedIdx.Spec.Serverless.ReadCapacity.Dedicated.Status.State) ``` ### Create a new namespace with a different schema ```go ctx := context.Background() // Get the index connection idx, err := pc.DescribeIndex(ctx, "my-serverless-index") if err != nil { log.Fatalf("Failed to describe index: %v", err) } idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host}) if err != nil { log.Fatalf("Failed to create IndexConnection: %v", err) } // Define a slightly different schema for the new namespace // (e.g., adding a new field, removing a field, or keeping some fields) namespaceSchema := &pinecone.MetadataSchema{ Fields: map[string]pinecone.MetadataSchemaField{ "category": {Filterable: true}, "rating": {Filterable: true}, "tags": {Filterable: true}, // New field not in index schema // Note: "year" is not included in this namespace schema }, } // Create the namespace with the custom schema namespace, err := idxConnection.CreateNamespace(ctx, &pinecone.CreateNamespaceParams{ Name: "my-custom-namespace", Schema: namespaceSchema, }) if err != nil { log.Fatalf("Failed to create namespace: %v", err) } fmt.Printf("Successfully created namespace: %s with %d records\n", namespace.Name, namespace.RecordCount) ``` ### Update / Fetch vectors by metadata ```go ctx := context.Background() idxConnectionWithNamespace, err := pc.Index(pinecone.NewIndexConnParams{ Host: idx.Host, Namespace: "my-custom-namespace", }) if err != nil { log.Fatalf("Failed to create IndexConnection with namespace: %v", err) } vectors := []*pinecone.Vector{ { Id: "vector-1", Values: &[]float32{0.1, 0.2, 0.3, 0.4}, // Example values Metadata: func() *pinecone.Metadata { m, err := pinecone.NewMetadata(map[string]interface{}{ "category": "electronics", "rating": 4.5, "tags": []string{"popular", "trending"}, }) if err != nil { panic(fmt.Errorf("error creating MetadataSchema: %w", err)) } return m }(), }, { Id: "vector-2", Values: &[]float32{0.2, 0.3, 0.4, 0.5}, Metadata: func() *pinecone.Metadata { m, _ := pinecone.NewMetadata(map[string]interface{}{ "category": "electronics", "rating": 3.8, "tags": []string{"new"}, }) if err != nil { panic(fmt.Errorf("error creating MetadataSchema: %w", err)) } }(), }, } _, err = idxConnectionWithNamespace.Upsert(ctx, vectors, nil) if err != nil { log.Fatalf("Failed to upsert vectors: %v", err) } // Update vectors by metadata filter // Update all vectors with category "electronics" and rating >= 4.0 filterMap := map[string]interface{}{ "$and": []interface{}{ map[string]interface{}{ "category": map[string]interface{}{"$eq": "electronics"}, }, map[string]interface{}{ "rating": map[string]interface{}{"$gte": 4.0}, }, }, } filter, err := pinecone.NewMetadataFilter(filterMap) if err != nil { log.Fatalf("Failed to create metadata filter: %v", err) } // Update metadata for matching vectors updateMetadata, err := pinecone.NewMetadata(map[string]interface{}{ "category": "electronics", "rating": 5.0, // Update rating to 5.0 "tags": []string{"popular", "trending", "featured"}, // Add new tag }) if err != nil { log.Fatalf("Failed to create update metadata: %v", err) } updateRes, err := idxConnectionWithNamespace.UpdateVectorsByMetadata(ctx, &pinecone.UpdateVectorsByMetadataRequest{ Filter: filter, Metadata: updateMetadata, }) if err != nil { log.Fatalf("Failed to update vectors by metadata: %v", err) } fmt.Printf("Updated %d vector(s) matching the filter\n", updateRes.MatchedRecords) // Fetch vectors by metadata filter // Fetch all vectors with category "electronics" and rating >= 4.5 fetchFilterMap := map[string]interface{}{ "$and": []interface{}{ map[string]interface{}{ "category": map[string]interface{}{"$eq": "electronics"}, }, map[string]interface{}{ "rating": map[string]interface{}{"$gte": 4.5}, }, }, } fetchFilter, err := pinecone.NewMetadataFilter(fetchFilterMap) if err != nil { log.Fatalf("Failed to create fetch filter: %v", err) } limit := uint32(10) fetchRes, err := idxConnectionWithNamespace.FetchVectorsByMetadata(ctx, &pinecone.FetchVectorsByMetadataRequest{ Filter: fetchFilter, Limit: &limit, }) if err != nil { log.Fatalf("Failed to fetch vectors by metadata: %v", err) } fmt.Printf("Fetched %d vector(s) matching the filter:\n", len(fetchRes.Vectors)) for id, vector := range fetchRes.Vectors { fmt.Printf(" Vector ID: %s, Metadata: %v\n", id, vector.Metadata) } ``` --- - To see the specific tasks where the Asana app for GitHub is being used, see below: - https://app.asana.com/0/0/1211666868889403 - https://app.asana.com/0/0/1211666868889406 - https://app.asana.com/0/0/1211666868889400
1 parent 5d64bbd commit 53c4647

File tree

10 files changed

+2817
-792
lines changed

10 files changed

+2817
-792
lines changed

codegen/apis

Submodule apis updated from 158e273 to d5ac931

internal/gen/db_data/rest/db_data_2025-10.oas.go

Lines changed: 8 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

justfile

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,43 @@
11
api_version := "2025-10"
22

3+
default:
4+
@just --list
5+
6+
build:
7+
go build -v ./...
8+
go vet ./...
9+
10+
build-clean:
11+
go build -a -v ./...
12+
go vet ./...
13+
14+
build-pinecone:
15+
go build -v ./pinecone
16+
go vet ./pinecone
17+
318
test:
419
#!/usr/bin/env bash
520
set -o allexport
621
source .env
722
set +o allexport
823
go test -count=1 -v ./pinecone
24+
925
test-unit:
1026
#!/usr/bin/env bash
1127
set -o allexport
1228
source .env
1329
set +o allexport
1430
go test -v -run Unit ./pinecone
31+
1532
bootstrap:
1633
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.32
1734
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3
1835
go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@v2.3.0
1936
go install golang.org/x/tools/cmd/godoc@latest
37+
2038
gen:
2139
./codegen/build-clients.sh {{api_version}}
40+
2241
docs:
2342
@echo "Serving docs at http://localhost:6060/pkg/github.com/pinecone-io/go-pinecone/v4/pinecone/"
2443
@godoc -http=:6060 >/dev/null

0 commit comments

Comments
 (0)