Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .ci/integration.cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,34 @@ steps:
exit 0
fi

- id: "cloud-storage"
name: golang:1
waitFor: ["compile-test-binary", "detect-changes"]
entrypoint: /bin/bash
env:
- "GOPATH=/gopath"
- "CLOUD_STORAGE_PROJECT=$PROJECT_ID"
- "SERVICE_ACCOUNT_EMAIL=$SERVICE_ACCOUNT_EMAIL"
secretEnv: ["CLIENT_ID"]
volumes:
- name: "go"
path: "/gopath"
args:
- -c
- |
PATTERN="cloudstorage|internal/server/|.ci/"

if grep -qE "$$PATTERN" /workspace/changed_files.txt; then
echo "Relevant changes detected. Running Cloud Storage tests..."
.ci/test_with_coverage.sh \
"Cloud Storage" \
cloudstorage \
cloudstorage
else
echo "No relevant changes for Cloud Storage. Skipping shard."
exit 0
fi

- id: "postgres"
name: golang:1
waitFor: ["compile-test-binary", "detect-changes"]
Expand Down
3 changes: 3 additions & 0 deletions cmd/internal/imports.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ import (
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistdefinespec"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistgeneratequery"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistmodifyspec"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragelistobjects"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragereadobject"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdbexecutesql"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdblistschemas"
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdblisttables"
Expand Down Expand Up @@ -261,6 +263,7 @@ import (
_ "github.com/googleapis/mcp-toolbox/internal/sources/cloudsqlmssql"
_ "github.com/googleapis/mcp-toolbox/internal/sources/cloudsqlmysql"
_ "github.com/googleapis/mcp-toolbox/internal/sources/cloudsqlpg"
_ "github.com/googleapis/mcp-toolbox/internal/sources/cloudstorage"
_ "github.com/googleapis/mcp-toolbox/internal/sources/cockroachdb"
_ "github.com/googleapis/mcp-toolbox/internal/sources/couchbase"
_ "github.com/googleapis/mcp-toolbox/internal/sources/dataplex"
Expand Down
4 changes: 4 additions & 0 deletions docs/en/integrations/cloud-storage/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: "Cloud Storage"
weight: 1
---
69 changes: 69 additions & 0 deletions docs/en/integrations/cloud-storage/source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: "Cloud Storage Source"
linkTitle: "Source"
type: docs
weight: 1
description: >
Cloud Storage is Google Cloud's managed service for storing unstructured objects (files) in buckets. Toolbox connects at the project level, allowing tools to list, read, and manage objects across any bucket the credentials can access.
no_list: true
---

## About

[Cloud Storage][gcs-docs] is Google Cloud's managed service for storing
unstructured data (blobs) in containers called *buckets*. Buckets live in a GCP
project; objects are addressed by `gs://<bucket>/<object>`.

If you are new to Cloud Storage, you can try the
[quickstart][gcs-quickstart] to create a bucket and upload your first objects.

The Cloud Storage source is configured at the **project** level. Individual
tools take a `bucket` parameter, so a single configured source can operate
against any bucket the underlying credentials are authorized for.

[gcs-docs]: https://cloud.google.com/storage/docs
[gcs-quickstart]: https://cloud.google.com/storage/docs/discover-object-storage-console

## Available Tools

{{< list-tools >}}

## Requirements

### IAM Permissions

Cloud Storage uses [Identity and Access Management (IAM)][iam-overview] to
control access to buckets and objects. Toolbox uses your
[Application Default Credentials (ADC)][adc] to authorize and authenticate when
interacting with Cloud Storage.

In addition to [setting the ADC for your server][set-adc], ensure the IAM
identity has the appropriate role for the tools being exposed. Common roles:

- `roles/storage.objectViewer` — read-only access to objects (sufficient for
`cloud-storage-list-objects` and `cloud-storage-read-object`)
- `roles/storage.objectUser` — read and write access to objects
- `roles/storage.admin` — full control, including bucket management

See [Cloud Storage IAM roles][gcs-iam] for the full list.

[iam-overview]: https://cloud.google.com/storage/docs/access-control/iam
[adc]: https://cloud.google.com/docs/authentication#adc
[set-adc]: https://cloud.google.com/docs/authentication/provide-credentials-adc
[gcs-iam]: https://cloud.google.com/storage/docs/access-control/iam-roles

## Example

```yaml
kind: source
name: my-gcs-source
type: "cloud-storage"
project: "my-project-id"
```

## Reference

| **field** | **type** | **required** | **description** |
|-----------|:--------:|:------------:|---------------------------------------------------------------------------------|
| type | string | true | Must be "cloud-storage". |
| project | string | true | Id of the GCP project the configured source is associated with (e.g. "my-project-id"). |
4 changes: 4 additions & 0 deletions docs/en/integrations/cloud-storage/tools/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: "Tools"
weight: 2
---
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: "cloud-storage-list-objects"
Comment thread
huangjiahua marked this conversation as resolved.
type: docs
weight: 1
description: >
A "cloud-storage-list-objects" tool lists objects in a Cloud Storage bucket, with optional prefix filtering and delimiter-based grouping.
---

## About

A `cloud-storage-list-objects` tool returns the objects in a
[Cloud Storage bucket][gcs-buckets]. It supports the usual GCS listing options:

- `prefix` — filter results to objects whose names begin with the given string.
- `delimiter` — group results by this character (typically `/`) so subdirectory-like
"common prefixes" are returned separately from the leaf objects.
- `max_results` / `page_token` — paginate through large listings.

The response is a JSON object with `objects` (the full object metadata as
returned by the Cloud Storage API — fields such as `Name`, `Size`, `ContentType`,
`Updated`, `StorageClass`, `MD5`, etc.), `prefixes` (the common prefixes when
`delimiter` is set), and `nextPageToken` (empty when there are no more pages).

[gcs-buckets]: https://cloud.google.com/storage/docs/buckets

## Compatible Sources

{{< compatible-sources >}}

## Parameters

| **parameter** | **type** | **required** | **description** |
|---------------|:--------:|:------------:|-------------------------------------------------------------------------------------------------------------------|
| bucket | string | true | Name of the Cloud Storage bucket to list objects from. |
| prefix | string | false | Filter results to objects whose names begin with this prefix. |
| delimiter | string | false | Delimiter used to group object names (typically '/'). When set, common prefixes are returned as `prefixes`. |
| max_results | integer | false | Maximum number of objects to return per page. A value of 0 uses the API default (1000); the maximum allowed is 1000. |
| page_token | string | false | A previously-returned page token for retrieving the next page of results. |

## Example

```yaml
kind: tool
name: list_objects
type: cloud-storage-list-objects
source: my-gcs-source
description: Use this tool to list objects in a Cloud Storage bucket.
```

## Reference

| **field** | **type** | **required** | **description** |
|-------------|:--------:|:------------:|---------------------------------------------------------|
| type | string | true | Must be "cloud-storage-list-objects". |
| source | string | true | Name of the Cloud Storage source to list objects from. |
| description | string | true | Description of the tool that is passed to the LLM. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: "cloud-storage-read-object"
Comment thread
huangjiahua marked this conversation as resolved.
type: docs
weight: 2
description: >
A "cloud-storage-read-object" tool reads the UTF-8 text content of a Cloud Storage object, optionally constrained to a byte range.
---

## About

A `cloud-storage-read-object` tool fetches the bytes of a single
[Cloud Storage object][gcs-objects] and returns them as plain UTF-8 text.

Only text objects are supported today: if the object bytes (or the requested
range) are not valid UTF-8 the tool returns an agent-fixable error. This is
because the MCP tool-result channel currently only carries text; binary
payloads will be supported once MCP can carry embedded resources.

Reads are capped at **8 MiB** per call to protect the server's memory and keep
LLM contexts manageable; objects or ranges larger than that are rejected with
an agent-fixable error. Use the optional `range` parameter to read a slice of
a larger object.

This tool is intended for small-to-medium textual content an LLM can process
directly. For bulk downloads of large files to the local filesystem, use
`cloud-storage-download-object` (coming in a follow-up release).

[gcs-objects]: https://cloud.google.com/storage/docs/objects

## Compatible Sources

{{< compatible-sources >}}

## Parameters

| **parameter** | **type** | **required** | **description** |
|---------------|:--------:|:------------:|---------------------------------------------------------------------------------------------------------------------------------------------------|
| bucket | string | true | Name of the Cloud Storage bucket containing the object. |
| object | string | true | Full object name (path) within the bucket, e.g. `path/to/file.txt`. |
| range | string | false | Optional HTTP byte range, e.g. `bytes=0-999` (first 1000 bytes), `bytes=-500` (last 500 bytes), or `bytes=500-` (from byte 500 to end). Empty reads the full object. |

## Example

```yaml
kind: tool
name: read_object
type: cloud-storage-read-object
source: my-gcs-source
description: Use this tool to read the content of a Cloud Storage object.
```

## Reference

| **field** | **type** | **required** | **description** |
|-------------|:--------:|:------------:|---------------------------------------------------------|
| type | string | true | Must be "cloud-storage-read-object". |
| source | string | true | Name of the Cloud Storage source to read the object from. |
| description | string | true | Description of the tool that is passed to the LLM. |
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
cloud.google.com/go/logging v1.16.0
cloud.google.com/go/longrunning v0.11.0
cloud.google.com/go/spanner v1.89.0
cloud.google.com/go/storage v1.62.0
github.com/ClickHouse/clickhouse-go/v2 v2.45.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.56.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.32.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,8 @@ go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 h1:88Y4s2C8oTui1LGM6bT
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRlrHf/hfPJPvVag77kKyvrLeD10kpMl+A=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 h1:3iZJKlCZufyRzPzlQhUIWVmfltrXuGyfjREgGP3UUjc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0/go.mod h1:/G+nUPfhq2e+qiXMGxMwumDrP5jtzU+mWN7/sjT2rak=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.42.0 h1:lSZHgNHfbmQTPfuTmWVkEu8J8qXaQwuV30pjCcAUvP8=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.42.0/go.mod h1:so9ounLcuoRDu033MW/E0AD4hhUjVqswrMF5FoZlBcw=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
Expand Down
Loading
Loading