Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/commands.generated.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,7 @@ Generated from `gog schema --json`.
- [`gog gmail (mail,email) settings vacation get (info,show)`](commands/gog-gmail-settings-vacation-get.md) - Get current vacation responder settings
- [`gog gmail (mail,email) settings vacation update (edit,set) [flags]`](commands/gog-gmail-settings-vacation-update.md) - Update vacation responder settings
- [`gog gmail (mail,email) settings watch <command>`](commands/gog-gmail-settings-watch.md) - Manage Gmail watch
- [`gog gmail (mail,email) settings watch pull [flags]`](commands/gog-gmail-settings-watch-pull.md) - Run Pub/Sub pull consumer
- [`gog gmail (mail,email) settings watch renew (update) [flags]`](commands/gog-gmail-settings-watch-renew.md) - Renew Gmail watch using stored config
- [`gog gmail (mail,email) settings watch serve [flags]`](commands/gog-gmail-settings-watch-serve.md) - Run Pub/Sub push handler
- [`gog gmail (mail,email) settings watch start (begin) [flags]`](commands/gog-gmail-settings-watch-start.md) - Start Gmail watch for Pub/Sub
Expand Down
3 changes: 2 additions & 1 deletion docs/commands/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Every `gog` command has a generated docs page. The source of truth is the live CLI schema; run `make docs-commands` after changing command names, flags, help text, aliases, or arguments.

Generated pages: 585.
Generated pages: 586.

## Top-level Commands

Expand Down Expand Up @@ -449,6 +449,7 @@ Generated pages: 585.
- [gog gmail settings vacation get](gog-gmail-settings-vacation-get.md) - Get current vacation responder settings
- [gog gmail settings vacation update](gog-gmail-settings-vacation-update.md) - Update vacation responder settings
- [gog gmail settings watch](gog-gmail-settings-watch.md) - Manage Gmail watch
- [gog gmail settings watch pull](gog-gmail-settings-watch-pull.md) - Run Pub/Sub pull consumer
- [gog gmail settings watch renew](gog-gmail-settings-watch-renew.md) - Renew Gmail watch using stored config
- [gog gmail settings watch serve](gog-gmail-settings-watch-serve.md) - Run Pub/Sub push handler
- [gog gmail settings watch start](gog-gmail-settings-watch-start.md) - Start Gmail watch for Pub/Sub
Expand Down
56 changes: 56 additions & 0 deletions docs/commands/gog-gmail-settings-watch-pull.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# `gog gmail settings watch pull`

> Generated from `gog schema --json`. Do not edit this page by hand; run `make docs-commands`.

Run Pub/Sub pull consumer

## Usage

```bash
gog gmail (mail,email) settings watch pull [flags]
```

## Parent

- [gog gmail settings watch](gog-gmail-settings-watch.md)

## Flags

| Flag | Type | Default | Help |
| --- | --- | --- | --- |
| `--access-token` | `string` | | Use provided access token directly (bypasses stored refresh tokens; token expires in ~1h) |
| `-a`<br>`--account`<br>`--acct` | `string` | | Account email for API commands (gmail/calendar/chat/classroom/drive/drivelabels/docs/slides/contacts/tasks/people/sheets/forms/sites/appscript/analytics/searchconsole/youtube/photos) |
| `--client` | `string` | | OAuth client name (selects stored credentials + token bucket) |
| `--color` | `string` | auto | Color output: auto\|always\|never |
| `--disable-commands` | `string` | | Comma-separated list of disabled commands; dot paths allowed |
| `-n`<br>`--dry-run`<br>`--dryrun`<br>`--noop`<br>`--preview` | `bool` | | Do not make changes; print intended actions and exit successfully |
| `--enable-commands` | `string` | | Comma-separated list of enabled command prefixes; dot paths allowed (restricts CLI) |
| `--enable-commands-exact` | `string` | | Comma-separated list of exact enabled commands; dot paths allowed and parent commands do not enable children |
| `--exclude-labels` | `string` | SPAM,TRASH | List of Gmail label IDs to exclude from hook payload (e.g. SPAM,TRASH,Label_123). Set to empty string to disable. |
| `--fetch-delay` | `string` | 3s | Delay before fetching Gmail history (seconds or duration) |
| `-y`<br>`--force`<br>`--assume-yes`<br>`--yes` | `bool` | | Skip confirmations for destructive commands |
| `--gmail-no-send` | `bool` | false | Block Gmail send operations (agent safety) |
| `-h`<br>`--help` | `kong.helpFlag` | | Show context-sensitive help. |
| `--history-types` | `[]string` | | History types to include (repeatable, comma-separated: messageAdded,messageDeleted,labelAdded,labelRemoved). Default: messageAdded |
| `--home` | `string` | | Override gogcli config/data/state/cache root (equivalent to GOG_HOME) |
| `--hook-token` | `string` | | Webhook bearer token |
| `--hook-url` | `string` | | Webhook URL to forward messages |
| `--include-body` | `bool` | | Include text/plain body in hook payload |
| `-j`<br>`--json`<br>`--machine` | `bool` | false | Output JSON to stdout (best for scripting) |
| `--local` | `bool` | | Use local timezone (default behavior, useful to override --timezone) |
| `--max-bytes` | `int` | 20000 | Max bytes of body to include |
| `--no-input`<br>`--non-interactive`<br>`--noninteractive` | `bool` | | Never prompt; fail instead (useful for CI) |
| `-p`<br>`--plain`<br>`--tsv` | `bool` | false | Output stable, parseable text to stdout (TSV; no colors) |
| `--results-only` | `bool` | | In JSON mode, emit only the primary result (drops envelope fields like nextPageToken) |
| `--save-hook` | `bool` | | Persist hook settings to watch state |
| `--select`<br>`--pick`<br>`--project` | `string` | | In JSON mode, select comma-separated fields (best-effort; supports dot paths). Desire path: use --fields for most commands. |
| `--subscription` | `string` | | Pub/Sub pull subscription (projects/.../subscriptions/...) |
| `-z`<br>`--timezone` | `string` | | Output timezone (IANA name, e.g. America/New_York, UTC). Default: local |
| `-v`<br>`--verbose` | `bool` | | Enable verbose logging |
| `--version` | `kong.VersionFlag` | | Print version and exit |
| `--wrap-untrusted` | `bool` | false | In JSON/raw output, wrap fetched text fields in external untrusted-content markers |

## See Also

- [gog gmail settings watch](gog-gmail-settings-watch.md)
- [Command index](README.md)
1 change: 1 addition & 0 deletions docs/commands/gog-gmail-settings-watch.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ gog gmail (mail,email) settings watch <command>

## Subcommands

- [gog gmail settings watch pull](gog-gmail-settings-watch-pull.md) - Run Pub/Sub pull consumer
- [gog gmail settings watch renew](gog-gmail-settings-watch-renew.md) - Renew Gmail watch using stored config
- [gog gmail settings watch serve](gog-gmail-settings-watch-serve.md) - Run Pub/Sub push handler
- [gog gmail settings watch start](gog-gmail-settings-watch-start.md) - Start Gmail watch for Pub/Sub
Expand Down
3 changes: 2 additions & 1 deletion docs/gmail-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,14 +66,15 @@ For account-specific send blocking, use the no-send config commands:
- [`gog config no-send list`](commands/gog-config-no-send-list.md)
- [`gog config no-send remove`](commands/gog-config-no-send-remove.md)

## Watches and Push
## Watches and Pub/Sub

Gmail watch/PubSub workflows are documented in [Gmail watch](watch.md).

Key command pages:

- [`gog gmail watch start`](commands/gog-gmail-settings-watch-start.md)
- [`gog gmail watch serve`](commands/gog-gmail-settings-watch-serve.md)
- [`gog gmail watch pull`](commands/gog-gmail-settings-watch-pull.md)
- [`gog gmail watch renew`](commands/gog-gmail-settings-watch-renew.md)
- [`gog gmail history`](commands/gog-gmail-history.md)

Expand Down
107 changes: 94 additions & 13 deletions docs/watch.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,59 @@
---
summary: "Gmail watch + Pub/Sub push in gog"
summary: "Gmail watch + Pub/Sub delivery in gog"
read_when:
- Adding Gmail watch/push support
- Adding Gmail watch/Pub/Sub support
- Wiring Gmail to downstream webhooks
---

# Gmail watch

Goal: Gmail push → Pub/Sub → `gog` HTTP handler → downstream webhook.
Goal: Gmail publishes mailbox notifications to Pub/Sub, then `gog` turns those
notifications into downstream webhook payloads.

Two delivery modes are supported:

- Pull: `gog gmail watch pull` reads a Pub/Sub subscription from the local
machine. This is the preferred local-agent shape because Google does not need
an inbound HTTP route to the machine running `gog`.
- Push: `gog gmail watch serve` exposes an HTTP handler for Pub/Sub push. Use it
when you intentionally operate a reachable HTTPS endpoint. Push and pull share
the same downstream hook delivery policy.

## Quick start

1) Create a Pub/Sub topic (GCP project).
2) Create a push subscription targeting your `gog gmail watch serve` endpoint.
3) Configure push auth:
2) Create a pull subscription for the topic.
3) Start watch:

```
gog gmail watch start \
--topic projects/<project>/topics/<topic> \
--label INBOX
```

4) Run pull consumer:

```
gog gmail watch pull \
--subscription projects/<project>/subscriptions/<subscription> \
--hook-url http://127.0.0.1:18789/hooks/agent
```

For push delivery instead:

1) Create a push subscription targeting your `gog gmail watch serve` endpoint.
2) Configure push auth:
- Preferred: OIDC JWT from a service account.
- Fallback/dev: shared token header `x-gog-token` or `?token=`.
4) Start watch:
3) Start watch:

```
gog gmail watch start \
--topic projects/<project>/topics/<topic> \
--label INBOX
```

5) Run handler:
4) Run handler:

```
gog gmail watch serve \
Expand Down Expand Up @@ -52,19 +81,35 @@ gog gmail watch serve \
[--include-body] [--max-bytes <n>] [--exclude-labels <id,id,...>] \
[--history-types <type>...] [--save-hook]

gog gmail watch pull \
--subscription projects/<project>/subscriptions/<subscription> \
[--hook-url <url>] [--hook-token <token>] \
[--fetch-delay <sec|duration>] \
[--include-body] [--max-bytes <n>] [--exclude-labels <id,id,...>] \
[--history-types <type>...] [--save-hook]

gog gmail history --since <historyId> [--max <n>] [--page <token>]
```

Notes:
- `watch start` stores `{historyId, expirationMs, topic, labels}` for account.
- `watch renew` reuses stored topic/labels.
- `watch stop` calls Gmail stop + clears state.
- `watch serve` uses stored hook if `--hook-url` not provided.
- `watch serve --exclude-labels` defaults to `SPAM,TRASH`; set to an empty string to disable.
- `watch serve` and `watch pull` use stored hook config if `--hook-url` is not
provided.
- `watch pull` needs Google credentials that can consume the Pub/Sub
subscription.
- `watch serve` needs an HTTP endpoint reachable by Pub/Sub.
- `watch serve` and `watch pull` default `--exclude-labels` to `SPAM,TRASH`; set to an empty string to disable.
- Exclude label IDs are matched exactly (case-sensitive opaque IDs).
- `watch serve --fetch-delay` delays Gmail history fetch after each push (default `3s`) to avoid indexing races; accepts seconds (`5`) or Go durations (`5s`).
- `watch serve --history-types` accepts `messageAdded`, `messageDeleted`, `labelAdded`, `labelRemoved` (repeatable or comma-separated). Default: `messageAdded` (for backward compatibility).
- `watch serve --history-types` must include at least one non-empty type.
- `watch serve --fetch-delay` and `watch pull --fetch-delay` delay Gmail
history fetch after each notification (default `3s`) to avoid indexing races;
accepts seconds (`5`) or Go durations (`5s`).
- `watch serve --history-types` and `watch pull --history-types` accept
`messageAdded`, `messageDeleted`, `labelAdded`, `labelRemoved` (repeatable or
comma-separated). Default: `messageAdded` (for backward compatibility).
- `watch serve --history-types` and `watch pull --history-types` must include at
least one non-empty type.

## State

Expand Down Expand Up @@ -136,8 +181,44 @@ Preferred:
Fallback (dev only):
- Shared token via `x-gog-token` header or `?token=`.

## Auth (pull)

Pull delivery does not expose a public HTTP receiver. The local `gog` process
must have Google credentials for:

- Gmail history reads for the watched account. These use the normal stored
`gog` Gmail OAuth account selected by `--account` / `--client`.
- Pub/Sub subscriber access on the configured subscription. These use the
Google Cloud client library credential chain, for example Application Default
Credentials or `GOOGLE_APPLICATION_CREDENTIALS`, not the stored Gmail OAuth
token. The credential must be able to consume the subscription; granting
`roles/pubsub.subscriber` on the subscription is the usual least-privilege
shape.

The downstream hook token is still local to the hook call from `gog` to the
configured `--hook-url`.

## Error handling

- Stale historyId: fall back to `messages.list` (last N) + reset historyId.
- Watch expired: `watch renew` error; rerun `watch start`.
- Hook failures: log and still advance historyId to avoid replay storms.
- Pull mode treats invalid Pub/Sub messages as poison messages: log and
acknowledge them rather than redelivering forever. Wrong-account
notifications are also terminal in both modes.
- Hook failures are retryable. `gog` records the hook failure status, preserves
the pre-hook watch cursor, and returns a delivery failure to Pub/Sub. This
lets Pub/Sub redeliver the notification after the downstream agent or gateway
comes back.
- This is an intentional reliability change for existing push deployments.
Older `watch serve` behavior acknowledged hook failures to avoid replay
storms, but that could silently lose Gmail wakeups when the downstream
OpenClaw gateway or agent was temporarily down. The supported behavior is now
delivery-before-cursor-advance for both push and pull: push returns non-2xx on
hook failure and pull nacks the message.
- Pub/Sub may retry the same notification until the hook succeeds or until the
subscription's retry/dead-letter policy takes over. Hook receivers should be
safe to call more than once for the same Gmail history notification.
- This retry policy is intended for normal Gmail notification volumes. If you
are processing very high mail rates, for example 1000 messages per minute, run
your own monitoring, alerting, backlog policy, and dead-letter/backpressure
setup instead of treating the default watcher as a complete queueing platform.
6 changes: 6 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/steipete/gogcli
go 1.26.2

require (
cloud.google.com/go/pubsub/v2 v2.6.0
filippo.io/age v1.3.1
github.com/99designs/keyring v1.2.2
github.com/alecthomas/kong v1.15.0
Expand All @@ -21,9 +22,11 @@ require (
)

require (
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.20.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.8.0 // indirect
filippo.io/hpke v0.4.0 // indirect
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
Expand All @@ -50,13 +53,16 @@ require (
github.com/spf13/cast v1.7.1 // indirect
github.com/stretchr/objx v0.5.3 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/time v0.15.0 // indirect
google.golang.org/genproto v0.0.0-20260414002931-afd174a4e478 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260414002931-afd174a4e478 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 // indirect
Expand Down
Loading
Loading