Google Drive - new source changes-to-files-in-drive using Changes API instead of File Subscriptions API #18397
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
WalkthroughAdds a new Google Drive source "Changes to Files in Drive" using the Drive Changes API; enforces a 10-file deploy-time limit for the "Changes to Specific Files" source with an info alert; fetches full file metadata and changes id generation in the shared-drive specific-files variant; adjusts common dedupe/default interval and webhook HTTP prop to support custom responses; and bumps several package/source versions. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor U as User
participant S as Source: Changes to Files in Drive
participant GD as Google Drive (Changes API)
participant ST as Stash Service (optional)
participant E as Event Stream
U->>S: Configure (drive, files, includeLink, dir, dedupe)
Note over S: Deploy seeds recent changes (last 30 days)
S->>GD: List recent changes
GD-->>S: Change candidates (file metadata + headers)
S->>S: Filter relevant files & check minimum interval
alt includeLink enabled
S->>ST: stashFile(file)
ST-->>S: temp link + metadata
end
S->>E: Emit event {file payload, meta}
sequenceDiagram
autonumber
participant SS as Source: Changes to Specific Files
participant Base as Base deploy
Note over SS: Deploy-time validation (10-file limit)
SS->>SS: if this.files.length > 10
alt > 10 files
SS-->>SS: throw ConfigurationError
else <= 10 files
SS->>Base: delegate deploy()
Base-->>SS: complete
end
sequenceDiagram
autonumber
participant H as Common Webhook
participant HTTP as HTTP Interface
participant S as Source Logic
HTTP->>H: Incoming HTTP notification (not timer)
H->>HTTP: respond({status:200}) %% customResponse path
H->>S: continue processing headers & changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (3 warnings)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (7)
components/google_drive/sources/changes-to-specific-files/changes-to-specific-files.mjs (1)
59-64: Null‑safe guard for files selection.Accessing
this.files.lengthcan throw iffilesis unset. Make the check null/array‑safe to avoid deploy‑time crashes.Apply this diff:
async deploy() { - if (this.files.length > 10) { - throw new ConfigurationError("This source only supports up to 10 files"); - } + if (!Array.isArray(this.files) || this.files.length === 0) { + throw new ConfigurationError("Select at least one file (max 10)."); + } + if (this.files.length > 10) { + throw new ConfigurationError("This source only supports up to 10 files."); + } await changesToSpecificFiles.hooks.deploy.bind(this)(); },components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs (6)
62-66: Explicitly request file fields needed for seeding.
fields: "files"risks omittingname/modifiedTime(used later). Request them explicitly to avoidNaNtimestamps and “Untitled” summaries during deploy seeding.Apply this diff:
- const args = this.getListFilesOpts({ - q: `mimeType != "application/vnd.google-apps.folder" and modifiedTime > "${timeString}" and trashed = false`, - fields: "files", - pageSize: 5, - }); + const args = this.getListFilesOpts({ + q: `mimeType != "application/vnd.google-apps.folder" and modifiedTime > "${timeString}" and trashed = false`, + // Include nextPageToken and explicit file subfields used downstream + fields: "nextPageToken, files(id, name, modifiedTime, mimeType)", + pageSize: 5, + });
132-145: Reduce noisy logs and guard empty input.The triple debug logs with full JSON can be very chatty in prod and may include large payloads. Add an early return and gate verbose logs.
Apply this diff:
- console.log(`Processing ${changedFiles.length} changed files`); - console.log(`Changed files: ${JSON.stringify(changedFiles, null, 2)}!!!`); - console.log(`Files: ${this.files}!!!`); + if (!Array.isArray(changedFiles) || changedFiles.length === 0) { + return; + } + if (process.env.PIPEDREAM_DEBUG === "true") { + console.log(`Processing ${changedFiles.length} changed files`); + }
83-101: Harden meta timestamp fallback.When headers are absent (deploy seeding) or
modifiedTimeis missing,Date.parseyieldsNaN. Provide a safe fallback toDate.now().Apply this diff:
- const ts = Date.parse(tsString); + const parsed = tsString ? Date.parse(tsString) : NaN; + const ts = Number.isFinite(parsed) ? parsed : Date.now();
120-129: Don’t fail the whole event if stashing the file fails.Wrap
stashFilein a try/catch so a transient download/export error doesn’t block emission of the change event.Apply this diff:
- if (this.includeLink) { - file.file = await stashFile(file, this.googleDrive, this.dir); - } + if (this.includeLink) { + try { + file.file = await stashFile(file, this.googleDrive, this.dir); + } catch (err) { + console.warn("stashFile failed, emitting without link:", err?.message || err); + } + }
106-119: Capture extra headers for diagnostics (optional).Including
x-goog-channel-idcan help correlate webhook deliveries during debugging.Apply this diff:
return { change: { state: headers["x-goog-resource-state"], resourceURI: headers["x-goog-resource-uri"], changed: headers["x-goog-changed"], // "Additional details about the changes. Possible values: content, parents, children, permissions" + channelId: headers["x-goog-channel-id"], }, };
103-105: Micro: speed up relevance checks for many files.For large
fileslists, consider aSetto avoid O(n) membership checks on each file.Example:
// initialize once (e.g., in activate/deploy or lazy getter) this._filesSet = new Set(this.files); // then: isFileRelevant(file) { return this._filesSet.has(file.id); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
components/google_drive/package.json(1 hunks)components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs(1 hunks)components/google_drive/sources/changes-to-specific-files/changes-to-specific-files.mjs(3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs (2)
components/google_drive/sources/changes-to-specific-files/changes-to-specific-files.mjs (2)
files(80-80)file(238-240)components/google_drive/google_drive.app.mjs (1)
changedFiles(393-395)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: pnpm publish
- GitHub Check: Publish TypeScript components
- GitHub Check: Lint Code Base
- GitHub Check: Verify TypeScript components
🔇 Additional comments (5)
components/google_drive/package.json (1)
3-3: Version bump looks right for adding a new source.Minor bump from 1.0.x → 1.1.0 aligns with a backwards‑compatible feature addition.
components/google_drive/sources/changes-to-specific-files/changes-to-specific-files.mjs (3)
6-6: Good addition: explicit ConfigurationError import.Enables clear config‑time validation without generic exceptions.
19-19: Version bump acknowledged.Patch bump to 0.3.1 is appropriate for a non‑breaking guard + copy change.
25-29: Helpful UX: info alert about the 10‑file limit.Clear guidance to use the Changes API–based source for >10 files.
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs (1)
1-74: Confirmed: source uses the Changes API (drive.changes.watch & drive.changes.list)components/google_drive/google_drive.app.mjs implements listChanges (calls drive.changes.list) and a watch path that calls drive.changes.watch; components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs references changes.watch and contains no files.watch calls (files.watch is used only by changes-to-specific-files/watchFile).
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
components/google_drive/sources/changes-to-specific-files-shared-drive/changes-to-specific-files-shared-drive.mjs (1)
139-142: Trim noisy logsThe triple “!!!” debug prints are noisy. Prefer
this.$logger.debug(or remove) and avoid logging entire file arrays in production paths.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs(1 hunks)components/google_drive/sources/changes-to-specific-files-shared-drive/changes-to-specific-files-shared-drive.mjs(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs
🧰 Additional context used
🧬 Code graph analysis (1)
components/google_drive/sources/changes-to-specific-files-shared-drive/changes-to-specific-files-shared-drive.mjs (1)
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs (4)
fileInfo(125-125)eventToEmit(126-129)changes(121-121)meta(130-130)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Lint Code Base
- GitHub Check: Verify TypeScript components
- GitHub Check: Publish TypeScript components
- GitHub Check: pnpm publish
🔇 Additional comments (1)
components/google_drive/sources/changes-to-specific-files-shared-drive/changes-to-specific-files-shared-drive.mjs (1)
31-31: Version bump to 0.3.1 looks good; ensure consistency across artifactsKeep this in sync with package/version docs and any release notes.
...ve/sources/changes-to-specific-files-shared-drive/changes-to-specific-files-shared-drive.mjs
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
components/google_drive/sources/common-dedupe-changes.mjs (2)
16-16: Default raised to 3 min — update the user-facing alert text to match.The alert still says “at least 1 minute.” Sync it to 3 minutes to avoid confusion.
Apply:
- content: `This source can emit many events in quick succession while a file is being edited. By default, it will not emit another event for the same file for at least 1 minute. + content: `This source can emit many events in quick succession while a file is being edited. By default, it will not emit another event for the same file for at least 3 minutes. \ You can change or disable this minimum interval using the prop \`Minimum Interval Per File\`.`,Also applies to: 6-9
1-19: Optional: de‑duplicate the “3” with a single constant to prevent future drift.Keeps the alert copy and the prop default in lockstep.
+const DEFAULT_PER_FILE_INTERVAL_MINUTES = 3; + export default { props: { intervalAlert: { type: "alert", alertType: "info", - content: `This source can emit many events in quick succession while a file is being edited. By default, it will not emit another event for the same file for at least 1 minute. + content: `This source can emit many events in quick succession while a file is being edited. By default, it will not emit another event for the same file for at least ${DEFAULT_PER_FILE_INTERVAL_MINUTES} minutes. \ You can change or disable this minimum interval using the prop \`Minimum Interval Per File\`.`, }, perFileInterval: { type: "integer", label: "Minimum Interval Per File", description: "How many minutes to wait until the same file can emit another event.\n\nIf set to `0`, this interval is disabled and all events will be emitted.", min: 0, max: 60, - default: 3, + default: DEFAULT_PER_FILE_INTERVAL_MINUTES, optional: true, }, },Also applies to: 16-16
components/google_drive/sources/common-webhook.mjs (1)
163-166: Early 200 before processing is good; add a try/catch around change processing for resilience.If processing throws after the ACK, we silently fail and may reprocess on the next delivery. Log explicitly to aid ops.
- const driveId = this.getDriveId(); - const changedFilesStream = this.googleDrive.listChanges(pageToken, driveId); - for await (const changedFilesPage of changedFilesStream) { - const { - changedFiles, - nextPageToken, - } = changedFilesPage; - - // Process all the changed files retrieved from the current page - await this.processChanges(changedFiles, headers); - - // After successfully processing the changed files, we store the page - // token of the next page - this._setPageToken(nextPageToken); - } + const driveId = this.getDriveId(); + try { + const changedFilesStream = this.googleDrive.listChanges(pageToken, driveId); + for await (const changedFilesPage of changedFilesStream) { + const { changedFiles, nextPageToken } = changedFilesPage; + await this.processChanges(changedFiles, headers); + this._setPageToken(nextPageToken); + } + } catch (err) { + console.error("Error processing Google Drive changes", { + err, + channelID, + pageToken, + driveId, + }); + }Also applies to: 196-210
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs(1 hunks)components/google_drive/sources/common-dedupe-changes.mjs(1 hunks)components/google_drive/sources/common-webhook.mjs(2 hunks)components/google_drive/sources/new-files-instant/new-files-instant.mjs(1 hunks)components/google_drive/sources/new-files-shared-drive/new-files-shared-drive.mjs(1 hunks)components/google_drive/sources/new-or-modified-comments/new-or-modified-comments.mjs(1 hunks)components/google_drive/sources/new-or-modified-files/new-or-modified-files.mjs(1 hunks)components/google_drive/sources/new-or-modified-folders/new-or-modified-folders.mjs(1 hunks)components/google_drive/sources/new-spreadsheet/new-spreadsheet.mjs(1 hunks)
✅ Files skipped from review due to trivial changes (6)
- components/google_drive/sources/new-or-modified-comments/new-or-modified-comments.mjs
- components/google_drive/sources/new-or-modified-files/new-or-modified-files.mjs
- components/google_drive/sources/new-files-shared-drive/new-files-shared-drive.mjs
- components/google_drive/sources/new-or-modified-folders/new-or-modified-folders.mjs
- components/google_drive/sources/new-spreadsheet/new-spreadsheet.mjs
- components/google_drive/sources/new-files-instant/new-files-instant.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- components/google_drive/sources/changes-to-files-in-drive/changes-to-files-in-drive.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Verify TypeScript components
- GitHub Check: Publish TypeScript components
- GitHub Check: Lint Code Base
🔇 Additional comments (1)
components/google_drive/sources/common-webhook.mjs (1)
13-16: Approve — customResponse enabled; no duplicate responders foundEnabling customResponse is correct for a fast 200 ACK. Repo search shows the only responder in components/google_drive is this.http.respond at components/google_drive/sources/common-webhook.mjs:164–166.
|
/approve |
Resolves #18315
Summary by CodeRabbit
New Features
Bug Fixes / Behavior
Chores