✨ server: implement chat hook and worker - #1252
Conversation
🦋 Changeset detectedLatest commit: f0aeafd The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughThe PR adds WhatsApp chat webhook ingestion, queue-backed processing, Mastra worker responses, runtime entrypoints, Sentry configuration, deployment wiring, and automated tests and evaluations. ChangesChat runtime
Estimated code review effort: 4 (Complex) | ~60 minutes Merge Risk: 🟠 High · up to The chat hook and worker can reject valid inbound messages, retain sender identifiers indefinitely, expose customer data in failure telemetry, and send duplicate replies after retries; partial startup failures can also leave clients open. These correctness, privacy, delivery, and cleanup issues make the PR unsafe to merge until fixed or explicitly accepted. Sequence Diagram(s)sequenceDiagram
participant WhatsAppWebhook
participant ChatHook
participant ChatQueue
participant ChatWorker
participant MastraAgents
participant WhatsAppGraphAPI
WhatsAppWebhook->>ChatHook: Deliver verified webhook event
ChatHook->>ChatQueue: Enqueue grouped chat job
ChatQueue->>ChatWorker: Deliver chat job
ChatWorker->>MastraAgents: Generate welcome or help response
ChatWorker->>WhatsAppGraphAPI: Send localized text response
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e311af71-bdf1-4969-86fd-56614006652d
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (26)
.changeset/chubby-melons-look.md.changeset/lucky-parrots-chat.md.changeset/nice-days-deny.md.changeset/public-vans-shine.md.github/workflows/server-base-sepolia.yamlcspell.jsoninfra/utils/modules.tspackage.jsonpnpm-workspace.yamlserver/hooks/bin/chat.tsserver/hooks/chat.tsserver/instrument.cjsserver/package.jsonserver/test/hooks/bin.test.tsserver/test/hooks/chat.test.tsserver/test/hooks/hook.test.tsserver/test/instrument.test.tsserver/test/utils/whatsapp.test.tsserver/test/workers/bin.test.tsserver/test/workers/chat.eval.tsserver/test/workers/chat.test.tsserver/utils/whatsapp.tsserver/workers/chat/bin.tsserver/workers/chat/job.tsserver/workers/chat/queue.tsserver/workers/chat/worker.ts
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
| .catch((error: unknown) => { | ||
| captureException(error, { extra: { sender }, tags: { job: "chat", queue: "chat" } }); | ||
| throw error; |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
How does the Sentry JavaScript SDK include captureException extra data in captured events?
💡 Result:
The Sentry JavaScript SDK provides two primary ways to include extra data when calling captureException: passing it directly as an argument or utilizing scope management [1][2][3]. Direct Inclusion via captureException You can include extra data directly in a single captureException call by passing a capture context object as the second argument [1][4][5]. This object accepts an extra property, which should be a key-value record of additional data [1][4]. Example: Sentry.captureException(error, { extra: { userId: 123, environment: 'production' } }); This data will be merged with existing data in the current scope [5][3]. Scope Management Alternatively, you can manage extra data using the SDK's scope-based methods. This is useful when you want to persist data across multiple events or limit the scope of the data to a specific block of code [2][3]. 1. Global/Current Scope Methods: You can use setExtra or setExtras to add data that will be included in all future events captured on that scope [1][6][7]. Sentry.setExtra('key', 'value'); Sentry.setExtras({ key1: 'value1', key2: 'value2' }); 2. Using withScope: To isolate extra data to a specific event or block of code, you can use withScope [2][8]. This creates a temporary scope that is automatically cleaned up after the callback executes [3]. Sentry.withScope((scope) => { scope.setExtra('key', 'value'); Sentry.captureException(error); }); While the SDK continues to support extra data for backward compatibility, Sentry documentation notes that unstructured "additional data" via setExtra is largely deprecated in favor of more structured Contexts (using setContext), which provide better organization in the Sentry UI [2][3].
Citations:
- 1: https://docs.sentry.io/platforms/javascript/configuration/apis/
- 2: https://docs.sentry.io/platforms/javascript/enriching-events/
- 3: https://docs.sentry.io/platforms/react-native/enriching-events/context/
- 4: https://www.mintlify.com/getsentry/sentry-javascript/api/capture/exception
- 5: feat: Explicit Scope for captureException and captureMessage getsentry/sentry-javascript#2627
- 6: https://docs.sentry.io/platforms/javascript/configuration/apis/?platform=browser
- 7: https://docs.sentry.io/platforms/javascript/guides/express/configuration/apis/
- 8: https://docs.sentry.io/platforms/javascript/enriching-events/scopes/
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- production hook ---'
sed -n '1,180p' server/hooks/chat.ts
printf '%s\n' '--- affected test sections ---'
sed -n '200,285p' server/test/hooks/chat.test.ts
printf '%s\n' '--- captureException definitions and usages ---'
rg -n --glob '*.{ts,tsx,js,jsx}' 'captureException|enqueue|sender' serverRepository: exactly/exa
Length of output: 50367
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- Sentry configuration ---'
rg -n -C 4 --glob '*.{ts,tsx,js,jsx}' 'Sentry\.init|beforeSend|beforeBreadcrumb|sendDefaultPii|normalizeDepth|extra' server . --glob '!node_modules/**' --glob '!dist/**' | head -n 240
printf '%s\n' '--- additional chat test context ---'
sed -n '1,90p' server/test/hooks/chat.test.ts
sed -n '240,275p' server/test/hooks/chat.test.ts
printf '%s\n' '--- focused structural probe ---'
python3 - <<'PY'
from pathlib import Path
hook = Path("server/hooks/chat.ts").read_text()
test = Path("server/test/hooks/chat.test.ts").read_text()
required_sender_fields = ["id", "from", "text", "contact", "phoneNumberId"]
missing = [field for field in required_sender_fields if field not in hook]
assert not missing, f"missing sender fields in production source: {missing}"
assert 'extra: { sender }' in hook, "production source does not pass sender as Sentry extra"
assert 'extra: { sender: { id: "whatsapp-1", from: "US.12345678", text: "Hi!", contact: "Jhon", phoneNumberId: "321" } }' in test, "test does not assert the complete sender payload"
print("production sender fields:", ", ".join(required_sender_fields))
print("production Sentry extra: complete sender object")
print("test expectation: complete sender object")
PYRepository: exactly/exa
Length of output: 19406
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- instrumentation candidates ---'
fd -HI -t f '(instrument|sentry|config)' . | head -n 120
printf '%s\n' '--- Sentry initialization ---'
rg -n -C 8 --glob '*.{ts,tsx,js,jsx}' 'init\s*\(|beforeSend|beforeSendTransaction|sendDefaultPii|Sentry' . --glob '!node_modules/**' --glob '!dist/**' | head -n 260
printf '%s\n' '--- relevant diff summary ---'
git diff --stat -- server/hooks/chat.ts server/test/hooks/chat.test.ts
git diff --unified=3 -- server/hooks/chat.ts server/test/hooks/chat.test.ts | sed -n '1,220p'Repository: exactly/exa
Length of output: 17919
🏁 Script executed:
#!/bin/bash
set -eu
cat -n server/instrument.cjs
printf '%s\n' '--- Sentry mock ---'
cat -n server/test/mocks/sentry.tsRepository: exactly/exa
Length of output: 2730
Remove customer content from Sentry event extras. sender includes contact, the WhatsApp user ID, and message text. beforeSend does not redact these fields before Sentry receives the event.
server/hooks/chat.ts#L82-L84: pass only allowlisted diagnostics, such asidandphoneNumberId.server/test/hooks/chat.test.ts#L228-L237: assert the redacted payload.
📍 Affects 2 files
server/hooks/chat.ts#L82-L84(this comment)server/test/hooks/chat.test.ts#L228-L237
| function parse({ entry }: v.InferOutput<typeof event>) { | ||
| return entry.flatMap(({ changes }) => | ||
| changes.flatMap(({ value: { contacts, messages, metadata } }) => | ||
| (messages ?? []).flatMap((message) => | ||
| message.text | ||
| ? [ | ||
| { | ||
| id: message.id, | ||
| from: message.from_user_id, | ||
| text: message.text.body, | ||
| contact: contacts?.find(({ user_id }) => user_id === message.from_user_id)?.profile?.name, | ||
| phoneNumberId: metadata.phone_number_id, | ||
| }, | ||
| ] | ||
| : [], | ||
| ), | ||
| ), | ||
| ); | ||
| } | ||
|
|
||
| const event = v.object({ | ||
| entry: v.array( | ||
| v.object({ | ||
| changes: v.array( | ||
| v.object({ | ||
| value: v.object({ | ||
| metadata: v.object({ phone_number_id: v.string() }), | ||
| contacts: v.optional( | ||
| v.array( | ||
| v.object({ | ||
| user_id: v.string(), | ||
| profile: v.optional(v.object({ name: v.optional(v.string()) })), | ||
| }), | ||
| ), | ||
| ), | ||
| messages: v.optional( | ||
| v.array( | ||
| v.object({ | ||
| id: v.string(), | ||
| from_user_id: v.string(), | ||
| type: v.string(), | ||
| text: v.optional(v.object({ body: v.string() })), |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
What fields identify the sender and contact in Meta WhatsApp Cloud API inbound message webhooks?
💡 Result:
In Meta WhatsApp Cloud API inbound message webhooks, the sender and contact information are identified primarily through the following fields located within the entries object of the JSON payload [1][2]: Sender Identification The sender of the message is identified within the messages array object by the from field [1][3]. This field contains the WhatsApp ID (wa_id) of the user who sent the message, which is typically their phone number [1][2]. Contact Identification Additional contact details, such as the user's profile name, are provided in the contacts array, which is present at the same level as the messages array within the webhook value object [1][4][2]. Each entry in the contacts array typically includes: wa_id: The WhatsApp ID of the user, which corresponds to the from field in the message object [1][2]. profile: An object containing user-specific information, such as the name field, which provides the sender's WhatsApp profile name [1][2]. These fields allow you to link the incoming message (via the from field) to the detailed contact information (via the wa_id and profile fields) provided in the same webhook notification [1][2].
Citations:
- 1: https://developers.facebook.com/documentation/business-messaging/whatsapp/webhooks/reference/messages
- 2: https://developers.facebook.com/docs/whatsapp/cloud-api/guides/set-up-webhooks/
- 3: https://github.com/WhatsApp/WhatsApp-Nodejs-SDK/blob/main/src/types/webhooks.ts
- 4: https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/components/
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- production hook ---'
sed -n '70,155p' server/hooks/chat.ts
printf '%s\n' '--- test fixture and assertions ---'
sed -n '210,285p' server/test/hooks/chat.test.ts
printf '%s\n' '--- field references ---'
rg -n '"(user_id|wa_id|from_user_id|from)"|user_id|wa_id|from_user_id' server/hooks/chat.ts server/test/hooks/chat.test.tsRepository: exactly/exa
Length of output: 6092
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
hook = Path("server/hooks/chat.ts").read_text()
fixture = Path("server/test/hooks/chat.test.ts").read_text()
checks = {
"production schema uses user_id": "user_id: v.string()" in hook,
"production schema uses from_user_id": "from_user_id: v.string()" in hook,
"production schema uses wa_id": "wa_id" in hook,
"production schema uses from field": "from: v.string()" in hook,
"test fixture uses user_id": "user_id:" in fixture,
"test fixture uses from_user_id": "from_user_id:" in fixture,
"test fixture uses wa_id": "wa_id:" in fixture,
"test fixture uses from field": "from:" in fixture,
}
for name, value in checks.items():
print(f"{name}: {value}")
# Model the relevant webhook transformation without importing repository code.
payload = {
"entry": [{
"changes": [{
"value": {
"metadata": {"phone_number_id": "321"},
"contacts": [{"wa_id": "US.12345678", "profile": {"name": "Jhon"}}],
"messages": [{
"id": "whatsapp-1",
"from": "US.12345678",
"type": "text",
"text": {"body": "Hi!"},
}],
},
}],
}],
}
value = payload["entry"][0]["changes"][0]["value"]
message = value["messages"][0]
schema_expected = {
"contact identifier": value["contacts"][0]["wa_id"],
"sender identifier": message["from"],
}
schema_current = {
"contact identifier": value["contacts"][0].get("user_id"),
"sender identifier": message.get("from_user_id"),
}
print("real webhook fields:", schema_expected)
print("current schema lookups:", schema_current)
PYRepository: exactly/exa
Length of output: 615
🏁 Script executed:
#!/bin/bash
set -e
sed -n '1,75p' server/hooks/chat.ts
printf '%s\n' '--- all webhook parsing and validation calls ---'
rg -n -C 3 'parse\(|event|safeParse|parse\(' server/hooks/chat.ts server/test/hooks/chat.test.tsRepository: exactly/exa
Length of output: 5928
Use the WhatsApp Cloud API field names. The schema requires contacts[].user_id and messages[].from_user_id, but inbound WhatsApp webhooks provide contacts[].wa_id and messages[].from. Text messages therefore fail validation before queue.enqueue.
server/hooks/chat.ts:102-143: validate and usewa_idandfrom.server/test/hooks/chat.test.ts:248-269: update the fixture to usewa_idandfrom.
📍 Affects 2 files
server/hooks/chat.ts#L102-L143(this comment)server/test/hooks/chat.test.ts#L248-L269
| vi.stubEnv("K_SERVICE", "sandbox-chat"); | ||
| const transaction = { transaction: "POST /" }; | ||
|
|
||
| const result = config.beforeSendTransaction?.(transaction as never, {} as never); |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- target file ---'
cat -n server/test/instrument.test.ts
printf '%s\n' '--- package manifests and sentry references ---'
rg -n --glob 'package.json' --glob 'pnpm-lock.yaml' --glob 'yarn.lock' --glob 'package-lock.json' '`@sentry/`(core|node)|sentry' .
printf '%s\n' '--- callback and fixture references ---'
rg -n 'beforeSendTransaction|EventHint|TransactionEvent|as never|type: "transaction"' serverRepository: exactly/exa
Length of output: 20998
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- instrument implementation ---'
cat -n server/instrument.cjs
printf '%s\n' '--- relevant TypeScript configuration ---'
fd -HI 'tsconfig*.json' -o -HI 'package.json' | sort | xargs -r grep -nH -E 'allowJs|checkJs|strict|typecheck|tsc|vitest'
printf '%s\n' '--- Sentry event types at the declared lockfile version ---'
curl -fsSL https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/src/types/event.ts |
rg -n -A18 -B8 'interface TransactionEvent|type TransactionEvent|interface EventHint|type EventHint|BeforeSendTransaction'Repository: exactly/exa
Length of output: 3497
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- project configuration ---'
find . -maxdepth 3 -type f \( -name 'tsconfig.json' -o -name 'tsconfig.*.json' \) -print
cat -n server/package.json | sed -n '1,115p'
printf '%s\n' '--- callback type definitions ---'
for url in \
https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/src/types-hoist/options.ts \
https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/src/types-hoist/event.ts \
https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/src/types/event.ts
do
echo "--- $url"
curl -fsSL "$url" | rg -n -A12 -B8 'beforeSendTransaction|ProcessEventTransaction|TransactionEvent'
done
printf '%s\n' '--- available compiler ---'
command -v tsc || true
tsc --version 2>/dev/null || trueRepository: exactly/exa
Length of output: 5367
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- TypeScript configuration ---'
for file in tsconfig.json server/tsconfig.json; do
echo "--- $file"
cat -n "$file"
done
printf '%s\n' '--- Sentry core source tree candidates ---'
curl -fsSL 'https://api.github.com/repos/getsentry/sentry-javascript/git/trees/10.70.0?recursive=1' |
jq -r '.tree[].path' |
rg 'packages/(core|node)/.*(options|types|client).*\.ts$' |
head -80
printf '%s\n' '--- callback declarations in the v10.70.0 tree ---'
paths=$(curl -fsSL 'https://api.github.com/repos/getsentry/sentry-javascript/git/trees/10.70.0?recursive=1' |
jq -r '.tree[].path' |
rg '^packages/(core|node)/.*\.ts$' |
rg '(options|types|client|baseclient)')
while IFS= read -r path; do
content=$(curl -fsSL "https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/$path") || continue
if printf '%s' "$content" | rg -q 'beforeSendTransaction|ProcessEventTransaction|TransactionEvent'; then
echo "--- $path"
printf '%s' "$content" | rg -n -A12 -B8 'beforeSendTransaction|ProcessEventTransaction|TransactionEvent'
fi
done <<< "$paths"Repository: exactly/exa
Length of output: 33989
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- public exports ---'
for path in packages/core/src/index.ts packages/core/src/types/index.ts packages/core/package.json; do
echo "--- $path"
curl -fsSL "https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/$path" |
rg -n -A8 -B4 'EventHint|TransactionEvent|types/event'
done
printf '%s\n' '--- event base shape ---'
curl -fsSL https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/src/types/event.ts |
sed -n '1,90p'
printf '%s\n' '--- compiler availability ---'
command -v tsc || true
tsc --version 2>/dev/null || trueRepository: exactly/exa
Length of output: 203
🏁 Script executed:
#!/bin/bash
set +e
printf '%s\n' '--- package metadata ---'
curl -fsSL https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/packages/core/package.json |
sed -n '1,180p'
printf '%s\n' '--- core source export candidates ---'
curl -fsSL 'https://api.github.com/repos/getsentry/sentry-javascript/git/trees/10.70.0?recursive=1' |
jq -r '.tree[].path' |
rg '^packages/core/src/.*(index|types).*\.ts$'
printf '%s\n' '--- exports containing event types ---'
for path in \
packages/core/src/index.ts \
packages/core/src/types.ts \
packages/core/src/types/index.ts \
packages/core/src/types-hoist.ts \
packages/core/src/types-hoist/index.ts
do
echo "--- $path"
body=$(curl -fsSL "https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/$path")
status=$?
if [ "$status" -eq 0 ]; then
printf '%s\n' "$body" | rg -n -A6 -B4 'EventHint|TransactionEvent|types/event|export \*' || true
else
echo "unavailable"
fi
done
printf '%s\n' '--- compiler availability ---'
command -v tsc || true
tsc --version 2>/dev/null || trueRepository: exactly/exa
Length of output: 7542
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- Sentry public export lists ---'
for path in packages/core/src/shared-exports.ts packages/core/src/server-exports.ts packages/core/src/browser-exports.ts; do
echo "--- $path"
curl -fsSL "https://raw.githubusercontent.com/getsentry/sentry-javascript/10.70.0/$path" |
rg -n -A5 -B3 'event|EventHint|TransactionEvent|export'
done
printf '%s\n' '--- standalone TypeScript fixture probe ---'
tmp=$(mktemp --suffix=.ts)
trap 'rm -f "$tmp"' EXIT
cat >"$tmp" <<'TS'
interface Event {
type?: "transaction" | "profile" | "replay_event" | "feedback";
transaction?: string;
extra?: Record<string, unknown>;
}
interface TransactionEvent extends Event {
type: "transaction";
}
interface EventHint {
originalException?: unknown;
}
declare const beforeSendTransaction:
| ((event: TransactionEvent, hint: EventHint) => TransactionEvent | null)
| undefined;
const transaction = { type: "transaction", transaction: "POST /" } satisfies TransactionEvent;
const ignored = { type: "transaction", extra: { "exa.ignore": true }, transaction: "POST /" } satisfies TransactionEvent;
const hint = {} satisfies EventHint;
transaction.transaction = "POST / · sandbox-chat";
ignored.transaction = "POST /";
const result = beforeSendTransaction?.(transaction, hint);
TS
tsc --strict --noEmit --skipLibCheck "$tmp"Repository: exactly/exa
Length of output: 27804
🏁 Script executed:
#!/bin/bash
set -e
tmp=$(mktemp --suffix=.ts)
trap 'rm -f "$tmp"' EXIT
cat >"$tmp" <<'TS'
interface Event {
type?: "transaction" | "profile" | "replay_event" | "feedback";
transaction?: string;
extra?: Record<string, unknown>;
}
interface TransactionEvent extends Event {
type: "transaction";
}
interface EventHint {
originalException?: unknown;
}
declare const beforeSendTransaction:
| ((event: TransactionEvent, hint: EventHint) => TransactionEvent | null)
| undefined;
const transaction = { type: "transaction", transaction: "POST /" } satisfies TransactionEvent;
const ignored = { type: "transaction", extra: { "exa.ignore": true }, transaction: "POST /" } satisfies TransactionEvent;
const hint = {} satisfies EventHint;
transaction.transaction = "POST / · sandbox-chat";
ignored.transaction = "POST /";
const result = beforeSendTransaction?.(transaction, hint);
TS
tsc --ignoreConfig --strict --noEmit --skipLibCheck "$tmp"Repository: exactly/exa
Length of output: 1224
🏁 Script executed:
#!/bin/bash
set -e
tmp=$(mktemp --suffix=.ts)
trap 'rm -f "$tmp"' EXIT
cat >"$tmp" <<'TS'
interface SentryEvent {
type?: "transaction" | "profile" | "replay_event" | "feedback";
transaction?: string;
extra?: Record<string, unknown>;
}
interface TransactionEvent extends SentryEvent {
type: "transaction";
}
interface EventHint {
originalException?: unknown;
}
declare const beforeSendTransaction:
| ((event: TransactionEvent, hint: EventHint) => TransactionEvent | null)
| undefined;
const transaction = { type: "transaction", transaction: "POST /" } satisfies TransactionEvent;
const ignored = { type: "transaction", extra: { "exa.ignore": true }, transaction: "POST /" } satisfies TransactionEvent;
const hint = {} satisfies EventHint;
transaction.transaction = "POST / · sandbox-chat";
ignored.transaction = "POST /";
const result = beforeSendTransaction?.(transaction, hint);
TS
tsc --ignoreConfig --strict --noEmit --skipLibCheck --lib ESNext "$tmp"Repository: exactly/exa
Length of output: 149
Use typed Sentry fixtures instead of never casts.
Add type: "transaction" to each fixture. Use EventHint and TransactionEvent from @sentry/core. Call beforeSendTransaction without casts.
Source: Coding guidelines
| const seen = await bullmq.exists(`whatsapp:seen:${data.from}`); | ||
| const { text } = await reply(data.text, { | ||
| requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", seen === 1]]), | ||
| }); | ||
| await whatsapp.send(data.from, text); | ||
| await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX"); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Set an expiry on the whatsapp:seen: key.
bullmq.set(\whatsapp:seen:${data.from}`, ..., "NX")` writes a key with no TTL. The key name contains the sender phone number. Every sender that ever writes creates one permanent Redis key. This grows without bound and retains a personal identifier indefinitely.
Add an expiry that matches the intended "returning sender" window.
🔒️ Proposed fix
- await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX");
+ await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "EX", seenTtl, "NX");Add the constant next to ttl:
const seenTtl = 90 * 24 * 3600;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const seen = await bullmq.exists(`whatsapp:seen:${data.from}`); | |
| const { text } = await reply(data.text, { | |
| requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", seen === 1]]), | |
| }); | |
| await whatsapp.send(data.from, text); | |
| await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX"); | |
| const seenTtl = 90 * 24 * 3600; | |
| const seen = await bullmq.exists(`whatsapp:seen:${data.from}`); | |
| const { text } = await reply(data.text, { | |
| requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", seen === 1]]), | |
| }); | |
| await whatsapp.send(data.from, text); | |
| await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "EX", seenTtl, "NX"); |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e398df1e9f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| import type { Redis } from "ioredis"; | ||
|
|
||
| export default function chat(bullmq: Redis) { | ||
| const queue = createQueue<Job>(name, attempts, bullmq); |
There was a problem hiding this comment.
This queue inherits removeOnComplete: true from server/workers/queue.ts, so the WhatsApp message ID stops providing deduplication immediately after processing. If a multi-sender webhook partially enqueues and then returns 500 because another enqueue failed—or if the 200 response is lost—a later webhook retry can re-enqueue an already completed message and send the user a duplicate reply. Retain completed jobs for the webhook retry window or store processed message IDs separately.
Useful? React with 👍 / 👎.
GitHub Actions
Inactive
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 885d52f342
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| v.array( | ||
| v.object({ | ||
| id: v.string(), | ||
| from_user_id: v.string(), |
There was a problem hiding this comment.
For actual WhatsApp Cloud API message callbacks, contacts identify the sender with wa_id and messages use from, but this schema requires user_id and from_user_id. Because the Valibot middleware runs before parse, every normal inbound text webhook is rejected with 400 bad chat; the tests currently mirror the incorrect field names instead of the external contract.
Useful? React with 👍 / 👎.
| body: JSON.stringify({ | ||
| messaging_product: "whatsapp", | ||
| recipient_type: "individual", | ||
| recipient, |
There was a problem hiding this comment.
to field
WhatsApp's Cloud API expects the destination number in the to request field; recipient is not the corresponding send-message field. Consequently every outbound request is rejected by the Graph API, and the worker exhausts its retries without delivering any reply.
Useful? React with 👍 / 👎.
| await whatsapp.send(data.from, text); | ||
| await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX"); |
There was a problem hiding this comment.
If WhatsApp accepts the message but the response is lost, the worker terminates, or the following Redis write fails, the BullMQ process rejects and retries the entire job up to ten times, including whatsapp.send. This can send the same reply repeatedly; persist an idempotent delivery state or otherwise keep post-send bookkeeping failures from replaying the external side effect.
Useful? React with 👍 / 👎.
GitHub Actions
Inactive
GitHub Actions
Inactive
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1ebf82c1c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
|
|
||
| function verify(body: string, signature?: string, secret?: string) { | ||
| if (!secret) return true; |
There was a problem hiding this comment.
If the Secret Manager version exists but contains an empty value, secret() returns "", so this branch accepts every request even when the signature header is missing. Because the chat Cloud Run service is public, that misconfiguration lets anyone submit schema-valid payloads and enqueue replies to attacker-selected numbers, consuming model/API quota and potentially sending spam; reserve unsigned mode for undefined or fail startup when the configured secret is empty.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 39943df5b7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| id: sender.id, | ||
| contact: sender.contact, | ||
| from: sender.from, | ||
| text: [sender, ...tail].map(({ text }) => text).join("\n"), |
There was a problem hiding this comment.
When a webhook batch contains multiple messages from one sender, the queued text includes every message but the job ID records only sender.id. If a retry delivers a tail message separately while the merged job is still retained, that tail ID is unknown and produces a duplicate reply; conversely, a batch whose first message already has an active job silently discards any new tail messages when BullMQ returns the existing job. Enqueue messages independently or reserve every included message ID before acknowledging the batch.
Useful? React with 👍 / 👎.
| import * as v from "valibot"; | ||
|
|
||
| import { attempts, name, type Job } from "./job"; | ||
| import sentry from "../../instrument.cjs"; |
There was a problem hiding this comment.
In the packaged worker, this local import is bundled into dist/workers/chat/bin.cjs, so evaluating it executes init(config) from server/instrument.cjs; the Docker entrypoint already preloads that same instrumentation with --require=./instrument.cjs before launching the bundle. Chat workers therefore initialize the Node SDK twice, replacing the active client and potentially installing logging/profiling instrumentation more than once. Move the reusable options into a side-effect-free module instead of importing the initializer.
Useful? React with 👍 / 👎.
| ); | ||
|
|
||
| const model = "anthropic/claude-sonnet-5"; | ||
| const ttl = 3600; |
There was a problem hiding this comment.
ResponseCache passes this TTL to the in-memory server cache in milliseconds, so 3600 keeps a translated script for only 3.6 seconds. Since messages from the same locale will ordinarily arrive farther apart than that, the worker repeatedly invokes the translator model—twice for every non-English help response—despite installing a response cache, adding avoidable latency and model cost. Set the value to the intended duration in milliseconds.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 5
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 16d42616-c862-4d92-bdff-ba3ee43ae201
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (6)
package.jsonpnpm-workspace.yamlserver/package.jsonserver/test/workers/chat.eval.tsserver/test/workers/chat.test.tsserver/workers/chat/worker.ts
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
| const model = { id: "anthropic/claude-sonnet-5", apiKey: env.ANTHROPIC_API_KEY } as const; | ||
| const { reply } = chat(env.ANTHROPIC_API_KEY ?? ""); |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fail fast when ANTHROPIC_API_KEY is missing.
Line 12 substitutes "" for a missing key, and Line 11 passes undefined to the judge model. The script then runs every case and fails on each one with a provider authentication error. Validate the variable once at startup so the failure names the missing configuration.
♻️ Proposed fix
-const model = { id: "anthropic/claude-sonnet-5", apiKey: env.ANTHROPIC_API_KEY } as const;
-const { reply } = chat(env.ANTHROPIC_API_KEY ?? "");
+const apiKey = parse(pipe(string(), nonEmpty()), env.ANTHROPIC_API_KEY);
+const model = { id: "anthropic/claude-sonnet-5", apiKey } as const;
+const { reply } = chat(apiKey);Add the valibot import:
import { nonEmpty, parse, pipe, string } from "valibot";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const model = { id: "anthropic/claude-sonnet-5", apiKey: env.ANTHROPIC_API_KEY } as const; | |
| const { reply } = chat(env.ANTHROPIC_API_KEY ?? ""); | |
| const apiKey = parse(pipe(string(), nonEmpty()), env.ANTHROPIC_API_KEY); | |
| const model = { id: "anthropic/claude-sonnet-5", apiKey } as const; | |
| const { reply } = chat(apiKey); |
| const ok = | ||
| judged.score === 1 && tools.length === 1 && tools[0] === called && dropped.length === 0 && broken.length === 0; | ||
| if (!ok) failed += 1; | ||
| const why = [ | ||
| tools.join(", ") !== called && `expected it to call ${called} — it called ${tools.join(", ") || "no tools"}`, | ||
| ...dropped.map((fragment) => `expected the reply to contain ${JSON.stringify(fragment)} — it does not`), | ||
| ...broken.map((url) => `expected ${url} exactly once — it appears ${text.split(url).length - 1} times`), | ||
| judged.score !== 1 && `rubric not satisfied — ${judged.reason?.replaceAll("\n", " ")}`, | ||
| ].filter((entry) => typeof entry === "string"); |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
Align the tool check between ok and why.
Line 218 marks a case failed when tools.length !== 1. Line 221 emits the explanation only when tools.join(", ") !== called. These conditions agree for the current cases, but they can diverge. Reuse one derived value so the report always explains a tool mismatch.
♻️ Proposed fix
+ const calledOnce = tools.length === 1 && tools[0] === called;
const ok =
- judged.score === 1 && tools.length === 1 && tools[0] === called && dropped.length === 0 && broken.length === 0;
+ judged.score === 1 && calledOnce && dropped.length === 0 && broken.length === 0;
if (!ok) failed += 1;
const why = [
- tools.join(", ") !== called && `expected it to call ${called} — it called ${tools.join(", ") || "no tools"}`,
+ !calledOnce && `expected it to call ${called} — it called ${tools.join(", ") || "no tools"}`,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const ok = | |
| judged.score === 1 && tools.length === 1 && tools[0] === called && dropped.length === 0 && broken.length === 0; | |
| if (!ok) failed += 1; | |
| const why = [ | |
| tools.join(", ") !== called && `expected it to call ${called} — it called ${tools.join(", ") || "no tools"}`, | |
| ...dropped.map((fragment) => `expected the reply to contain ${JSON.stringify(fragment)} — it does not`), | |
| ...broken.map((url) => `expected ${url} exactly once — it appears ${text.split(url).length - 1} times`), | |
| judged.score !== 1 && `rubric not satisfied — ${judged.reason?.replaceAll("\n", " ")}`, | |
| ].filter((entry) => typeof entry === "string"); | |
| const calledOnce = tools.length === 1 && tools[0] === called; | |
| const ok = | |
| judged.score === 1 && calledOnce && dropped.length === 0 && broken.length === 0; | |
| if (!ok) failed += 1; | |
| const why = [ | |
| !calledOnce && `expected it to call ${called} — it called ${tools.join(", ") || "no tools"}`, | |
| ...dropped.map((fragment) => `expected the reply to contain ${JSON.stringify(fragment)} — it does not`), | |
| ...broken.map((url) => `expected ${url} exactly once — it appears ${text.split(url).length - 1} times`), | |
| judged.score !== 1 && `rubric not satisfied — ${judged.reason?.replaceAll("\n", " ")}`, | |
| ].filter((entry) => typeof entry === "string"); |
| const observed = createChatWorker({ | ||
| anthropicKey: "anthropic", | ||
| bullmq: dedicated, | ||
| whatsapp, | ||
| }); | ||
| const listener = listen.mock.calls.find(([event]) => event === "error")?.[1]; | ||
| if (!listener) throw new Error("missing error listener"); | ||
| listener(new Error("socket closed")); | ||
| expect(captureException).toHaveBeenCalledExactlyOnceWith(new Error("socket closed")); | ||
| await observed.close(); | ||
| await dedicated.quit(); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win
Await observed.ready before you close the worker.
The test at Line 229 awaits observed.ready before close(). This test does not. observed.close() and dedicated.quit() can run while the BullMQ worker is still connecting, which makes the test flaky and can produce an unhandled rejection.
♻️ Proposed fix
const listener = listen.mock.calls.find(([event]) => event === "error")?.[1];
if (!listener) throw new Error("missing error listener");
listener(new Error("socket closed"));
expect(captureException).toHaveBeenCalledExactlyOnceWith(new Error("socket closed"));
+ await observed.ready;
await observed.close();
await dedicated.quit();📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const observed = createChatWorker({ | |
| anthropicKey: "anthropic", | |
| bullmq: dedicated, | |
| whatsapp, | |
| }); | |
| const listener = listen.mock.calls.find(([event]) => event === "error")?.[1]; | |
| if (!listener) throw new Error("missing error listener"); | |
| listener(new Error("socket closed")); | |
| expect(captureException).toHaveBeenCalledExactlyOnceWith(new Error("socket closed")); | |
| await observed.close(); | |
| await dedicated.quit(); | |
| const observed = createChatWorker({ | |
| anthropicKey: "anthropic", | |
| bullmq: dedicated, | |
| whatsapp, | |
| }); | |
| const listener = listen.mock.calls.find(([event]) => event === "error")?.[1]; | |
| if (!listener) throw new Error("missing error listener"); | |
| listener(new Error("socket closed")); | |
| expect(captureException).toHaveBeenCalledExactlyOnceWith(new Error("socket closed")); | |
| await observed.ready; | |
| await observed.close(); | |
| await dedicated.quit(); |
| await whatsapp.send(data.from, text); | ||
| await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX"); |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Record the sender before sending the message.
whatsapp.send runs before the whatsapp:seen: write. If the write fails, or if v.parse(v.string(), id) throws because id is undefined, the job fails after the message was already delivered. The worker then retries and sends a second message to the same person.
Write the key first, and gate the send on the NX result so a retry does not deliver a duplicate.
🐛 Proposed fix
- async process({ data, id }) {
- const seen = await bullmq.exists(`whatsapp:seen:${data.from}`);
- const { text } = await reply(data.text, {
- requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", seen === 1]]),
- });
- await whatsapp.send(data.from, text);
- await bullmq.set(`whatsapp:seen:${data.from}`, v.parse(v.string(), id), "NX");
- },
+ async process({ data, id }) {
+ const key = `whatsapp:seen:${data.from}`;
+ const first = await bullmq.set(key, v.parse(v.string(), id), "EX", seenTtl, "NX");
+ const { text } = await reply(data.text, {
+ requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", first === null]]),
+ });
+ await whatsapp.send(data.from, text);
+ },| Promise.all( | ||
| scripts[id].blocks.map(async ({ text, url }) => | ||
| [ | ||
| language === "en" || language.startsWith("en-") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Compare the locale without case sensitivity.
The model supplies locale, and the schema at Line 136 accepts any [\w-]{2,35} value. If the model returns EN or EN-US, this check fails and the English script goes to the translator. The person then receives a re-worded English reply and the process pays for an extra model call.
♻️ Proposed fix
- language === "en" || language.startsWith("en-")
+ language.toLowerCase() === "en" || language.toLowerCase().startsWith("en-")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| language === "en" || language.startsWith("en-") | |
| language.toLowerCase() === "en" || language.toLowerCase().startsWith("en-") |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d7544f97-96ca-45f6-b4f9-d95dded97c45
📒 Files selected for processing (8)
infra/Pulumi.base-sepolia.yamlinfra/Pulumi.sandbox.yamlinfra/utils/modules.tsserver/hooks/bin/chat.tsserver/test/hooks/bin.test.tsserver/test/workers/bin.test.tsserver/vitest.config.mtsserver/workers/chat/bin.ts
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
| Promise.all([ | ||
| secret("redis-url", secrets).then((redisUrl) => connect(redisUrl)), | ||
| Promise.resolve(parse(pipe(string("whatsapp id"), nonEmpty("whatsapp id")), env.WHATSAPP_PHONE_NUMBER_ID)), | ||
| secret("chat-whatsapp-app-secret", secrets), | ||
| secret("chat-whatsapp-verify-token", secrets), | ||
| ]).then(([bullmq, whatsappFrom, whatsappSecret, whatsappVerifyToken]) => | ||
| own( | ||
| createChatHook({ bullmq, whatsappFrom, whatsappSecret, whatsappVerifyToken }), | ||
| () => bullmq.quit(), | ||
| () => secrets.close(), | ||
| ), | ||
| ), |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
printf '%s\n' '--- repository conventions ---'
find /tmp/coderabbit-repo-knowledge/exactly-exa-f26ca19b -mindepth 2 -maxdepth 2 -type f -name '*.md' -print -exec sh -c 'head -80 "$1"' _ {} \;
printf '%s\n' '--- target files ---'
sed -n '1,120p' server/hooks/bin/chat.ts
sed -n '1,120p' server/workers/chat/bin.ts
printf '%s\n' '--- directly bound symbols ---'
rg -n --glob '*.{ts,tsx,js,jsx}' 'function own|const own|export .*own|function secret|const secret|export .*secret|function connect|const connect|export .*connect|function parse|export .*parse' serverRepository: exactly/exa
Length of output: 49191
🏁 Script executed:
printf '%s\n' '--- supervision and ownership contracts ---'
sed -n '1,180p' server/supervise.ts
sed -n '1,140p' server/utils/secret.ts
sed -n '1,130p' server/workers/worker.ts
printf '%s\n' '--- analogous startup flows ---'
sed -n '1,100p' server/workers/hook/bin.ts
sed -n '1,100p' server/workers/refund/bin.ts
rg -n --glob 'server/**/*.ts' 'Promise\.all\(|\.finally\(|secrets\.close\(|bullmq\.quit\(' serverRepository: exactly/exa
Length of output: 19009
Close clients on partial startup failure.
If a later secret() call rejects after connect() resolves, own() never registers bullmq.quit() or secrets.close(). Add rejection cleanup in both startup files. Add a test that asserts each client closes once.
📍 Affects 2 files
server/hooks/bin/chat.ts#L14-L25(this comment)server/workers/chat/bin.ts#L16-L27
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d314f1472b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| crema: ["redis-address", "redis-password", "redis-username"], | ||
| services: { | ||
| chat: { | ||
| env: { WHATSAPP_PHONE_NUMBER_ID: "whatsappPhoneNumberId" }, |
There was a problem hiding this comment.
Adding this environment mapping makes infra/index.ts call config.require("whatsappPhoneNumberId") while constructing both chat modules for every stack. Only the sandbox and base-sepolia stack files define that key; the checked infra/Pulumi.base.yaml and infra/Pulumi.production.yaml files do not, so their deployment workflows now fail during pulumi up before any resources can be updated. Add the configuration to those stacks or omit the chat modules where they are not intended to run.
Useful? React with 👍 / 👎.
| text: [sender, ...tail].map(({ text }) => text).join("\n"), | ||
| }) | ||
| .catch((error: unknown) => { | ||
| captureException(error, { extra: { sender }, tags: { job: "chat", queue: "chat" } }); |
There was a problem hiding this comment.
When Redis rejects an enqueue, this sends the complete sender object to Sentry, including the customer's WhatsApp identifier, profile name, and free-form message text. During a queue outage every attempted support message can therefore be copied into the error-monitoring system and retained outside the chat path; users may include credentials or financial details in those messages. Report only non-content identifiers needed to diagnose the queue failure.
Useful? React with 👍 / 👎.
| criteria: rubric.map((description) => ({ description })), | ||
| }).run({ input, output: text }); | ||
| const executions = Object.values(judged.judge ?? {}).flatMap((step) => step.executions); | ||
| const agentUsage = { input: totalUsage.inputTokens ?? 0, output: totalUsage.outputTokens ?? 0 }; |
There was a problem hiding this comment.
For every non-English evaluation case, the tool independently calls translator.generate once for the welcome script or twice for the help script, but those results are reduced to their text fields and their usage is discarded. Consequently this totalUsage accounts only for the outer chat-agent call, so the printed agent-token totals materially undercount the multilingual cases and cannot be used to compare their actual model consumption. Accumulate the translator results' usage alongside the outer result.
Useful? React with 👍 / 👎.
| "You also get access to a dollar account in the US, all 100% free.", | ||
| "Create your account and activate your card here:", | ||
| ].join("\n"), | ||
| url: appOrigin, |
There was a problem hiding this comment.
In the production deployment, server-production.yaml identifies the app domain as web.exactly.app, but the Pulumi stack contains no exa:domain, so infra/index.ts gives this worker the fallback APP_DOMAIN=production.exactly.app. Since appOrigin is embedded here in every welcome reply, production users receive an account-creation URL for the wrong host; the support script has the same problem. Persist or pass the workflow's production domain into the Pulumi deployment.
Useful? React with 👍 / 👎.
| }, | ||
| { | ||
| text: "If you don't find the answer there, write to us from the support chat inside the Exa app:", | ||
| url: `${appOrigin}/?support`, |
There was a problem hiding this comment.
For returning users this advertises /?support as the way to open the in-app support chat, but a repo-wide search of the route parameter consumers and the Intercom helpers finds no code that reads a support query parameter or calls present() in response to it. Clicking the link therefore only opens the normal root route and never presents the promised chat. Add a root/deep-link handler for this parameter or link to a route that already opens support.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d76b0ff3a0
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const seen = await bullmq.exists(`whatsapp:seen:${data.from}`); | ||
| const { text } = await reply(data.text, { | ||
| requestContext: new RequestContext<InferPublicSchema<typeof context>>([["seen", seen === 1]]), | ||
| }); | ||
| await whatsapp.send(data.from, text); |
There was a problem hiding this comment.
When an earlier job for a sender fails transiently, BullMQ delays its retry and can process a later job from that sender first. The later job then observes seen as false, sends the welcome response, and records the sender; when the older job retries, it observes seen as true and sends the help response afterward, reversing the conversation and changing the older job's response between attempts. Serialize jobs by sender or otherwise prevent a delayed predecessor from being overtaken.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c8f477c42
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const development = stack === "localhost"; | ||
|
|
||
| init({ | ||
| /** @type {import("@sentry/node").NodeOptions} */ |
There was a problem hiding this comment.
This newly added type comment is JSDoc, which the repository-wide comment convention explicitly prohibits; only suppression annotations, cspell:ignore, and TODO-style markers are permitted. Restructure the configuration so its type is inferred or checked without adding JSDoc to this entrypoint.
AGENTS.md reference: AGENTS.md:L94-L98
Useful? React with 👍 / 👎.
co-authored-by: danilo neves cruz <cruzdanilo@gmail.com>
2 participants
Summary by CodeRabbit