Skip to content

[fix : story] data unavailable due to wrong query_id - #1416

Open
socallmebertille wants to merge 2 commits into
mainfrom
fix/story_building_now_reject_object_block_with_wrong_query_id
Open

[fix : story] data unavailable due to wrong query_id#1416
socallmebertille wants to merge 2 commits into
mainfrom
fix/story_building_now_reject_object_block_with_wrong_query_id

Conversation

@socallmebertille

@socallmebertille socallmebertille commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Issue

Some LLM models take the liberty of assigning common English names to query_ids when querying data, and then passing them to stories to create blocks of charts, tables, or maps.

Solution

To prevent this issue from recurring, I added a validator that checks the format of the query ID and verifies it corresponds to an execute_sql result produced in the same chat. Mismatches are surfaced as non-blocking template_warnings so the agent fixes the reference before finishing.

Review in cubic

…nders object blocks that aren't linked to actual available data
@socallmebertille socallmebertille self-assigned this Aug 18, 2026
@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

🚀 Preview Deployment

URL https://pr-1416-af3db03.preview.getnao.io
Commit af3db03

⚠️ No LLM API keys configured - you'll see the API key setup flow when trying to chat.


Preview will be automatically removed when this PR is closed.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread apps/backend/src/services/story-template-validation.ts Outdated
Comment thread apps/backend/src/services/story-template-validation.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/shared/src/story-segments.ts">

<violation number="1" location="apps/shared/src/story-segments.ts:689">
P3: The rewritten `extractQueryIds` now routes every tag through `parseChartAttributes`, whose attribute regex requires `name=` with no surrounding whitespace and a quoted value. The previous regex accepted `query_id\s*=\s*"..."` (optional whitespace around `=` and before the quote) and also matched query ids in tags that never close with `>`. For references written as `<chart query_id = "q_1" ...>` or an unclosed `<chart query_id="q_1"`, the old code still recorded `q_1` (and would warn on it), while the new code silently drops it, so those references bypass the new validator. Generated tags use `query_id="..."` so it is an edge case, but it is a coverage regression in the very validator this PR adds.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

for (const tagRegex of [chartTagRegex('g'), tableTagRegex('g'), mapTagRegex('g')]) {
let match: RegExpExecArray | null;
while ((match = tagRegex.exec(code)) !== null) {
const { query_id } = parseChartAttributes(match[1]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The rewritten extractQueryIds now routes every tag through parseChartAttributes, whose attribute regex requires name= with no surrounding whitespace and a quoted value. The previous regex accepted query_id\s*=\s*"..." (optional whitespace around = and before the quote) and also matched query ids in tags that never close with >. For references written as <chart query_id = "q_1" ...> or an unclosed <chart query_id="q_1", the old code still recorded q_1 (and would warn on it), while the new code silently drops it, so those references bypass the new validator. Generated tags use query_id="..." so it is an edge case, but it is a coverage regression in the very validator this PR adds.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/shared/src/story-segments.ts, line 689:

<comment>The rewritten `extractQueryIds` now routes every tag through `parseChartAttributes`, whose attribute regex requires `name=` with no surrounding whitespace and a quoted value. The previous regex accepted `query_id\s*=\s*"..."` (optional whitespace around `=` and before the quote) and also matched query ids in tags that never close with `>`. For references written as `<chart query_id = "q_1" ...>` or an unclosed `<chart query_id="q_1"`, the old code still recorded `q_1` (and would warn on it), while the new code silently drops it, so those references bypass the new validator. Generated tags use `query_id="..."` so it is an edge case, but it is a coverage regression in the very validator this PR adds.</comment>

<file context>
@@ -683,10 +683,14 @@ function extractSeriesFromRawAttrs(attrString: string): ParsedChartBlock['series
+	for (const tagRegex of [chartTagRegex('g'), tableTagRegex('g'), mapTagRegex('g')]) {
+		let match: RegExpExecArray | null;
+		while ((match = tagRegex.exec(code)) !== null) {
+			const { query_id } = parseChartAttributes(match[1]);
+			if (query_id) {
+				ids.add(query_id);
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant