feat: improve multi-step agentic workflows and add model benchmark suite - #506
Merged
Conversation
- Hide markdown-to-blocks and create-block-content from AI tool list (ai_hidden meta) — models were calling these instead of create-post, producing converted blocks that never got saved - Auto-detect markdown in create-post/update-post content and convert to Gutenberg blocks server-side via MarkdownToBlocks::convert() - Add featured_image_id parameter to create-post and update-post so models can set featured images from import-stock-image results - Add composite create-post-with-image ability that creates a post AND imports a stock image as featured image in a single tool call - Add ToolResultTruncator to prevent context bloat in long workflows — truncates large tool results (plugin lists, DB queries, HTML) before adding to conversation history while keeping full results in logs - Add cross-reference hint to import-stock-image results pointing models to use attachment_id as featured_image_id - Improve system prompt with explicit Content Creation section and instruction to call all needed tools in one response - Improve create-post tool description to emphasize it as the PRIMARY content creation tool with markdown auto-conversion - Add model benchmark suite (tests/benchmark/model-benchmark.mjs) that tests 7 WP admin prompts against synthetic.new models with automated scoring on tool selection, content quality, and argument validity Benchmark results: average score improved from 54% to 77% across 6 models on 7 WordPress admin operation prompts. Multi-step post+image task improved from 46% to 80%+ for all models.
📝 WalkthroughWalkthroughAdds featured-image support and markdown-to-blocks conversion for post abilities, hides specific block abilities from the AI, truncates large tool results before appending history, updates system prompts, and adds a Node.js model benchmarking script plus related ignore and TODO updates. Changes
Sequence Diagram(s)sequenceDiagram
participant Model
participant AgentLoop
participant Tool as Tool/Ability
participant Truncator as ToolResultTruncator
participant History
Model->>AgentLoop: requests action / calls tool
AgentLoop->>Tool: invoke tool (ability)
Tool-->>AgentLoop: returns large result
AgentLoop->>Truncator: truncate(result, tool_name)
Truncator-->>AgentLoop: truncated result
AgentLoop->>History: append truncated result
AgentLoop-->>Model: deliver truncated tool response
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can scan for known vulnerabilities in your dependencies using OSV Scanner.OSV Scanner will automatically detect and report security vulnerabilities in your project's dependencies. No additional configuration is required. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
includes/Abilities/StockImageAbilities.php (1)
195-201: Output schema doesn't include the newtipfield.The
output_schemaregistered at lines 64-73 only listsattachment_id,url,alt,title, anderror. For schema completeness, consider addingtipto the output schema so documentation and validators reflect the actual return structure.📝 Suggested schema update (lines 64-73)
'output_schema' => [ 'type' => 'object', 'properties' => [ 'attachment_id' => [ 'type' => 'integer' ], 'url' => [ 'type' => 'string' ], 'alt' => [ 'type' => 'string' ], 'title' => [ 'type' => 'string' ], 'error' => [ 'type' => 'string' ], + 'tip' => [ 'type' => 'string' ], ], ],🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@includes/Abilities/StockImageAbilities.php` around lines 195 - 201, The output schema in StockImageAbilities.php currently registers keys ('attachment_id','url','alt','title','error') but the function that returns the response (the array including 'attachment_id','url','alt','title','tip') adds a 'tip' field; update the registered output_schema (the array defined around output_schema) to include 'tip' so the schema matches the actual return shape and validators/docs remain correct—locate the output_schema declaration near the top (around lines where output_schema is defined) and add 'tip' as a string field consistent with the other entries.tests/benchmark/model-benchmark.mjs (1)
624-646: API key fallback path may fail silently.
getApiKey()reads credentials from a hardcoded path using regex. If the file exists but has a different format or key name, the function silently falls through toprocess.exit(1)without useful diagnostics. The empty catch block at line 639 swallows all errors.🛠️ Suggested improvement
try { const creds = readFileSync( `${ process.env.HOME }/.config/aidevops/tenants/default/credentials.sh`, 'utf8' ); const match = creds.match( /SYNTHETIC_NEW_API_KEY="([^"]+)"/ ); if ( match ) { return match[ 1 ]; } - } catch { - // ignore + console.warn( 'Credentials file found but SYNTHETIC_NEW_API_KEY not matched' ); + } catch ( err ) { + // File not found is expected; other errors might be useful to log + if ( err.code !== 'ENOENT' ) { + console.warn( 'Error reading credentials file:', err.message ); + } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/benchmark/model-benchmark.mjs` around lines 624 - 646, The getApiKey function currently swallows errors and gives no diagnostics when the credentials file exists but doesn't match the expected pattern; update getApiKey to log useful diagnostics: inside the catch around readFileSync include the caught error in processLogger.error or console.error (include the file path and error.message), and if the file is read but the regex (SYNTHETIC_NEW_API_KEY) doesn't match, log the file path and a short excerpt or a message stating the key was not found so users know the file format differs before calling process.exit(1); reference getApiKey, readFileSync, credentials.sh and the SYNTHETIC_NEW_API_KEY regex when applying the changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@includes/Core/ToolResultTruncator.php`:
- Around line 81-97: The truncation logic inside the ToolResultTruncator for the
case 'gratis-ai-agent/get-plugins' is extracting non-existent fields "status"
and "slug"; update the array_map to pull the real plugin fields returned by
WordPressAbilities (use 'active' and 'file' instead of 'status' and 'slug')
while preserving 'name', and keep setting 'total' and '_truncated' based on
self::MAX_ARRAY_ITEMS; locate the switch case for 'gratis-ai-agent/get-plugins'
in ToolResultTruncator.php and replace references to 'status'/'slug' with
'active'/'file' accordingly so plugin metadata isn't lost.
In `@tests/benchmark/model-benchmark.mjs`:
- Around line 947-983: In the 'image_relevance' case where toolCalls for
'ai-agent/import-stock-image' are inspected, the code reads args.query but the
schema uses args.keyword; update the parsing to use const query = ((args.keyword
|| args.query || '')).toLowerCase() (or otherwise prefer args.keyword with a
fallback to args.query) so relevance detection works, keeping the existing
imgScore and weight logic and preserving the try/catch around JSON.parse for the
same function name checks.
- Around line 130-151: Update the function schema for
"ai-agent/import-stock-image" to match StockImageAbilities.php: rename the
parameter "query" to "keyword", add "width" and "height" properties (both
integers) under properties, and ensure "keyword" remains in the required array;
adjust any enum/description text as needed so the function signature and
parameters (keyword, width, height) align with the implementation used by the
import-stock-image ability.
- Around line 67-74: The benchmark JSON schema in model-benchmark.mjs is out of
sync: replace the featured_image_url string property with featured_image_id as
an integer and update the required array to match the implementation in
PostAbilities.php (which only requires 'title'); specifically, change the
property name and type to featured_image_id (type: 'integer') and modify
required to [ 'title' ] so the test schema aligns with the create-post schema
defined in PostAbilities.php.
- Around line 82-104: The `update-post` benchmark schema is missing fields and
an enum value: update the parameters object for the `update-post` schema to add
properties featured_image_id (integer), meta (object), and site_url (string),
and include 'future' in the status enum; ensure required still contains
'post_id' and the new properties are added under properties alongside post_id,
title, content, status, excerpt, categories, and tags so the schema matches the
implementation.
---
Nitpick comments:
In `@includes/Abilities/StockImageAbilities.php`:
- Around line 195-201: The output schema in StockImageAbilities.php currently
registers keys ('attachment_id','url','alt','title','error') but the function
that returns the response (the array including
'attachment_id','url','alt','title','tip') adds a 'tip' field; update the
registered output_schema (the array defined around output_schema) to include
'tip' so the schema matches the actual return shape and validators/docs remain
correct—locate the output_schema declaration near the top (around lines where
output_schema is defined) and add 'tip' as a string field consistent with the
other entries.
In `@tests/benchmark/model-benchmark.mjs`:
- Around line 624-646: The getApiKey function currently swallows errors and
gives no diagnostics when the credentials file exists but doesn't match the
expected pattern; update getApiKey to log useful diagnostics: inside the catch
around readFileSync include the caught error in processLogger.error or
console.error (include the file path and error.message), and if the file is read
but the regex (SYNTHETIC_NEW_API_KEY) doesn't match, log the file path and a
short excerpt or a message stating the key was not found so users know the file
format differs before calling process.exit(1); reference getApiKey,
readFileSync, credentials.sh and the SYNTHETIC_NEW_API_KEY regex when applying
the changes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: bb607eb4-0746-4b3d-b4eb-f5f7827020b4
📒 Files selected for processing (7)
.gitignoreincludes/Abilities/BlockAbilities.phpincludes/Abilities/PostAbilities.phpincludes/Abilities/StockImageAbilities.phpincludes/Core/AgentLoop.phpincludes/Core/ToolResultTruncator.phptests/benchmark/model-benchmark.mjs
Comment on lines
+81
to
+97
| case 'gratis-ai-agent/get-plugins': | ||
| if ( isset( $result['plugins'] ) && is_array( $result['plugins'] ) ) { | ||
| $total = count( $result['plugins'] ); | ||
| $result['plugins'] = array_map( | ||
| function ( $plugin ) { | ||
| return [ | ||
| 'name' => $plugin['name'] ?? $plugin['slug'] ?? '', | ||
| 'status' => $plugin['status'] ?? '', | ||
| 'slug' => $plugin['slug'] ?? '', | ||
| ]; | ||
| }, | ||
| array_slice( $result['plugins'], 0, self::MAX_ARRAY_ITEMS ) | ||
| ); | ||
| $result['total'] = $total; | ||
| $result['_truncated'] = $total > self::MAX_ARRAY_ITEMS; | ||
| } | ||
| break; |
There was a problem hiding this comment.
Field mismatch: actual plugin data uses active and file, not status and slug.
Per WordPressAbilities.php (lines 175-200), get-plugins returns plugins with fields file, name, version, description, author, and active. The truncator tries to extract status and slug which don't exist in the actual response, resulting in empty strings and lost data.
🔧 Proposed fix
case 'gratis-ai-agent/get-plugins':
if ( isset( $result['plugins'] ) && is_array( $result['plugins'] ) ) {
$total = count( $result['plugins'] );
$result['plugins'] = array_map(
function ( $plugin ) {
return [
- 'name' => $plugin['name'] ?? $plugin['slug'] ?? '',
- 'status' => $plugin['status'] ?? '',
- 'slug' => $plugin['slug'] ?? '',
+ 'name' => $plugin['name'] ?? '',
+ 'active' => $plugin['active'] ?? false,
+ 'file' => $plugin['file'] ?? '',
];
},
array_slice( $result['plugins'], 0, self::MAX_ARRAY_ITEMS )
);
$result['total'] = $total;
$result['_truncated'] = $total > self::MAX_ARRAY_ITEMS;
}
break;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@includes/Core/ToolResultTruncator.php` around lines 81 - 97, The truncation
logic inside the ToolResultTruncator for the case 'gratis-ai-agent/get-plugins'
is extracting non-existent fields "status" and "slug"; update the array_map to
pull the real plugin fields returned by WordPressAbilities (use 'active' and
'file' instead of 'status' and 'slug') while preserving 'name', and keep setting
'total' and '_truncated' based on self::MAX_ARRAY_ITEMS; locate the switch case
for 'gratis-ai-agent/get-plugins' in ToolResultTruncator.php and replace
references to 'status'/'slug' with 'active'/'file' accordingly so plugin
metadata isn't lost.
Comment on lines
+67
to
+74
| featured_image_url: { | ||
| type: 'string', | ||
| description: 'URL of image to set as featured image.', | ||
| }, | ||
| }, | ||
| required: [ 'title', 'content' ], | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Schema mismatch: featured_image_url should be featured_image_id (integer).
The benchmark declares featured_image_url as a string (lines 67-70), but the actual create-post schema in PostAbilities.php (lines 133-136) uses featured_image_id as an integer. This also requires ['title', 'content'] but the actual implementation only requires ['title'].
🔧 Proposed fix
tags: {
type: 'array',
items: { type: 'string' },
description: 'Tag names to assign.',
},
- featured_image_url: {
- type: 'string',
- description: 'URL of image to set as featured image.',
+ featured_image_id: {
+ type: 'integer',
+ description: 'Attachment ID to set as the featured image (e.g. from import-stock-image result).',
+ },
+ meta: {
+ type: 'object',
+ description: 'Key-value pairs of post meta to set.',
+ },
+ site_url: {
+ type: 'string',
+ description: 'Subsite URL for multisite. Omit for the main site.',
},
},
- required: [ 'title', 'content' ],
+ required: [ 'title' ],📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| featured_image_url: { | |
| type: 'string', | |
| description: 'URL of image to set as featured image.', | |
| }, | |
| }, | |
| required: [ 'title', 'content' ], | |
| }, | |
| }, | |
| tags: { | |
| type: 'array', | |
| items: { type: 'string' }, | |
| description: 'Tag names to assign.', | |
| }, | |
| featured_image_id: { | |
| type: 'integer', | |
| description: 'Attachment ID to set as the featured image (e.g. from import-stock-image result).', | |
| }, | |
| meta: { | |
| type: 'object', | |
| description: 'Key-value pairs of post meta to set.', | |
| }, | |
| site_url: { | |
| type: 'string', | |
| description: 'Subsite URL for multisite. Omit for the main site.', | |
| }, | |
| }, | |
| required: [ 'title' ], | |
| }, | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/benchmark/model-benchmark.mjs` around lines 67 - 74, The benchmark JSON
schema in model-benchmark.mjs is out of sync: replace the featured_image_url
string property with featured_image_id as an integer and update the required
array to match the implementation in PostAbilities.php (which only requires
'title'); specifically, change the property name and type to featured_image_id
(type: 'integer') and modify required to [ 'title' ] so the test schema aligns
with the create-post schema defined in PostAbilities.php.
Comment on lines
+82
to
+104
| parameters: { | ||
| type: 'object', | ||
| properties: { | ||
| post_id: { | ||
| type: 'integer', | ||
| description: 'The ID of the post to update.', | ||
| }, | ||
| title: { type: 'string', description: 'New title.' }, | ||
| content: { type: 'string', description: 'New content.' }, | ||
| status: { | ||
| type: 'string', | ||
| enum: [ 'draft', 'publish', 'pending', 'private', 'trash' ], | ||
| }, | ||
| excerpt: { type: 'string' }, | ||
| categories: { | ||
| type: 'array', | ||
| items: { type: 'string' }, | ||
| }, | ||
| tags: { type: 'array', items: { type: 'string' } }, | ||
| }, | ||
| required: [ 'post_id' ], | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Schema mismatch: update-post missing fields and status enum incomplete.
The benchmark's update-post schema is missing featured_image_id, meta, and site_url fields. The status enum also lacks 'future' which the actual implementation supports.
🔧 Proposed fix
status: {
type: 'string',
- enum: [ 'draft', 'publish', 'pending', 'private', 'trash' ],
+ enum: [ 'draft', 'publish', 'pending', 'private', 'future', 'trash' ],
},
excerpt: { type: 'string' },
categories: {
type: 'array',
items: { type: 'string' },
},
tags: { type: 'array', items: { type: 'string' } },
+ featured_image_id: {
+ type: 'integer',
+ description: 'Attachment ID to set as the featured image.',
+ },
+ meta: {
+ type: 'object',
+ description: 'Key-value pairs of post meta to update.',
+ },
+ site_url: {
+ type: 'string',
+ description: 'Subsite URL for multisite. Omit for the main site.',
+ },
},🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/benchmark/model-benchmark.mjs` around lines 82 - 104, The `update-post`
benchmark schema is missing fields and an enum value: update the parameters
object for the `update-post` schema to add properties featured_image_id
(integer), meta (object), and site_url (string), and include 'future' in the
status enum; ensure required still contains 'post_id' and the new properties are
added under properties alongside post_id, title, content, status, excerpt,
categories, and tags so the schema matches the implementation.
Comment on lines
+130
to
+151
| { | ||
| type: 'function', | ||
| function: { | ||
| name: 'ai-agent/import-stock-image', | ||
| description: | ||
| 'Search and import a stock image from Pexels/Unsplash into the WordPress media library.', | ||
| parameters: { | ||
| type: 'object', | ||
| properties: { | ||
| query: { | ||
| type: 'string', | ||
| description: 'Search query for the stock image.', | ||
| }, | ||
| orientation: { | ||
| type: 'string', | ||
| enum: [ 'landscape', 'portrait', 'square' ], | ||
| }, | ||
| }, | ||
| required: [ 'query' ], | ||
| }, | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Schema mismatch: query should be keyword for import-stock-image.
The benchmark uses query as the parameter name (line 139-141), but the actual StockImageAbilities.php implementation (lines 42-63) uses keyword. Additionally, the real schema includes width and height parameters that are missing here. This mismatch means benchmark scoring for image-related tasks won't accurately reflect real tool usage.
🔧 Proposed fix
{
type: 'function',
function: {
name: 'ai-agent/import-stock-image',
description:
'Search and import a stock image from Pexels/Unsplash into the WordPress media library.',
parameters: {
type: 'object',
properties: {
- query: {
+ keyword: {
type: 'string',
- description: 'Search query for the stock image.',
+ description: 'Search term for finding a relevant image (e.g. "dogs", "mountain landscape", "coffee shop")',
},
- orientation: {
+ site_url: {
type: 'string',
- enum: [ 'landscape', 'portrait', 'square' ],
+ description: 'Subsite URL to import into (e.g. "https://example.com/mysite"). Omit for the main site.',
+ },
+ width: {
+ type: 'integer',
+ description: 'Image width in pixels (default: 1200)',
+ },
+ height: {
+ type: 'integer',
+ description: 'Image height in pixels (default: 800)',
},
},
- required: [ 'query' ],
+ required: [ 'keyword' ],
},
},
},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| { | |
| type: 'function', | |
| function: { | |
| name: 'ai-agent/import-stock-image', | |
| description: | |
| 'Search and import a stock image from Pexels/Unsplash into the WordPress media library.', | |
| parameters: { | |
| type: 'object', | |
| properties: { | |
| query: { | |
| type: 'string', | |
| description: 'Search query for the stock image.', | |
| }, | |
| orientation: { | |
| type: 'string', | |
| enum: [ 'landscape', 'portrait', 'square' ], | |
| }, | |
| }, | |
| required: [ 'query' ], | |
| }, | |
| }, | |
| }, | |
| { | |
| type: 'function', | |
| function: { | |
| name: 'ai-agent/import-stock-image', | |
| description: | |
| 'Search and import a stock image from Pexels/Unsplash into the WordPress media library.', | |
| parameters: { | |
| type: 'object', | |
| properties: { | |
| keyword: { | |
| type: 'string', | |
| description: 'Search term for finding a relevant image (e.g. "dogs", "mountain landscape", "coffee shop")', | |
| }, | |
| site_url: { | |
| type: 'string', | |
| description: 'Subsite URL to import into (e.g. "https://example.com/mysite"). Omit for the main site.', | |
| }, | |
| width: { | |
| type: 'integer', | |
| description: 'Image width in pixels (default: 1200)', | |
| }, | |
| height: { | |
| type: 'integer', | |
| description: 'Image height in pixels (default: 800)', | |
| }, | |
| }, | |
| required: [ 'keyword' ], | |
| }, | |
| }, | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/benchmark/model-benchmark.mjs` around lines 130 - 151, Update the
function schema for "ai-agent/import-stock-image" to match
StockImageAbilities.php: rename the parameter "query" to "keyword", add "width"
and "height" properties (both integers) under properties, and ensure "keyword"
remains in the required array; adjust any enum/description text as needed so the
function signature and parameters (keyword, width, height) align with the
implementation used by the import-stock-image ability.
Comment on lines
+947
to
+983
| case 'image_relevance': { | ||
| let imgScore = 0; | ||
| for ( const tc of toolCalls ) { | ||
| if ( | ||
| tc.function?.name === 'ai-agent/import-stock-image' | ||
| ) { | ||
| try { | ||
| const args = JSON.parse( | ||
| tc.function?.arguments || '{}' | ||
| ); | ||
| const query = ( args.query || '' ).toLowerCase(); | ||
| const relevant = [ | ||
| 'medical', | ||
| 'healthcare', | ||
| 'health', | ||
| 'ai', | ||
| 'technology', | ||
| 'doctor', | ||
| 'hospital', | ||
| 'medicine', | ||
| 'digital health', | ||
| ]; | ||
| if ( | ||
| relevant.some( ( kw ) => | ||
| query.includes( kw ) | ||
| ) | ||
| ) | ||
| imgScore = 1; | ||
| else imgScore = 0.3; // At least tried. | ||
| } catch { | ||
| // ignore | ||
| } | ||
| } | ||
| } | ||
| score = imgScore * weight; | ||
| break; | ||
| } |
There was a problem hiding this comment.
Scoring checks args.query but actual schema uses keyword.
The image_relevance scoring logic reads args.query (line 957), but per the schema mismatch, models will actually provide args.keyword. This will cause image relevance scoring to always fail.
🔧 Proposed fix
case 'image_relevance': {
let imgScore = 0;
for ( const tc of toolCalls ) {
if (
tc.function?.name === 'ai-agent/import-stock-image'
) {
try {
const args = JSON.parse(
tc.function?.arguments || '{}'
);
- const query = ( args.query || '' ).toLowerCase();
+ const query = ( args.keyword || args.query || '' ).toLowerCase();
const relevant = [📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| case 'image_relevance': { | |
| let imgScore = 0; | |
| for ( const tc of toolCalls ) { | |
| if ( | |
| tc.function?.name === 'ai-agent/import-stock-image' | |
| ) { | |
| try { | |
| const args = JSON.parse( | |
| tc.function?.arguments || '{}' | |
| ); | |
| const query = ( args.query || '' ).toLowerCase(); | |
| const relevant = [ | |
| 'medical', | |
| 'healthcare', | |
| 'health', | |
| 'ai', | |
| 'technology', | |
| 'doctor', | |
| 'hospital', | |
| 'medicine', | |
| 'digital health', | |
| ]; | |
| if ( | |
| relevant.some( ( kw ) => | |
| query.includes( kw ) | |
| ) | |
| ) | |
| imgScore = 1; | |
| else imgScore = 0.3; // At least tried. | |
| } catch { | |
| // ignore | |
| } | |
| } | |
| } | |
| score = imgScore * weight; | |
| break; | |
| } | |
| case 'image_relevance': { | |
| let imgScore = 0; | |
| for ( const tc of toolCalls ) { | |
| if ( | |
| tc.function?.name === 'ai-agent/import-stock-image' | |
| ) { | |
| try { | |
| const args = JSON.parse( | |
| tc.function?.arguments || '{}' | |
| ); | |
| const query = ( args.keyword || args.query || '' ).toLowerCase(); | |
| const relevant = [ | |
| 'medical', | |
| 'healthcare', | |
| 'health', | |
| 'ai', | |
| 'technology', | |
| 'doctor', | |
| 'hospital', | |
| 'medicine', | |
| 'digital health', | |
| ]; | |
| if ( | |
| relevant.some( ( kw ) => | |
| query.includes( kw ) | |
| ) | |
| ) | |
| imgScore = 1; | |
| else imgScore = 0.3; // At least tried. | |
| } catch { | |
| // ignore | |
| } | |
| } | |
| } | |
| score = imgScore * weight; | |
| break; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/benchmark/model-benchmark.mjs` around lines 947 - 983, In the
'image_relevance' case where toolCalls for 'ai-agent/import-stock-image' are
inspected, the code reads args.query but the schema uses args.keyword; update
the parsing to use const query = ((args.keyword || args.query ||
'')).toLowerCase() (or otherwise prefer args.keyword with a fallback to
args.query) so relevance detection works, keeping the existing imgScore and
weight logic and preserving the try/catch around JSON.parse for the same
function name checks.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
TODO.md (1)
34-34: Prefer hyphenated compound adjective in heading.Use “High-Priority Bugs (P1)” for consistency with compound-adjective style.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@TODO.md` at line 34, Update the heading text "High Priority Bugs (P1)" to the hyphenated form "High-Priority Bugs (P1)" in TODO.md; locate the heading string "High Priority Bugs (P1)" and replace it with "High-Priority Bugs (P1)" to maintain compound-adjective style consistency.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@TODO.md`:
- Line 34: Update the heading text "High Priority Bugs (P1)" to the hyphenated
form "High-Priority Bugs (P1)" in TODO.md; locate the heading string "High
Priority Bugs (P1)" and replace it with "High-Priority Bugs (P1)" to maintain
compound-adjective style consistency.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ccddfe29-1283-4fc3-86c1-3ffe069c6477
📒 Files selected for processing (2)
.task-counterTODO.md
✅ Files skipped from review due to trivial changes (1)
- .task-counter
This was referenced Mar 20, 2026
This was referenced Mar 25, 2026
This was referenced Apr 6, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
markdown-to-blocksandcreate-block-contentare now hidden (ai_hiddenmeta) from the model's tool list. Models were calling these instead ofcreate-post, producing converted blocks that never got saved. The abilities still exist for internal/programmatic use.create-postandupdate-postnow detect markdown content (## headings, bold, - lists) and auto-convert to serialized Gutenberg blocks server-side. Models can write natural markdown instead of wrestling with block HTML.create-post-with-imageability — creates a post AND imports a stock image as featured image in a single tool call, eliminating the most common multi-step failure pattern.featured_image_idparameter added tocreate-postandupdate-postso models can set featured images fromimport-stock-imageresults.ToolResultTruncatorclass prevents context bloat in long workflows by truncating large tool results (plugin lists, DB queries, HTML) before adding to conversation history. Full results remain in the tool call log.import-stock-imageresults now include atipfield pointing models to useattachment_idasfeatured_image_id.tests/benchmark/model-benchmark.mjstests 7 WP admin prompts against synthetic.new models with automated scoring.Benchmark Results
Tested against 6 models on synthetic.new (GLM-4.7, GLM-4.7-Flash, Kimi-K2.5, Kimi-K2.5-NVFP4, MiniMax-M2.5, Nemotron-3-Super-120B):
Files Changed
includes/Abilities/PostAbilities.phpfeatured_image_id, markdown auto-convert, composite abilityincludes/Abilities/BlockAbilities.phpai_hiddenmeta onmarkdown-to-blocksandcreate-block-contentincludes/Core/AgentLoop.phpai_hiddenfilter, tool result truncation, improved system promptincludes/Core/ToolResultTruncator.phpincludes/Abilities/StockImageAbilities.phptipin resultstests/benchmark/model-benchmark.mjs.gitignoreSummary by CodeRabbit
New Features
Improvements
Chores