Skip to content

Commit 7703ae5

Browse files
Kenoclaude
andcommitted
Support metadata in list_problems tool
Updated list_problems tool to handle both old (string list) and new (dict list with metadata) formats from LLMBenchSimple. Changes: - ListProblemsTool now checks if problems are Dicts or Strings - For Dict format, merges problem metadata with default fields - Maintains backward compatibility with string-only problem lists - Ensures all problems have id, name, description, category fields This enables LLMBenchSimple modules to provide rich metadata like enable_anthropic_api, custom descriptions, etc. through list_problems. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent cf0705c commit 7703ae5

File tree

1 file changed

+22
-6
lines changed

1 file changed

+22
-6
lines changed

src/tools/list_problems.jl

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,15 +31,31 @@ function ClaudeMCPTools.execute(tool::ListProblemsTool, params::Dict)
3131
try
3232
problems = Base.invokelatest(tool.list_fn)
3333

34-
# Format as JSON with problem list
35-
problems_json = Dict(
36-
"problems" => [Dict(
37-
"id" => p,
38-
"name" => p,
34+
# Handle two formats:
35+
# 1. Vector of Dicts (new format with metadata)
36+
# 2. Vector of Strings (old format, backward compatible)
37+
problems_list = if !isempty(problems) && problems[1] isa Dict
38+
# New format: problems already have metadata
39+
# Ensure each has at least an id, name, description, category
40+
[merge(
41+
Dict{String,Any}(
42+
"name" => get(p, "id", ""),
43+
"description" => "",
44+
"category" => "general"
45+
),
46+
p # Metadata from @bench overrides defaults
47+
) for p in problems]
48+
else
49+
# Old format: convert strings to dicts
50+
[Dict{String,Any}(
51+
"id" => string(p),
52+
"name" => string(p),
3953
"description" => "",
4054
"category" => "general"
4155
) for p in problems]
42-
)
56+
end
57+
58+
problems_json = Dict("problems" => problems_list)
4359

4460
return Dict(
4561
"content" => [Dict(

0 commit comments

Comments
 (0)