Skip to content

Commit 30bc3c2

Browse files
author
mcp-release-bot
committed
dynamic field spesicifcation
1 parent fb1bc15 commit 30bc3c2

File tree

4 files changed

+45
-26
lines changed

4 files changed

+45
-26
lines changed

src/mcp_as_a_judge/models.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,8 @@ class JudgeResponse(BaseModel):
5858
)
5959

6060
# Use a lazy default factory to avoid importing workflow at module import time
61-
def _default_workflow_guidance(): # type: ignore[no-redef]
61+
@staticmethod
62+
def _default_workflow_guidance() -> "WorkflowGuidance":
6263
from mcp_as_a_judge.workflow import WorkflowGuidance
6364

6465
return WorkflowGuidance(
@@ -261,6 +262,10 @@ class SystemVars(BaseModel):
261262
default="",
262263
description="Workflow guidance from task state to use as evaluation criteria (optional)",
263264
)
265+
plan_required_fields_json: str = Field(
266+
default="[]",
267+
description="JSON array of required fields for judge_coding_plan dynamic validation (optional)",
268+
)
264269

265270

266271
class JudgeCodingPlanUserVars(BaseModel):

src/mcp_as_a_judge/prompts/system/judge_coding_plan.md

Lines changed: 24 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -297,25 +297,31 @@ You must respond with a JSON object that matches this schema:
297297

298298
**CRITICAL: You MUST evaluate ALL requirements in ONE PASS. NO iterative discovery allowed.**
299299

300-
### STEP 1: CHECK REQUIRED FIELDS (Reject immediately if ANY missing)
301-
- [ ] `plan` field exists and non-empty
302-
- [ ] `design` field exists and non-empty
303-
- [ ] `research` field exists and non-empty
304-
- [ ] `problem_domain` field exists and non-empty
305-
- [ ] `problem_non_goals` array exists (can be empty)
306-
- [ ] `library_plan` array exists and non-empty
307-
- [ ] `internal_reuse_components` array exists (empty with greenfield note OK)
308-
309-
### STEP 2: CHECK CONDITIONAL FIELDS (Based on task metadata)
310-
- [ ] If `research_required=true``research_urls` array must exist and be non-empty
311-
- [ ] If `risk_assessment_required=true``identified_risks` AND `risk_mitigation_strategies` arrays must exist with 1:1 mapping
312-
- [ ] If `design_patterns_enforcement=true``design_patterns` array must exist with name/area objects
313-
314-
### STEP 3: APPROVE OR REJECT WITH COMPLETE FEEDBACK
315-
- **IF ALL FIELDS PRESENT**: Approve with brief positive feedback
316-
- **IF ANY FIELDS MISSING**: Reject with ALL missing items listed at once
300+
### DYNAMIC FIELD VALIDATION
301+
302+
The following fields are required based on the current task metadata:
303+
304+
{{ plan_required_fields_json }}
305+
306+
**VALIDATION STEPS:**
307+
308+
1. **Parse the field requirements** from the JSON specification above
309+
2. **Check each required field** (where `required: true`) exists and is non-empty
310+
3. **Check conditional fields** (where `conditional_on` is specified) only when their condition is met
311+
4. **Validate data types** match the specified `type` for each field
312+
5. **Approve or reject** with complete feedback listing ALL missing fields at once
313+
314+
### FIELD VALIDATION RULES:
315+
- **String fields**: Must exist and contain non-whitespace content
316+
- **Array fields**: Must exist as arrays (can be empty only if explicitly noted in description)
317+
- **Object arrays**: Must exist and contain properly structured objects as described
318+
- **Conditional fields**: Only validate when the specified task metadata condition is true
319+
320+
### APPROVAL CRITERIA:
321+
-**APPROVE** if all required and applicable conditional fields are populated with reasonable content
322+
-**REJECT** only if required or applicable conditional fields are missing or empty
317323
- **DO NOT**: Ask for additional details, implementation specifics, or code snippets
318-
- **DO NOT**: Discover new requirements not listed above
324+
- **DO NOT**: Discover new requirements beyond the specified field list
319325

320326
### SIMPLIFIED APPROVAL CRITERIA:
321327
**APPROVE** if all required/conditional fields are populated with reasonable content

src/mcp_as_a_judge/prompts/system/workflow_guidance.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -144,13 +144,12 @@ When recommending judge_coding_plan, you MUST:
144144
2. **Include ALL required fields** (both always-required and conditional based on task metadata)
145145
3. **Provide clear examples** for complex field types like library_plan and design_patterns
146146

147-
The plan_required_fields array will be automatically populated based on task metadata and should include:
148-
- Always required: plan, design, research, problem_domain, problem_non_goals, library_plan, internal_reuse_components
149-
- Conditional on research_required=true: research_urls
150-
- Conditional on risk_assessment_required=true: identified_risks, risk_mitigation_strategies
151-
- Conditional on design_patterns_enforcement=true: design_patterns
147+
The plan_required_fields array will be automatically populated based on task metadata. The dynamic validation system will:
148+
- Include always-required fields (plan, design, research, problem_domain, problem_non_goals, library_plan, internal_reuse_components)
149+
- Add conditional fields based on task metadata flags (research_urls, identified_risks, risk_mitigation_strategies, design_patterns)
150+
- Provide detailed field specifications with types, descriptions, and examples
152151

153-
**FAILURE TO POPULATE CONDITIONAL FIELDS WILL RESULT IN REJECTION**
152+
**FAILURE TO POPULATE REQUIRED OR APPLICABLE CONDITIONAL FIELDS WILL RESULT IN REJECTION**
154153

155154
### CRITICAL: judge_code_change Usage Rules
156155

src/mcp_as_a_judge/server.py

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,14 +39,14 @@
3939
DesignPattern,
4040
JudgeCodeChangeUserVars,
4141
JudgeCodingPlanUserVars,
42-
JudgeResponse,
4342
ResearchValidationResponse,
4443
ResearchValidationUserVars,
4544
SystemVars,
4645
WorkflowGuidance,
4746
)
4847
from mcp_as_a_judge.models.enhanced_responses import (
4948
EnhancedResponseFactory,
49+
JudgeResponse,
5050
TaskAnalysisResult,
5151
TaskCompletionResult,
5252
)
@@ -1292,11 +1292,20 @@ async def _evaluate_coding_plan(
12921292

12931293
workflow_guidance_text = "\n".join(guidance_parts)
12941294

1295+
# Generate plan required fields for dynamic validation
1296+
from mcp_as_a_judge.workflow.workflow_guidance import _generate_plan_required_fields
1297+
1298+
plan_required_fields = _generate_plan_required_fields(task_metadata)
1299+
plan_required_fields_json = json.dumps(
1300+
[field.model_dump() for field in plan_required_fields], indent=2
1301+
)
1302+
12951303
# Create system and user messages from templates
12961304
system_vars = SystemVars(
12971305
response_schema=json.dumps(JudgeResponse.model_json_schema()),
12981306
max_tokens=MAX_TOKENS,
12991307
workflow_guidance=workflow_guidance_text,
1308+
plan_required_fields_json=plan_required_fields_json,
13001309
)
13011310
user_vars = JudgeCodingPlanUserVars(
13021311
user_requirements=user_requirements,

0 commit comments

Comments
 (0)