feat: enforce mandatory online research with List[str] URLs and collaborative workflow #11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
August 30, 2025 15:27
- Add required research_urls parameter to judge_coding_plan tool - AI assistants must now provide URLs visited during research - Enhanced validation for implementation approach research - Updated prompts to validate URL quality and relevance - Ensures research includes online investigation of best practices and libraries - Prevents reinventing the wheel by requiring evidence of research Breaking change: judge_coding_plan now requires research_urls parameter Benefits: - ✅ Forces AI assistants to do actual online research - ✅ Validates research quality through URL evidence - ✅ Ensures investigation of existing solutions and best practices - ✅ Improves implementation approach validation - ✅ All tests updated and passing (41/41)
…ibility - Make online research MANDATORY in docstring and prompts - AI assistants MUST perform online research and provide URLs - research_urls parameter optional for backward compatibility but strongly enforced - Enhanced prompts to REJECT submissions without research URLs - Clear instruction that online research is not optional - URLs should be comma-separated list demonstrating actual research - Prioritize existing solutions: current repo > well-known libraries > in-house Key changes: - ✅ MANDATORY online research requirement in all documentation - ✅ Backward compatible research_urls parameter (optional with empty default) - ✅ Enhanced validation to reject missing URLs as research failure - ✅ Clear guidance that online research is required, not optional - ✅ Instructs AI to actually DO research, not just ask for URLs - ✅ All tests passing (41/41) Fixes validation error while maintaining mandatory research requirement.
…borative workflow - Add mandatory minimum 3 URLs validation for research_urls parameter - Change research_urls from str to List[str] for proper URL handling - Add validation that rejects submissions with insufficient research URLs - Update docstring to emphasize collaborative workflow with user - Clarify that AI must 'collaborate with the user' and 'perform ONLINE research' - Restructure workflow order: requirements → online research → repo analysis → design → plan - Enhanced prompts to properly format URL lists with Jinja templating - All tests updated to provide minimum 3 URLs and use List[str] format Breaking changes: - research_urls parameter now requires List[str] instead of str - Minimum 3 URLs required (validates and rejects if insufficient) - Function signature maintains backward compatibility with None default Benefits: - ✅ Forces actual online research with evidence (minimum 3 URLs) - ✅ Prevents AI from working unilaterally - requires collaboration - ✅ Proper data structure for URL handling (List[str]) - ✅ Clear workflow prioritizing existing solutions over custom development - ✅ All tests passing with proper URL validation (41/41)
mcp-as-a-judge-semantic-release bot
pushed a commit
that referenced
this pull request
Aug 30, 2025
## [1.0.0](v0.1.8...v1.0.0) (2025-08-30) ### ⚠ BREAKING CHANGES * judge_coding_plan now requires research_urls parameter Benefits: - ✅ Forces AI assistants to do actual online research - ✅ Validates research quality through URL evidence - ✅ Ensures investigation of existing solutions and best practices - ✅ Improves implementation approach validation - ✅ All tests updated and passing (41/41) * feat: mandate online research with URL validation and backward compatibility - Make online research MANDATORY in docstring and prompts - AI assistants MUST perform online research and provide URLs - research_urls parameter optional for backward compatibility but strongly enforced - Enhanced prompts to REJECT submissions without research URLs - Clear instruction that online research is not optional - URLs should be comma-separated list demonstrating actual research - Prioritize existing solutions: current repo > well-known libraries > in-house Key changes: - ✅ MANDATORY online research requirement in all documentation - ✅ Backward compatible research_urls parameter (optional with empty default) - ✅ Enhanced validation to reject missing URLs as research failure - ✅ Clear guidance that online research is required, not optional - ✅ Instructs AI to actually DO research, not just ask for URLs - ✅ All tests passing (41/41) Fixes validation error while maintaining mandatory research requirement. * feat: enforce mandatory online research with List[str] URLs and collaborative workflow - Add mandatory minimum 3 URLs validation for research_urls parameter - Change research_urls from str to List[str] for proper URL handling - Add validation that rejects submissions with insufficient research URLs - Update docstring to emphasize collaborative workflow with user - Clarify that AI must 'collaborate with the user' and 'perform ONLINE research' - Restructure workflow order: requirements → online research → repo analysis → design → plan - Enhanced prompts to properly format URL lists with Jinja templating - All tests updated to provide minimum 3 URLs and use List[str] format Breaking changes: - research_urls parameter now requires List[str] instead of str - Minimum 3 URLs required (validates and rejects if insufficient) - Function signature maintains backward compatibility with None default Benefits: - ✅ Forces actual online research with evidence (minimum 3 URLs) - ✅ Prevents AI from working unilaterally - requires collaboration - ✅ Proper data structure for URL handling (List[str]) - ✅ Clear workflow prioritizing existing solutions over custom development - ✅ All tests passing with proper URL validation (41/41) ### 🚀 Features * add CODEOWNERS file to require approval from [@hepivax](https://github.com/hepivax) ([1bd5b29](1bd5b29)) * configure semantic release with GitHub App token for branch protection bypass ([315cb37](315cb37)) * enforce mandatory online research with List[str] URLs and collaborative workflow ([#11](#11)) ([3b76dc0](3b76dc0)) * separate user and system messages with type-safe Pydantic models ([3f0a688](3f0a688)) ### 🐛 Bug Fixes * correct version to 0.1.9 ([4072bdb](4072bdb)) * move prompts into package for reliable installation ([3d2fde6](3d2fde6)) * resolve prompts directory not found in installed package ([d8f7964](d8f7964)) ### ♻️ Code Refactoring * use importlib.resources for prompt loading (standard Python approach) ([f106137](f106137))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR enforces mandatory online research with proper URL validation and collaborative workflow improvements.
Key Changes
Breaking Changes
Benefits
Testing
All tests updated and passing:
Fixes the original validation error while maintaining mandatory research requirement.