Skip to content

Conversation

@jvm123
Copy link
Contributor

@jvm123 jvm123 commented Jun 9, 2025

Implements the artifact side channel responses feature also for LLM feedback.

  • Any LLM feedback response that is not numeric is classified as artifact
  • _llm_evaluate returns an EvaluationResult() to evaluate_program()
  • evaluate_program() merges the responses

=> LLM feedback strings can contain improvement suggestions that will be available for program improvement

To use this, it is sufficient to use as evaluation prompt something like

Return your evaluation as a JSON object with the following format:
{{
    "metric1": score1,
    "improvement_suggestion": "..",
}}

@jvm123 jvm123 force-pushed the feat-llm-artifact-side-channel branch from d681774 to 0fe928f Compare June 16, 2025 00:23
@codelion codelion merged commit d1d1a57 into algorithmicsuperintelligence:main Jun 19, 2025
3 checks passed
codelion added a commit that referenced this pull request Jun 19, 2025
…nnel"

This reverts commit d1d1a57, reversing
changes made to 59872a7.
codelion added a commit that referenced this pull request Jun 19, 2025
Revert "Merge pull request #68 from jvm123/feat-llm-artifact-side-cha…
0x0f0f0f pushed a commit to 0x0f0f0f/openevolve that referenced this pull request Jul 7, 2025
…3/feat-llm-artifact-side-channel"

This reverts commit eb06dc1, reversing
changes made to a839321.
0x0f0f0f pushed a commit to 0x0f0f0f/openevolve that referenced this pull request Jul 7, 2025
…rt-old-Pr

Revert "Merge pull request algorithmicsuperintelligence#68 from jvm123/feat-llm-artifact-side-cha…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants