Skip to content

feat: persist formatted agent prompts in run outputs#75

Merged
dippatel1994 merged 4 commits intollmsresearch:mainfrom
claytonlin1110:feat/save-prompts-run-artifacts
Mar 9, 2026
Merged

feat: persist formatted agent prompts in run outputs#75
dippatel1994 merged 4 commits intollmsresearch:mainfrom
claytonlin1110:feat/save-prompts-run-artifacts

Conversation

@claytonlin1110
Copy link
Contributor

@claytonlin1110 claytonlin1110 commented Mar 6, 2026

Summary

Closes #76

  • Adds optional prompt recording to save the formatted prompts used by Optimizer/Retriever/Planner/Stylist/Visualizer/Critic into outputs/<run_id>/prompts/*.txt.
  • Wires output.save_prompts from YAML into Settings.save_prompts and adds CLI toggles: --save-prompts/--no-save-prompts for generate and plot.
  • Adds offline tests verifying prompts are saved when enabled and absent when disabled.

Key changes

  • New helper: paperbanana/core/prompt_recorder.py
  • Config plumbing: paperbanana/core/config.py
  • Agent integration: paperbanana/agents/base.py + agent prompt labels
  • Pipeline wiring: paperbanana/core/pipeline.py
  • CLI flags: paperbanana/cli.py
  • Tests: tests/test_pipeline/test_save_prompts.py

@claytonlin1110
Copy link
Contributor Author

Closes #76

@claytonlin1110
Copy link
Contributor Author

Hi, @dippatel1994 Would you please review my first contribution? Thanks.

@dippatel1994
Copy link
Member

Hi @claytonlin1110, thanks for the PR. The overall change looks clean and the prompt recording approach is well scoped. I ran the new prompt-saving tests locally and they passed.

I found one small issue to fix before merge: in paperbanana/agents/critic.py, the prompt gets recorded before user_feedback is appended. That means in continue-run flows, the saved critic prompt is not the exact prompt actually sent to the model, which makes the artifact a bit misleading for debugging. Could you please update that path so the recorded prompt includes the appended user feedback as well?

Also, issue #76 is still open, and this PR does not currently show it as an auto-closing linked issue in GitHub metadata. You may want to add Closes #76 to the PR description so it closes automatically on merge.

Once those are addressed, this looks in good shape. Thanks!

@claytonlin1110
Copy link
Contributor Author

@dippatel1994 Fixed everything. Please check

@dippatel1994
Copy link
Member

changes looks good @claytonlin1110 just check failing ci pipelines.

@claytonlin1110
Copy link
Contributor Author

@dippatel1994 Just fixed

@dippatel1994
Copy link
Member

sorry @claytonlin1110 but one of the ci/lint is still failing.

@dippatel1994
Copy link
Member

Hi @claytonlin1110 just one more ci pipeline fix and we can close this ticket!

@claytonlin1110
Copy link
Contributor Author

@dippatel1994 Fixed.

@dippatel1994 dippatel1994 merged commit 22deffc into llmsresearch:main Mar 9, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add save_prompts run artifacts to persist formatted agent prompts

2 participants