Add developer content check workflow #461

scosman · 2025-07-29T15:59:03Z

Summary

add workflow that searches for leftover developer code like console.log, Python print(, and TODO/FIXME comments
reword TODO comments to keep intent while avoiding the keyword

Testing

uvx ruff check --select I
uvx ruff format --check .
uv run pyright .
uv run python3 -m pytest --benchmark-quiet -q .
npm run format_check in app/web_ui
npm run lint in app/web_ui
npm run check in app/web_ui
npm run test_run in app/web_ui
npm run build in app/web_ui

https://chatgpt.com/codex/tasks/task_e_6888e8b79d5883328a34ac7cf92cea71

Summary by CodeRabbit

Chores
- Updated comments throughout the codebase to rephrase or clarify existing notes, removing or rewording TODOs and related statements.
- Minor adjustments to provider model IDs and comments in model configuration.
- Added a new GitHub Actions workflow to automatically detect and flag leftover debug content (such as console.log, print statements, TODOs, and FIXMEs) in pull requests and pushes to the main branch.
- Enforced stricter linting rules to disallow console.log usage while permitting other console methods.
Style
- Replaced print statements with sys.stdout.write in some test scripts for output consistency.

coderabbitai · 2025-07-29T15:59:11Z

Caution

Review failed

The pull request is closed.

Walkthrough

This set of changes primarily updates comments across various files, rewording or removing "TODO" notes to more descriptive or declarative statements without affecting any functional code. Additionally, a new GitHub Actions workflow, "Debug Detector," is introduced to automatically scan for debug statements and TODOs in the codebase during CI runs. Some test scripts also shift output from print to sys.stdout.write.

Changes

Cohort / File(s)	Change Summary
CI: Debug Detector Workflow `.github/workflows/debug_detector.yml`	Adds a GitHub Actions workflow that scans for developer debug content (e.g., `print(`, `TODO`, `FIXME`) on pushes and PRs, failing if any are found.
CI: Windows Release Build Workflow `.github/workflows/windows_release_build.yml`	Rewords a comment about DLL signing; no logic changes.
Desktop App Build and Globals `app/desktop/build_desktop_app.sh`, `app/desktop/desktop.py`	Updates comments to be more descriptive regarding PyInstaller usage and global variables.
Web UI Route Comments `app/web_ui/src/routes/(app)/dataset/[project_id]/[task_id]/[run_id]/run/+page.svelte`, `app/web_ui/src/routes/(app)/generate/[project_id]/[task_id]/synth_data_guidance_datamodel.ts`, `app/web_ui/src/routes/(app)/run/+page.svelte`, `app/web_ui/src/routes/(app)/run/run.svelte`	Removes or rephrases TODO comments; no code changes.
Test Output Method Updates `app/desktop/studio_server/test_generate_docs.py`, `libs/core/kiln_ai/adapters/test_remote_config.py`, `libs/core/kiln_ai/datamodel/test_model_perf.py`	Replaces `print` statements with `sys.stdout.write` or comments out print lines; no logic changes.
Model Provider List and Comments `libs/core/kiln_ai/adapters/ml_model_list.py`	Clarifies comments, updates model provider IDs for Gemma 3 models, and removes an `openrouter` entry for Gemma 3 1B.
Adapters: Data Gen, Model, Repair, Registry `libs/core/kiln_ai/adapters/data_gen/data_gen_task.py`, `libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py`, `libs/core/kiln_ai/adapters/repair/repair_task.py`, `libs/core/kiln_ai/adapters/test_adapter_registry.py`	Updates comments from TODOs to statements of intent or clarification; no code logic changes.
Adapters: Eval Data Test `libs/core/kiln_ai/adapters/eval/test_g_eval_data.py`	Rewords a comment about inspecting pickled data; no functional changes.
Datamodel: Project, Task, BaseModel `libs/core/kiln_ai/datamodel/project.py`, `libs/core/kiln_ai/datamodel/task.py`, `libs/core/kiln_ai/datamodel/basemodel.py`	Updates comments about future improvements; no logic changes.
ESLint Configuration `app/web_ui/.eslintrc.cjs`	Adds "no-console" rule disallowing `console.log` but permitting `console.warn`, `console.error`, and `console.info`.

Sequence Diagram(s)

sequenceDiagram
    participant Developer
    participant GitHub Actions
    participant Debug Detector Script

    Developer->>GitHub Actions: Push or create PR
    GitHub Actions->>Debug Detector Script: Run debug content check
    Debug Detector Script->>GitHub Actions: Report findings (fail if debug content found)
    GitHub Actions->>Developer: Pass/fail status with debug output (if any)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐇
A sweep of TODOs, now reworded with care,
No more "fix me" whispers left hanging in air.
A new detector hops in, alert and astute—
Scanning for prints and logs, on its nightly commute.
With comments now clearer and checks in the night,
This rabbit approves—your code’s looking right!

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 668293d and d60f5a1.

📒 Files selected for processing (2)

.github/workflows/debug_detector.yml (1 hunks)
app/web_ui/.eslintrc.cjs (1 hunks)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai generate unit tests to generate unit tests for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (2)

.github/workflows/developer_content_check.yml (2)
25-25: Consider refining the print statement detection pattern.

The current pattern 'print(' will match any occurrence of "print(" including in strings, comments, or documentation where it might be legitimate.

Consider using a more specific pattern to avoid false positives:
-          prints=$(grep -nR --include='*.py' --exclude-dir=node_modules --exclude-dir=.venv --exclude-dir=.git --exclude-dir=.github --exclude-dir=build --exclude-dir=dist --exclude-dir=.svelte-kit -e 'print(' . || true)
+          prints=$(grep -nR --include='*.py' --exclude-dir=node_modules --exclude-dir=.venv --exclude-dir=.git --exclude-dir=.github --exclude-dir=build --exclude-dir=dist --exclude-dir=.svelte-kit -E '^\s*print\(' . || true)
This pattern looks for print statements at the beginning of lines (with optional whitespace), reducing false positives from strings or comments.

32-32: Consider case-insensitive matching for TODO/FIXME.

The current pattern only matches exact case. Developers sometimes use different cases like "todo" or "fixme".

Consider adding case-insensitive matching:
-          notes=$(grep -nR --exclude-dir=node_modules --exclude-dir=.venv --exclude-dir=.git --exclude-dir=.github --exclude-dir=build --exclude-dir=dist --exclude-dir=.svelte-kit -e 'TODO' -e 'FIXME' . || true)
+          notes=$(grep -nRi --exclude-dir=node_modules --exclude-dir=.venv --exclude-dir=.git --exclude-dir=.github --exclude-dir=build --exclude-dir=dist --exclude-dir=.svelte-kit -e 'TODO' -e 'FIXME' . || true)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4fc567c and 8249a4f.

📒 Files selected for processing (20)

.github/workflows/developer_content_check.yml (1 hunks)
.github/workflows/windows_release_build.yml (1 hunks)
app/desktop/build_desktop_app.sh (1 hunks)
app/desktop/desktop.py (1 hunks)
app/desktop/studio_server/test_generate_docs.py (2 hunks)
app/web_ui/src/routes/(app)/dataset/[project_id]/[task_id]/[run_id]/run/+page.svelte (1 hunks)
app/web_ui/src/routes/(app)/generate/[project_id]/[task_id]/synth_data_guidance_datamodel.ts (1 hunks)
app/web_ui/src/routes/(app)/run/+page.svelte (1 hunks)
app/web_ui/src/routes/(app)/run/run.svelte (1 hunks)
libs/core/kiln_ai/adapters/data_gen/data_gen_task.py (2 hunks)
libs/core/kiln_ai/adapters/eval/test_g_eval_data.py (1 hunks)
libs/core/kiln_ai/adapters/ml_model_list.py (5 hunks)
libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py (1 hunks)
libs/core/kiln_ai/adapters/repair/repair_task.py (1 hunks)
libs/core/kiln_ai/adapters/test_adapter_registry.py (2 hunks)
libs/core/kiln_ai/adapters/test_remote_config.py (1 hunks)
libs/core/kiln_ai/datamodel/basemodel.py (1 hunks)
libs/core/kiln_ai/datamodel/project.py (1 hunks)
libs/core/kiln_ai/datamodel/task.py (1 hunks)
libs/core/kiln_ai/datamodel/test_model_perf.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/project.mdc)

**/*.py: Always assume pydantic 2 (not pydantic 1)
The project supports Python 3.10 and above

Files:

libs/core/kiln_ai/datamodel/task.py
libs/core/kiln_ai/adapters/test_adapter_registry.py
libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py
libs/core/kiln_ai/datamodel/test_model_perf.py
libs/core/kiln_ai/adapters/repair/repair_task.py
app/desktop/desktop.py
app/desktop/studio_server/test_generate_docs.py
libs/core/kiln_ai/datamodel/project.py
libs/core/kiln_ai/adapters/test_remote_config.py
libs/core/kiln_ai/adapters/data_gen/data_gen_task.py
libs/core/kiln_ai/datamodel/basemodel.py
libs/core/kiln_ai/adapters/ml_model_list.py
libs/core/kiln_ai/adapters/eval/test_g_eval_data.py

**/test_*.py

📄 CodeRabbit Inference Engine (.cursor/rules/project.mdc)

**/test_*.py: Always use pytest for tests in Python code
Test brevity is important. Use approaches for re-use and brevity including using fixtures for repeated code, and using pytest parameterize for similar tests

Files:

libs/core/kiln_ai/adapters/test_adapter_registry.py
libs/core/kiln_ai/datamodel/test_model_perf.py
app/desktop/studio_server/test_generate_docs.py
libs/core/kiln_ai/adapters/test_remote_config.py
libs/core/kiln_ai/adapters/eval/test_g_eval_data.py

app/web_ui/src/**/*.{js,jsx,ts,tsx,svelte}

📄 CodeRabbit Inference Engine (.cursor/rules/project.mdc)

Our web UI component framework is DaisyUI v4 (based on Tailwind). Be sure to use v4 docs and features, not v5

Files:

app/web_ui/src/routes/(app)/run/run.svelte
app/web_ui/src/routes/(app)/dataset/[project_id]/[task_id]/[run_id]/run/+page.svelte
app/web_ui/src/routes/(app)/generate/[project_id]/[task_id]/synth_data_guidance_datamodel.ts
app/web_ui/src/routes/(app)/run/+page.svelte

🧠 Learnings (17)

📓 Common learnings

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: Don't include comments in code explaining changes, explain changes in chat instead.

libs/core/kiln_ai/datamodel/task.py (1)