Fix array comparison and add solver logging #11
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Train heuristic classifier on combined training and evaluation tasks and expose CLI. [S:DESIGN v1] pass [S:ALG v1] train_from_task_pairs pass [S:TRAIN v1] dataset=train+eval pass [S:TEST v1] unit=103 pass
|
Caution Review failedThe pull request is closed. WalkthroughThe PR updates documentation progress markers; adds operation priors to beam search and integrates them into enhanced search; refactors grid equality; adds deterministic initialization and training APIs for neural guidance; introduces structured logging and enhanced-vs-baseline fallback in the solver; adds training scripts and a model config; and expands tests. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant ARCSolver
participant Enhanced as EnhancedSearch
participant Baseline as BaselineSearch
participant Logger
Client->>ARCSolver: solve_task_two_attempts(task)
ARCSolver->>Logger: info("start")
alt enhancements enabled
ARCSolver->>Enhanced: synthesize_with_enhancements(task)
Enhanced-->>ARCSolver: enhanced_program?
ARCSolver->>Enhanced: predict_two_enhanced(task)
Enhanced-->>ARCSolver: enhanced_attempts?
ARCSolver->>ARCSolver: _validate_solution(enhanced_attempts)
alt valid
ARCSolver->>Baseline: synthesize_baseline + predict_two_baseline(task)
Baseline-->>ARCSolver: baseline_attempts
ARCSolver->>Logger: info("enhanced success")
ARCSolver-->>Client: [attempt1=enhanced, attempt2=baseline]
else invalid or error
ARCSolver->>Logger: exception("enhanced failed")
ARCSolver->>Baseline: synthesize_baseline + predict_two_baseline(task)
Baseline-->>ARCSolver: baseline_attempts
ARCSolver->>Logger: info("fallback to baseline")
ARCSolver-->>Client: baseline_attempts
end
else enhancements disabled
ARCSolver->>Baseline: synthesize_baseline + predict_two_baseline(task)
Baseline-->>ARCSolver: baseline_attempts
ARCSolver-->>Client: baseline_attempts
end
sequenceDiagram
autonumber
participant Enhanced as EnhancedSearch
participant Guidance as NeuralGuidance
participant Beam as beam_search
Enhanced->>Guidance: score_operations(train_pairs)
Guidance-->>Enhanced: op_scores {op: score}
Enhanced->>Beam: beam_search(train_pairs, ..., op_scores=op_scores)
Beam-->>Enhanced: programs/candidates (scores weighted by op_scores)
sequenceDiagram
autonumber
participant CLI as train_guidance_on_arc.py
participant Loader as _load_tasks
participant Guidance as NeuralGuidance
participant FS as Filesystem
CLI->>Loader: load training/eval tasks
Loader-->>CLI: tasks [[(inp, out), ...], ...]
CLI->>Guidance: train_from_task_pairs(tasks, epochs)
Guidance-->>CLI: trained model in memory
CLI->>FS: save_model(models/guidance_arc.json)
FS-->>CLI: OK
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (15)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
Codex Review: Here are some suggestions.
Reply with @codex fix comments to fix any unresolved comments.
About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
tools/train_guidance_on_arc.pyCLI and mark multi-modal reasoning step completemodels/guidance_arc.jsonTesting
pip install -r requirements.txtpytestpython tools/train_guidance_on_arc.py --epochs 5https://chatgpt.com/codex/tasks/task_e_68c3fda055488322804ea540d87df916
Summary by CodeRabbit