Skip to content

Commit 5966e44

Browse files
authored
Update README.md
1 parent da50b58 commit 5966e44

File tree

1 file changed

+66
-31
lines changed

1 file changed

+66
-31
lines changed

README.md

Lines changed: 66 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Enhanced ARC-AGI-2 Solver
1+
# PUMA: Program Understanding & Meta-learning Architecture
22

33
This repository contains an advanced solver for the **ARC Prize 2025** competition (ARC‑AGI‑2), implementing the complete blueprint from neuroscience-inspired research. It combines symbolic reasoning with neural guidance, episodic retrieval, program sketches, and test-time training to achieve superior performance on abstract reasoning tasks.
44

@@ -8,22 +8,36 @@ This repository contains an advanced solver for the **ARC Prize 2025** competiti
88
<img src="docs/images/rft_behavioral_approach.svg" alt="Behavioral RFT approach" width="400"/>
99
</p>
1010

11-
We are exploring a behavioral perspective grounded in **Relational Frame Theory (RFT)** to tackle ARC. RFT models language and cognition as networks of learned relational frames, giving us a principled way to compose and generalize transformations across tasks.
11+
We are implementing a behavioral perspective grounded in **Relational Frame Theory (RFT)** to tackle ARC through explicit relational reasoning. RFT models cognition as networks of learned relational frames, providing a principled foundation for understanding spatial and contextual relationships between objects.
1212

13-
For more details, see [profile/README.md](profile/README.md).
13+
### RFT Implementation Strategy
14+
15+
Our RFT approach focuses on learning explicit relational contexts between objects:
16+
17+
- **Relational Fact Extraction**: Parse visual scenes to identify objects and their spatial relationships (e.g., “blue square is always at top position”)
18+
- **Contextual Rule Learning**: Extract invariant relationships across training examples (e.g., “if blue square at top, then red square at position (blue_y + 1, blue_x)”)
19+
- **Compositional Reasoning**: Combine learned relational frames to generate predictions for novel configurations
20+
- **Behavioral Generalization**: Apply relational rules systematically rather than relying on pattern matching
21+
22+
This approach complements the neural components by providing explicit, interpretable relational knowledge that can be composed and reasoned about symbolically.
23+
24+
For more details, see <profile/README.md>.
1425

1526
## Key Features
1627

17-
### 🧠 Neuroscience-Inspired Architecture
28+
### Neuroscience-Inspired Architecture
29+
1830
- **Neural guidance**: Predicts relevant DSL operations using task features
19-
- **Episodic retrieval**: Maintains database of solved tasks for analogical reasoning
31+
- **Episodic retrieval**: Maintains database of solved tasks for analogical reasoning
2032
- **Program sketches**: Mines common operation sequences as macro-operators
2133
- **Test-time training**: Adapts scoring functions to each specific task
2234
- **Multi-demand network analog**: Prioritizes candidate programs using learned heuristics
2335

24-
### 🔧 Enhanced Capabilities
36+
### Enhanced Capabilities
37+
2538
- **Object-centric parsing** with connected component analysis
2639
- **Compact DSL** with composable primitives (rotate, flip, translate, recolor, etc.)
40+
- **Relational reasoning** through explicit fact extraction and rule learning
2741
- **Two-attempt diversity** as required by ARC Prize 2025 rules
2842
- **Fallback resilience** with graceful degradation to baseline methods
2943
- **Performance monitoring** with detailed statistics and benchmarking
@@ -62,7 +76,7 @@ arc_solver_project/
6276

6377
## Quick Start
6478

65-
### 1. Basic Usage (Kaggle-ready)
79+
### Basic Usage (Kaggle-ready)
6680

6781
```bash
6882
# Generate submission file (uses enhanced solver by default)
@@ -72,7 +86,7 @@ python arc_submit.py
7286
ARC_USE_BASELINE=1 python arc_submit.py
7387
```
7488

75-
### 2. Training Neural Components
89+
### Training Neural Components
7690

7791
```bash
7892
# Train neural guidance (requires training data)
@@ -82,7 +96,7 @@ python tools/train_guidance.py
8296
python tools/benchmark.py
8397
```
8498

85-
### 3. Python API
99+
### Python API
86100

87101
```python
88102
from arc_solver.solver import solve_task_enhanced, ARCSolver
@@ -95,7 +109,7 @@ solver = ARCSolver(use_enhancements=True)
95109
result = solver.solve_task(task)
96110
```
97111

98-
### 4. Public Evaluation Runner
112+
### Public Evaluation Runner
99113

100114
```bash
101115
scripts/eval_public.sh
@@ -112,15 +126,18 @@ make eval_public
112126
### Enhanced Pipeline
113127

114128
1. **Feature Extraction**: Extract task-level features (colors, objects, transformations)
115-
2. **Neural Guidance**: Predict which DSL operations are likely relevant
116-
3. **Episodic Retrieval**: Query database for similar previously solved tasks
117-
4. **Sketch-Based Search**: Use mined program templates with parameter filling
118-
5. **Test-Time Adaptation**: Fine-tune scoring function using task demonstrations
119-
6. **Program Selection**: Rank and select top 2 diverse candidate programs
129+
1. **Relational Context Analysis**: Identify spatial and contextual relationships between objects using RFT principles
130+
1. **Neural Guidance**: Predict which DSL operations are likely relevant
131+
1. **Episodic Retrieval**: Query database for similar previously solved tasks
132+
1. **Sketch-Based Search**: Use mined program templates with parameter filling
133+
1. **Rule-Based Reasoning**: Apply learned relational facts to generate candidate solutions
134+
1. **Test-Time Adaptation**: Fine-tune scoring function using task demonstrations
135+
1. **Program Selection**: Rank and select top 2 diverse candidate programs
120136

121137
### Fallback Strategy
122138

123139
If enhanced components fail, the solver gracefully falls back to:
140+
124141
- Heuristic single-step transformations
125142
- Brute-force enumeration of 2-step programs
126143
- Identity transformation as last resort
@@ -130,10 +147,12 @@ If enhanced components fail, the solver gracefully falls back to:
130147
The solver supports extensive configuration through environment variables and config files:
131148

132149
### Environment Variables
150+
133151
- `ARC_USE_BASELINE=1`: Force baseline solver only
134152
- `ARC_DISABLE_ENHANCEMENTS=1`: Disable enhanced features
135153

136154
### Configuration File
155+
137156
```json
138157
{
139158
"use_neural_guidance": true,
@@ -148,24 +167,28 @@ The solver supports extensive configuration through environment variables and co
148167
## Neural Components
149168

150169
### Neural Guidance
170+
151171
- **Purpose**: Predict which DSL operations are relevant for a given task
152172
- **Architecture**: Simple MLP with task-level features
153173
- **Training**: Uses extracted features from training demonstrations
154174
- **Output**: Operation relevance scores to guide search
155175

156176
### Episodic Retrieval
177+
157178
- **Purpose**: Reuse solutions from similar previously solved tasks
158179
- **Method**: Task signature matching with feature-based similarity
159180
- **Storage**: JSON-based database of solved programs with metadata
160181
- **Retrieval**: Cosine similarity on numerical features + boolean feature matching
161182

162183
### Program Sketches
184+
163185
- **Purpose**: Capture common operation sequences as reusable templates
164186
- **Mining**: Extract frequent 1-step and 2-step operation patterns
165187
- **Usage**: Instantiate sketches with different parameter combinations
166188
- **Adaptation**: Learn from successful programs during solving
167189

168190
### Test-Time Training
191+
169192
- **Purpose**: Adapt scoring function to each specific task
170193
- **Method**: Fine-tune lightweight scorer on task demonstrations
171194
- **Features**: Program length, operation types, success rate, complexity
@@ -174,6 +197,7 @@ The solver supports extensive configuration through environment variables and co
174197
## Performance and Evaluation
175198

176199
### Benchmarking
200+
177201
```python
178202
from benchmark import Benchmark, SolverConfig
179203

@@ -184,7 +208,9 @@ print(f"Success rate: {results['performance_stats']['success_rate']:.3f}")
184208
```
185209

186210
### Monitoring
211+
187212
The solver tracks detailed statistics:
213+
188214
- Success rates for enhanced vs baseline methods
189215
- Component usage (episodic hits, neural guidance, TTT adaptation)
190216
- Timing breakdown per component
@@ -193,12 +219,14 @@ The solver tracks detailed statistics:
193219
## Implementation Notes
194220

195221
### Kaggle Compatibility
222+
196223
- **Offline execution**: No internet access required
197224
- **Dependency-light**: Uses only NumPy for core operations
198225
- **Compute budget**: Optimized for ~$0.42 per task limit
199226
- **Output format**: Exactly 2 attempts per test input as required
200227

201228
### Code Quality
229+
202230
- **Type hints**: Full typing support for better maintainability
203231
- **Documentation**: Comprehensive docstrings and comments
204232
- **Error handling**: Robust fallback mechanisms
@@ -207,29 +235,32 @@ The solver tracks detailed statistics:
207235
## Extending the Solver
208236

209237
### Adding New DSL Operations
238+
210239
1. Define operation function in `dsl.py`
211-
2. Add parameter generation in `sketches.py`
212-
3. Update feature extraction in `features.py`
213-
4. Retrain neural guidance if needed
240+
1. Add parameter generation in `sketches.py`
241+
1. Update feature extraction in `features.py`
242+
1. Retrain neural guidance if needed
214243

215244
### Improving Neural Components
245+
216246
1. **Better features**: Add domain-specific feature extractors
217-
2. **Advanced models**: Replace MLP with transformer/GNN
218-
3. **Meta-learning**: Implement few-shot adaptation algorithms
219-
4. **Hybrid methods**: Combine symbolic and neural reasoning
247+
1. **Advanced models**: Replace MLP with transformer/GNN
248+
1. **Meta-learning**: Implement few-shot adaptation algorithms
249+
1. **Hybrid methods**: Combine symbolic and neural reasoning
220250

221251
### Advanced Techniques
252+
222253
- **Probabilistic programming**: Sample programs from learned distributions
223254
- **Curriculum learning**: Train on tasks of increasing difficulty
224255
- **Multi-agent reasoning**: Ensemble of specialized solvers
225256
- **Causal reasoning**: Incorporate causal structure learning
226257

227258
## Research Foundation
228259

229-
This implementation is based on the research blueprint "ARC Prize 2025 & Human Fluid Intelligence" which draws from cognitive neuroscience findings about:
260+
This implementation is based on the research blueprint ARC Prize 2025 & Human Fluid Intelligence which draws from cognitive neuroscience findings about:
230261

231262
- **Multiple-demand (MD) network**: Neural guidance mimics executive control
232-
- **Basal ganglia gating**: Operation selection and working memory control
263+
- **Basal ganglia gating**: Operation selection and working memory control
233264
- **Hippocampal-mPFC loop**: Episodic retrieval and schema integration
234265
- **Test-time adaptation**: Rapid task-specific learning from few examples
235266

@@ -238,18 +269,21 @@ The solver architecture directly maps these biological systems to computational
238269
## Competition Strategy
239270

240271
### Short-term (Immediate)
241-
- ✅ Strong symbolic baseline with neural enhancements
242-
- ✅ Episodic retrieval for common patterns
243-
- ✅ Test-time adaptation for task specialization
244-
- ✅ Kaggle-ready submission format
272+
273+
- Strong symbolic baseline with neural enhancements
274+
- Episodic retrieval for common patterns
275+
- Test-time adaptation for task specialization
276+
- Kaggle-ready submission format
245277

246278
### Medium-term (During Contest)
279+
247280
- Train neural guidance on public training data
248-
- Mine program sketches from successful solutions
281+
- Mine program sketches from successful solutions
249282
- Analyze semi-private feedback for failure modes
250283
- Expand DSL based on discovered patterns
251284

252285
### Long-term (Advanced Research)
286+
253287
- Probabilistic program synthesis
254288
- Hybrid symbolic-neural architecture
255289
- Broader cognitive priors and meta-learning
@@ -265,12 +299,13 @@ If you use this solver or build upon its ideas, please cite the research bluepri
265299

266300
## Contributing
267301

268-
Contributions are welcome! Focus areas:
302+
Contributions are welcome. Focus areas include:
303+
269304
- Neural architecture improvements
270305
- New DSL operations based on failure analysis
271306
- Advanced meta-learning techniques
272307
- Performance optimizations for Kaggle constraints
273308

274-
---
309+
-----
275310

276-
**Ready to win ARC Prize 2025!** 🏆
311+
**Ready to compete in ARC Prize 2025**

0 commit comments

Comments
 (0)