Skip to content

Commit f23ea1e

Browse files
author
Frankie Robertson
committed
Add CLAUDE and architecture
1 parent 1f427ad commit f23ea1e

File tree

2 files changed

+280
-0
lines changed

2 files changed

+280
-0
lines changed

‎ARCHITECTURE.md

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# ComputerAdaptiveTesting.jl Architecture
2+
3+
## Overview
4+
5+
ComputerAdaptiveTesting.jl is organized into several interconnected modules that implement various aspects of Computer Adaptive Testing. The architecture follows a modular design with clear separation of concerns.
6+
7+
## Core Module Structure
8+
9+
### 1. Base Layer
10+
11+
#### ConfigBase (`src/ConfigBase.jl`)
12+
- Defines `CatConfigBase` abstract type
13+
- Provides `walk` function for traversing configuration trees
14+
- Foundation for all configuration structs
15+
16+
#### Responses (`src/Responses.jl`)
17+
- `Response`: Individual test response
18+
- `BareResponses`: Collection of responses
19+
- `AbilityLikelihood`: Likelihood calculation for abilities
20+
- Response type handling for boolean and multinomial responses
21+
22+
### 2. Aggregation Layer
23+
24+
#### Aggregators (`src/Aggregators/`)
25+
Main module for ability estimation and tracking:
26+
- **Ability Estimators**:
27+
- `LikelihoodAbilityEstimator`: Maximum likelihood estimation
28+
- `PosteriorAbilityEstimator`: Bayesian estimation with priors
29+
- `MeanAbilityEstimator`: Expected a posteriori (EAP)
30+
- `ModeAbilityEstimator`: Maximum a posteriori (MAP)
31+
- **Ability Trackers**:
32+
- `NullAbilityTracker`: No tracking
33+
- `PointAbilityTracker`: Tracks point estimates
34+
- `GriddedAbilityTracker`: Grid-based tracking
35+
- `ClosedFormNormalAbilityTracker`: Analytical normal approximation
36+
- **Integration**: Various integrators for numerical integration
37+
- **Optimization**: Optimizers for finding maximum likelihood/posterior
38+
39+
### 3. CAT Logic Layer
40+
41+
#### NextItemRules (`src/NextItemRules/`)
42+
Implements item selection algorithms:
43+
- **Strategies**:
44+
- `ExhaustiveSearch`: Evaluates all items
45+
- `RandomNextItemRule`: Random selection
46+
- `RandomesqueStrategy`: Top-k random selection
47+
- `BalancedStrategy`: Content balancing
48+
- **Criteria**:
49+
- `InformationItemCriterion`: Fisher information
50+
- `AbilityVariance`: Minimize ability estimate variance
51+
- `UrryItemCriterion`: Difficulty matching
52+
- `ExpectationBasedItemCriterion`: Expected criteria values
53+
- **Multi-dimensional support**: D-Rule, T-Rule for MIRT
54+
55+
#### TerminationConditions (`src/TerminationConditions.jl`)
56+
- `FixedLength`: Stop after N items
57+
- `RunForever`: No termination
58+
- `TerminationTest`: Custom termination logic
59+
60+
### 4. Execution Layer
61+
62+
#### Rules (`src/Rules.jl`)
63+
- `CatRules`: Main configuration struct combining:
64+
- Next item selection rule
65+
- Termination condition
66+
- Ability estimator
67+
- Ability tracker
68+
69+
#### Sim (`src/Sim/`)
70+
Simulation and execution:
71+
- `CatLoop`: Configuration for running a CAT
72+
- `run_cat`: Main execution function
73+
- `CatRecorder`: Recording CAT sessions
74+
- Response callbacks and hooks
75+
76+
### 5. Advanced Features
77+
78+
#### DecisionTree (`src/DecisionTree/`)
79+
- Pre-computation of CAT decisions into decision trees
80+
- `MaterializedDecisionTree`: Stored decision tree
81+
- Memory-mapped storage for large trees
82+
83+
#### Stateful (`src/Stateful.jl`)
84+
Stateful interface for CAT implementations:
85+
- `StatefulCat`: Abstract interface
86+
- `StatefulCatRules`: Implementation using CatRules
87+
- Methods: `next_item`, `add_response!`, `get_ability`, etc.
88+
89+
#### Comparison (`src/Comparison/`)
90+
Tools for comparing CAT configurations:
91+
- Execution strategies for benchmarking
92+
- Statistical comparison tools
93+
- Watchdog for timeout handling
94+
95+
#### Compatibility (`src/Compat/`)
96+
- `CatR`: Compatibility with R's catR package
97+
- `MirtCAT`: Compatibility with R's mirtCAT package
98+
99+
### 6. Support Modules
100+
101+
#### ItemBanks (`src/logitembank.jl`)
102+
- `LogItemBank`: Logarithmic scale item banks
103+
- Temporary workaround to get logprobs, likely to be removed later on
104+
105+
#### Precompilation (`src/precompiles.jl`)
106+
- Precompilation workloads for faster package loading
107+
108+
## Key Design Patterns
109+
110+
### 1. Configuration Trees
111+
Most components use a tree structure of configuration objects:
112+
```julia
113+
CatRules(
114+
next_item = ItemCriterionRule(...),
115+
termination_condition = FixedLength(20),
116+
ability_estimator = MeanAbilityEstimator(...),
117+
ability_tracker = GriddedAbilityTracker(...)
118+
)
119+
```
120+
121+
### 2. Implicit Constructors
122+
Many types support implicit construction where components are automatically selected:
123+
```julia
124+
# Explicit
125+
MeanAbilityEstimator(PosteriorAbilityEstimator(), integrator)
126+
127+
# Implicit - finds appropriate components
128+
MeanAbilityEstimator(bits...)
129+
```
130+
131+
### 3. Preallocatable Pattern
132+
Components can be preallocated for performance:
133+
```julia
134+
rule = preallocate(next_item_rule)
135+
```
136+
137+
### 4. Tracked Responses
138+
Responses are wrapped with tracking information:
139+
```julia
140+
TrackedResponses(
141+
responses::BareResponses,
142+
item_bank::AbstractItemBank,
143+
ability_tracker::AbilityTracker
144+
)
145+
```
146+
147+
## Performance Optimizations
148+
149+
1. **Preallocation**: Many components support preallocation
150+
2. **Specialization**: Type-stable code with concrete types
151+
3. **Threading**: Thread-safe operations with `init_thread`
152+
4. **Integration caching**: Grid-based integrators cache evaluations
153+
5. **Log-space computations**: Avoid numerical underflow
154+
155+
## Extension Points
156+
157+
1. **Custom Item Selection**: Extend `ItemCriterion` or `NextItemRule`
158+
2. **Custom Ability Estimation**: Extend `AbilityEstimator`
159+
3. **Custom Termination**: Implement `TerminationCondition`
160+
4. **Custom Item Banks**: Via FittedItemBanks.jl interface
161+
162+
## Testing Architecture
163+
164+
- Unit tests for individual components
165+
- Integration tests for complete CAT runs
166+
- Smoke tests for various configurations
167+
- Aqua.jl for code quality
168+
- JET.jl for type stability
169+
170+
## Documentation Structure
171+
172+
- API documentation via Documenter.jl
173+
- Examples using Literate.jl
174+
- Interactive demos with Makie.jl plots
175+
- Compatibility with R packages documented

‎CLAUDE.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Claude Code Assistant Guide for ComputerAdaptiveTesting.jl
2+
3+
## Overview
4+
5+
ComputerAdaptiveTesting.jl is a Julia package that implements Computer Adaptive Testing (CAT) algorithms. The package provides fast implementations of well-known CAT approaches with flexible scaffolding to support new approaches and non-standard item banks.
6+
7+
## Development Workflow
8+
9+
This project uses a **pull request workflow**. All changes should be made in feature branches and submitted via pull requests for review.
10+
11+
### Steps for Contributing:
12+
13+
1. **Create a feature branch** for your changes:
14+
```bash
15+
git checkout -b feature/your-feature-name
16+
```
17+
18+
2. **Make your changes** following the project structure and conventions
19+
20+
3. **Run tests** before committing:
21+
```bash
22+
julia --project=test test/runtests.jl
23+
```
24+
25+
4. **Commit your changes** with descriptive commit messages
26+
27+
5. **Push to your branch** and create a pull request
28+
29+
6. **Wait for review** before merging
30+
31+
## Project Structure
32+
33+
See @ARCHITECTURE.md for detailed information about the codebase structure.
34+
35+
## Key Areas for Assistance
36+
37+
### 1. Core Modules
38+
- **Aggregators**: Ability estimation and tracking
39+
- **NextItemRules**: Item selection algorithms
40+
- **Responses**: Response handling and storage
41+
- **Sim**: Simulation and CAT execution
42+
43+
### 2. Testing
44+
- Tests are in the `test/` directory
45+
- Run tests with `julia --project=test test/runtests.jl`
46+
- Note: `Pkg.test()` currently doesn't work (see issue #52)
47+
48+
### 3. Documentation
49+
- Documentation source is in `docs/src/`
50+
- Examples are in `docs/examples/`
51+
- Build docs with `julia --project=docs docs/make.jl`
52+
53+
## Common Tasks
54+
55+
### Adding a New Item Selection Rule
56+
1. Create new file in `src/NextItemRules/criteria/`
57+
2. Implement the criterion interface
58+
3. Add tests in `test/`
59+
4. Update documentation
60+
61+
### Adding a New Ability Estimator
62+
1. Create new file in `src/Aggregators/`
63+
2. Extend the `AbilityEstimator` abstract type
64+
3. Implement required methods
65+
4. Add integration tests
66+
67+
### Working with Item Banks
68+
- Item banks are provided by FittedItemBanks.jl
69+
- See examples in `test/dummy.jl` for creating test data
70+
71+
## Code Style Guidelines
72+
73+
1. Use Julia formatting conventions
74+
2. Follow the [SciML style](https://docs.sciml.ai/SciMLStyle/stable/)
75+
2. Add docstrings for public functions using DocStringExtensions
76+
3. Keep functions focused and modular
77+
4. Use meaningful variable names (except in performance-critical inner loops)
78+
79+
## Performance Considerations
80+
81+
- The package is designed for speed with large item banks
82+
- Use `@benchmark` from BenchmarkTools.jl for performance testing
83+
- Profile with `profile/next_items.jl` for optimization work
84+
- Other benchmarks are in the external CATExperiment
85+
- This area is in flux and could be improved
86+
87+
## Dependencies
88+
89+
Key dependencies include:
90+
- FittedItemBanks.jl for item bank management
91+
- PsychometricsBazaarBase.jl for baseline utilities for the PsychometricsBazaar ecosystem
92+
- Distributions.jl for statistical distributions
93+
- ForwardDiff.jl for automatic differentiation
94+
95+
## Useful Resources
96+
97+
- [Package Documentation](https://juliapsychometricsbazaar.github.io/ComputerAdaptiveTesting.jl/dev/)
98+
- [catR](https://cran.r-project.org/web/packages/catR/index.html) - R alternative
99+
- [mirtCAT](https://cran.r-project.org/web/packages/mirtCAT/index.html) - Another R alternative
100+
101+
## Getting Help
102+
103+
- Check existing issues on GitHub
104+
- Review the test files for usage examples
105+
- Consult the documentation for API details

0 commit comments

Comments
 (0)