Skip to content

Commit e0f2a7a

Browse files
mpfmpf
committed
chore: remove experiments directory and Claude-specific files from main branch
1 parent d202fa5 commit e0f2a7a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+2569
-9225
lines changed

‎.taskmaster/config.json‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{
2+
"models": {
3+
"main": {
4+
"provider": "anthropic",
5+
"modelId": "claude-3-7-sonnet-20250219",
6+
"maxTokens": 120000,
7+
"temperature": 0.2
8+
},
9+
"research": {
10+
"provider": "perplexity",
11+
"modelId": "sonar-pro",
12+
"maxTokens": 8700,
13+
"temperature": 0.1
14+
},
15+
"fallback": {
16+
"provider": "anthropic",
17+
"modelId": "claude-3-5-sonnet-20240620",
18+
"maxTokens": 8192,
19+
"temperature": 0.1
20+
}
21+
},
22+
"global": {
23+
"logLevel": "info",
24+
"debug": false,
25+
"defaultSubtasks": 5,
26+
"defaultPriority": "medium",
27+
"projectName": "Taskmaster",
28+
"ollamaBaseURL": "http://localhost:11434/api",
29+
"bedrockBaseURL": "https://bedrock.us-east-1.amazonaws.com",
30+
"azureOpenaiBaseURL": "https://your-endpoint.openai.azure.com/",
31+
"userId": "1234567890"
32+
}
33+
}

‎.taskmaster/docs/prd.txt‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
<context>
2+
# Overview
3+
DualPerspective.jl is a Julia package that provides a novel approach to solving regularized optimization problems by reformulating them as instances of the regularized relative-entropy problem. The package transforms various problem classes (nonnegative least-squares, linear programming, optimal transport, and Hausdorff moment recovery) into a unified framework that exploits the conic extension of the probability simplex.
4+
5+
The key innovation is the dual perspective approach that reformulates problems to have globally Lipschitz-smooth and strongly-convex objectives with uniformly bounded Hessian matrices, enabling efficient algorithms with strong convergence guarantees.
6+
7+
# Core Features
8+
## Unified Problem Formulation
9+
- Reformulates various optimization problems as regularized relative-entropy problems
10+
- Leverages the conic extension of the probability simplex for nonnegative cone constraints
11+
- Provides a compactified problem formulation that enables efficient solution techniques
12+
13+
## Dual Perspective Model (DPModel)
14+
- Extends AbstractNLPModel interface for seamless integration with Julia optimization ecosystem
15+
- Encapsulates data for regularized relative-entropy problems
16+
- Supports flexible regularization parameters and custom reference points
17+
18+
## Advanced Optimization Algorithms
19+
- Trust-region Newton methods with quadratic convergence guarantees
20+
- Gauss-Newton solver with multiple linesearch strategies (Armijo-Goldstein, backtracking)
21+
- Level-set methods for constrained problems
22+
- Sequential scaling algorithms for improved numerical stability
23+
24+
## Mathematical Foundations
25+
- Implements Kullback-Leibler divergence and its perspective transform
26+
- Provides primal and dual objective functions with analytical gradients and Hessians
27+
- Includes primal-from-dual solution mapping for recovering original variables
28+
29+
## Cross-Language Support
30+
- Native Julia implementation for maximum performance
31+
- Python interface via JuliaCall for broader accessibility
32+
- MATLAB compatibility for researchers using legacy code
33+
34+
# User Experience
35+
## Target Users
36+
- Researchers in optimization and computational mathematics
37+
- Machine learning practitioners working with regularized problems
38+
- Engineers solving large-scale transportation and resource allocation problems
39+
- Scientists requiring robust solutions to ill-conditioned inverse problems
40+
41+
## Key User Flows
42+
1. **Problem Setup**: Users define their optimization problem using the DPModel constructor with constraint matrix A, target vector b, and optional parameters
43+
2. **Algorithm Selection**: Choose appropriate solver based on problem characteristics (Newton methods for smooth problems, Gauss-Newton for least-squares structure)
44+
3. **Solution Retrieval**: Obtain both primal and dual solutions with convergence diagnostics
45+
4. **Analysis**: Access detailed convergence metrics, optimality measures, and solution quality indicators
46+
47+
## API Design Principles
48+
- Intuitive constructors that mirror mathematical notation
49+
- Consistent interface across different problem types
50+
- Comprehensive logging and debugging capabilities
51+
- Integration with Julia's optimization ecosystem (NLPModels.jl)
52+
</context>
53+
<PRD>
54+
# Technical Architecture
55+
## System Components
56+
### Core Models
57+
- `DPModel`: Main model type extending AbstractNLPModel
58+
- `OTModel`: Specialized model for optimal transport problems
59+
- `LPModel`: Linear programming with entropic regularization
60+
- `SSModel`: Self-scaling model variant
61+
62+
### Objectives and Operations
63+
- Primal objective function with KL divergence regularization
64+
- Dual objective function with log-sum-exp operations
65+
- Value function computation for compactified problems
66+
- Gradient and Hessian computations with numerical stability
67+
68+
### Solvers
69+
- Newton-CG: Newton method with conjugate gradient for linear systems
70+
- Newton-LS: Newton method with linesearch strategies
71+
- Gauss-Newton: Specialized solver for nonlinear least-squares structure
72+
- Sequential scaling: Adaptive scaling for improved conditioning
73+
- Level-set methods: Constraint-aware optimization
74+
75+
### Utilities
76+
- Log-sum-exp implementations with numerical stability
77+
- Preconditioners for iterative solvers
78+
- Linear operators for matrix-free computations
79+
- Convergence diagnostics and logging
80+
81+
## Data Models
82+
- Sparse and dense matrix support via SparseArrays.jl
83+
- Efficient storage of probability distributions
84+
- Lazy evaluation of Jacobian operators
85+
- In-place operations for memory efficiency
86+
87+
## Infrastructure Requirements
88+
- Julia 1.6+ for modern language features
89+
- NLPModels.jl for optimization interface
90+
- Krylov.jl for iterative linear solvers
91+
- LinearOperators.jl for matrix-free operations
92+
- Logging.jl for configurable output
93+
- Test infrastructure with comprehensive coverage
94+
95+
# Development Roadmap
96+
## Phase 1: Core Functionality Completion
97+
- Complete implementation of all solver variants
98+
- Finalize API for DPModel and related types
99+
- Implement remaining analytical properties (Hessian computations)
100+
- Add comprehensive error handling and input validation
101+
- Create extensive unit tests for all components
102+
103+
## Phase 2: Documentation and Theory
104+
- Complete the theory.md documentation sections:
105+
- Convergence analysis with theoretical guarantees
106+
- Relationship to interior point and entropic regularization methods
107+
- Implementation details and algorithmic choices
108+
- Numerical experiments and benchmarks
109+
- Add docstrings to all exported functions and types
110+
- Create tutorial notebooks demonstrating key features
111+
- Write mathematical derivations for key algorithms
112+
113+
## Phase 3: Performance Optimization
114+
- Profile and optimize hot paths in solver implementations
115+
- Implement specialized methods for structured problems
116+
- Add GPU support for large-scale problems
117+
- Create benchmark suite comparing to state-of-the-art solvers
118+
- Optimize memory allocation patterns
119+
120+
## Phase 4: Advanced Features
121+
- Implement warm-start capabilities for sequential problems
122+
- Add support for box constraints and general convex sets
123+
- Create adaptive regularization strategies
124+
- Implement parallel variants for distributed computing
125+
- Add stochastic/incremental variants for large-scale problems
126+
127+
## Phase 5: Integration and Ecosystem
128+
- Register package in Julia General registry
129+
- Create JuMP.jl extension for modeling interface
130+
- Integrate with Convex.jl for problem specification
131+
- Add Plots.jl recipes for visualization
132+
- Create interfaces to popular optimization benchmarks
133+
134+
## Phase 6: Applications and Examples
135+
- Optimal transport examples with visualization
136+
- Machine learning applications (regularized regression, classification)
137+
- Image processing examples (denoising, deblurring)
138+
- Portfolio optimization with transaction costs
139+
- Network flow problems with congestion
140+
141+
# Logical Dependency Chain
142+
## Foundation (Must be completed first)
143+
1. Core DPModel implementation with proper AbstractNLPModel interface
144+
2. Basic primal and dual objective functions
145+
3. Gradient computations with numerical stability
146+
4. Essential utility functions (log-sum-exp, KL divergence)
147+
148+
## Core Algorithms (Depends on Foundation)
149+
1. Basic Newton method implementation
150+
2. Linesearch strategies for globalization
151+
3. Conjugate gradient for Newton systems
152+
4. Convergence criteria and stopping rules
153+
154+
## Advanced Solvers (Depends on Core Algorithms)
155+
1. Gauss-Newton for least-squares structure
156+
2. Trust-region methods for robustness
157+
3. Sequential scaling for conditioning
158+
4. Level-set methods for constraints
159+
160+
## Testing and Validation (Parallel with development)
161+
1. Unit tests for individual components
162+
2. Integration tests for complete workflows
163+
3. Numerical accuracy tests
164+
4. Performance benchmarks
165+
166+
## Documentation and Examples (After Core Complete)
167+
1. API documentation with examples
168+
2. Mathematical theory documentation
169+
3. Tutorial notebooks
170+
4. Application examples
171+
172+
## Distribution and Integration (Final Phase)
173+
1. Package registration
174+
2. Python interface finalization
175+
3. Continuous integration setup
176+
4. Community engagement
177+
178+
# Risks and Mitigations
179+
## Technical Challenges
180+
- **Numerical Stability**: The log-sum-exp operations can suffer from overflow/underflow
181+
- Mitigation: Implement robust numerical techniques with careful scaling
182+
183+
- **Convergence Issues**: Newton methods may fail for poorly conditioned problems
184+
- Mitigation: Implement robust globalization strategies and preconditioners
185+
186+
- **Performance Bottlenecks**: Large-scale problems may be computationally intensive
187+
- Mitigation: Profile early and often, implement matrix-free operations
188+
189+
## Algorithm Complexity
190+
- **Theoretical Guarantees**: Proving convergence for all problem classes
191+
- Mitigation: Focus on well-studied cases first, collaborate with theorists
192+
193+
- **Parameter Selection**: Choosing appropriate regularization parameters
194+
- Mitigation: Implement adaptive strategies and provide guidance
195+
196+
## Ecosystem Integration
197+
- **API Stability**: Changes may break dependent packages
198+
- Mitigation: Follow semantic versioning, maintain backwards compatibility
199+
200+
- **Cross-Language Interface**: Python/MATLAB integration complexity
201+
- Mitigation: Start with minimal interface, expand based on user needs
202+
203+
# Appendix
204+
## Mathematical Background
205+
The package implements the regularized relative-entropy problem:
206+
min_{x∈ℝⁿ₊} ⟨c, x⟩ + 1/(2λ)||Ax - b||²_{C⁻¹} + ∑ⱼ xⱼlog(xⱼ/x̄ⱼ)
207+
208+
Key insight: ℝⁿ₊ = ⋃_{τ≥0} τΔⁿ where Δⁿ is the probability simplex
209+
210+
## Related Work
211+
- Interior point methods for conic programming
212+
- Entropic regularization in optimal transport
213+
- Proximal methods for composite optimization
214+
- Trust-region methods for nonlinear optimization
215+
216+
## Performance Targets
217+
- Solve medium-scale problems (n~10⁴) in seconds
218+
- Handle sparse problems with millions of variables
219+
- Achieve 1e-8 relative accuracy for well-conditioned problems
220+
- Scale linearly with problem size for structured problems
221+
222+
## Testing Strategy
223+
- Unit tests achieving >90% code coverage
224+
- Integration tests for complete workflows
225+
- Stress tests for numerical stability
226+
- Performance regression tests
227+
- Comparison with established solvers (MOSEK, Gurobi for applicable problems)
228+
</PRD>

‎.taskmaster/tasks/task_001.txt‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Task ID: 1
2+
# Title: Implement DPModel Core Structure
3+
# Status: pending
4+
# Dependencies: None
5+
# Priority: high
6+
# Description: Create the main DPModel type that extends AbstractNLPModel from NLPModels.jl with all necessary fields and constructors.
7+
# Details:
8+
Create the DPModel struct with fields for:
9+
- Constraint matrix A (supporting both sparse and dense formats)
10+
- Target vector b
11+
- Cost vector c
12+
- Regularization parameter λ
13+
- Reference point x̄
14+
- Problem dimensions
15+
- Internal state for caching
16+
17+
Implement constructors with sensible defaults and type stability. Ensure proper inheritance from AbstractNLPModel to maintain compatibility with the Julia optimization ecosystem. Include validation for inputs (dimensions, positivity constraints).
18+
19+
Example structure:
20+
```julia
21+
struct DPModel <: AbstractNLPModel
22+
meta::NLPModelMeta
23+
counters::Counters
24+
A::Union{Matrix, SparseMatrixCSC}
25+
b::Vector
26+
c::Vector
27+
λ::Real
28+
x̄::Vector
29+
# Additional fields for caching
30+
# ...
31+
32+
# Inner constructor with validation
33+
function DPModel(A, b, c, λ, x̄; kwargs...)
34+
# Dimension validation
35+
# Parameter validation
36+
# Initialize meta and counters
37+
# Return new instance
38+
end
39+
end
40+
```
41+
42+
# Test Strategy:
43+
Create unit tests that verify:
44+
1. Proper construction with various input types
45+
2. Error handling for invalid inputs (dimension mismatch, negative λ)
46+
3. Inheritance from AbstractNLPModel
47+
4. Proper initialization of meta and counters
48+
5. Compatibility with NLPModels.jl interface functions

‎.taskmaster/tasks/task_002.txt‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Task ID: 2
2+
# Title: Implement Basic Utility Functions
3+
# Status: pending
4+
# Dependencies: None
5+
# Priority: high
6+
# Description: Develop essential utility functions for numerical stability, including log-sum-exp operations and Kullback-Leibler divergence calculations.
7+
# Details:
8+
Implement the following utility functions with numerical stability considerations:
9+
10+
1. Numerically stable log-sum-exp:
11+
```julia
12+
function logsumexp(x::AbstractVector)
13+
u = maximum(x)
14+
return u + log(sum(exp.(x .- u)))
15+
end
16+
```
17+
18+
2. Kullback-Leibler divergence between probability distributions:
19+
```julia
20+
function kl_divergence(p::AbstractVector, q::AbstractVector)
21+
result = 0.0
22+
for (pi, qi) in zip(p, q)
23+
if pi > 0
24+
result += pi * log(pi / qi)
25+
end
26+
end
27+
return result
28+
end
29+
```
30+
31+
3. Scaled KL divergence for the regularization term:
32+
```julia
33+
function scaled_kl(x::AbstractVector, x̄::AbstractVector)
34+
sum(xi > 0 ? xi * log(xi / x̄i) : 0.0 for (xi, x̄i) in zip(x, x̄))
35+
end
36+
```
37+
38+
4. Safe logarithm and division operations:
39+
```julia
40+
safe_log(x) = x > 0 ? log(x) : -Inf
41+
safe_div(x, y) = y != 0 ? x / y : (x == 0 ? 0.0 : Inf * sign(x))
42+
```
43+
44+
Ensure all functions handle edge cases appropriately and maintain numerical stability.
45+
46+
# Test Strategy:
47+
Test utility functions with:
48+
1. Normal input ranges
49+
2. Edge cases (zeros, very small values, very large values)
50+
3. Verify against known analytical results
51+
4. Test for numerical stability with ill-conditioned inputs
52+
5. Benchmark performance against naive implementations

0 commit comments

Comments
 (0)