Merge pull request #38 from JuliaDecisionFocusedLearning/improve-dvsp

Cleanup DVSP and better documentation

Author: BatyLeo

README.md

@@ -6,15 +6,84 @@
 [![Coverage](https://codecov.io/gh/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl/branch/main/graph/badge.svg)](https://app.codecov.io/gh/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl)
 [![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/JuliaDiff/BlueStyle)
 
-This repository contains a collection of benchmark problems for decision-focused learning algorithms.
-It provides a common interface for creating datasets, associated statistical models and combinatorial optimization maximizers for building decision-focused learning pipelines.
-They can be used for instance as benchmarks for tools in [InferOpt.jl](https://github.com/JuliaDecisionFocusedLearning/InferOpt.jl), but can be used in any other context as well.
-
-Currently, this package provides the following benchmark problems (many more to come!):
-- `SubsetSelectionBenchmark`: a minimalist subset selection problem.
-- `FixedSizeShortestPathBenchmark`: shortest path problem with on a graph with fixed size.
-- `WarcraftBenchmark`: shortest path problem on image maps
-- `PortfolioOptimizationBenchmark`: portfolio optimization problem.
-- `StochasticVehicleSchedulingBenchmark`: stochastic vehicle scheduling problem.
-
-See the [documentation](https://JuliaDecisionFocusedLearning.github.io/DecisionFocusedLearningBenchmarks.jl/stable/) for more details.
+## What is Decision-Focused Learning?
+
+Decision-focused learning (DFL) is a paradigm that integrates machine learning prediction with combinatorial optimization to make better decisions under uncertainty. Unlike traditional "predict-then-optimize" approaches that optimize prediction accuracy independently of downstream decision quality, DFL directly optimizes end-to-end decision performance.
+
+A typical DFL algorithm involves training a parametrized policy that combines a statistical predictor with an optimization component:
+```math
+\xrightarrow[\text{Instance}]{x}
+\fbox{Statistical model $\varphi_w$}
+\xrightarrow[\text{Parameters}]{\theta}
+\fbox{CO algorithm $f$}
+\xrightarrow[\text{Solution}]{y}
+```
+
+Where:
+- **Instance** $x$: input data (e.g., features, context)
+- **Statistical model** $\varphi_w$: machine learning predictor (e.g., neural network)
+- **Parameters** $\theta$: predicted parameters for the optimization problem
+- **CO algorithm** $f$: combinatorial optimization solver
+- **Solution** $y$: final decision/solution
+
+## Package Overview
+
+**DecisionFocusedLearningBenchmarks.jl** provides a comprehensive collection of benchmark problems for evaluating decision-focused learning algorithms. The package offers:
+
+- **Standardized benchmark problems** spanning diverse application domains
+- **Common interfaces** for datasets, statistical models, and optimization components  
+- **Ready-to-use pipelines** compatible with [InferOpt.jl](https://github.com/JuliaDecisionFocusedLearning/InferOpt.jl) and the whole [JuliaDecisionFocusedLearning](https://github.com/JuliaDecisionFocusedLearning) ecosystem
+- **Evaluation tools** for comparing algorithm performance
+
+## Benchmark Categories
+
+The package organizes benchmarks into three main categories based on their problem structure:
+
+### Static Benchmarks (`AbstractBenchmark`)
+Single-stage optimization problems with no randomness involved:
+- [`ArgmaxBenchmark`](@ref): argmax toy problem
+- [`Argmax2DBenchmark`](@ref): 2D argmax toy problem
+- [`RankingBenchmark`](@ref): ranking problem
+- [`SubsetSelectionBenchmark`](@ref): select optimal subset of items
+- [`PortfolioOptimizationBenchmark`](@ref): portfolio optimization problem
+- [`FixedSizeShortestPathBenchmark`](@ref): find shortest path on grid graphs with fixed size
+- [`WarcraftBenchmark`](@ref): shortest path on image maps
+
+### Stochastic Benchmarks (`AbstractStochasticBenchmark`)  
+Single-stage problems with random noise affecting the objective:
+- [`StochasticVehicleSchedulingBenchmark`](@ref): stochastic vehicle scheduling under delay uncertainty
+
+### Dynamic Benchmarks (`AbstractDynamicBenchmark`)
+Multi-stage sequential decision-making problems:
+- [`DynamicVehicleSchedulingBenchmark`](@ref): multi-stage vehicle scheduling under customer uncertainty
+- [`DynamicAssortmentBenchmark`](@ref): sequential product assortment selection
+
+## Getting Started
+
+In a few lines of code, you can create benchmark instances, generate datasets, initialize learning components, and evaluate performance, using the same syntax across all benchmarks:
+
+```julia
+using DecisionFocusedLearningBenchmarks
+
+# Create a benchmark instance for the argmax problem
+benchmark = ArgmaxBenchmark()
+
+# Generate training data
+dataset = generate_dataset(benchmark, 100)
+
+# Initialize policy components
+model = generate_statistical_model(benchmark)
+maximizer = generate_maximizer(benchmark)
+
+# Training algorithm you want to use
+# ... your training code here ...
+
+# Evaluate performance
+gap = compute_gap(benchmark, dataset, model, maximizer)
+```
+
+## Related Packages
+
+This package is part of the [JuliaDecisionFocusedLearning](https://github.com/JuliaDecisionFocusedLearning) organization, and built to be compatible with other packages in the ecosystem:
+- **[InferOpt.jl](https://github.com/JuliaDecisionFocusedLearning/InferOpt.jl)**: differentiable optimization layers and losses for decision-focused learning
+- **[DecisionFocusedLearningAlgorithms.jl](https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningAlgorithms.jl)**: collection of generic black-box implementations of decision-focused learning algorithms

docs/make.jl

@@ -9,12 +9,11 @@ tutorial_dir = joinpath(@__DIR__, "src", "tutorials")
 benchmarks_dir = joinpath(@__DIR__, "src", "benchmarks")
 api_dir = joinpath(@__DIR__, "src", "api")
 
-api_files = map(x -> joinpath("api", x), readdir(api_dir))
 tutorial_files = readdir(tutorial_dir)
 md_tutorial_files = [split(file, ".")[1] * ".md" for file in tutorial_files]
 benchmark_files = [joinpath("benchmarks", e) for e in readdir(benchmarks_dir)]
 
-include_tutorial = true
+include_tutorial = false
 
 if include_tutorial
     for file in tutorial_files
@@ -29,10 +28,13 @@ makedocs(;
     sitename="DecisionFocusedLearningBenchmarks.jl",
     format=Documenter.HTML(; size_threshold=typemax(Int)),
     pages=[
-        "Home" => "index.md",
+        "Home" => [
+            "Getting started" => "index.md",
+            "Understanding Benchmark Interfaces" => "benchmark_interfaces.md",
+        ],
         "Tutorials" => include_tutorial ? md_tutorial_files : [],
         "Benchmark problems list" => benchmark_files,
-        "API reference" => "api/api.md",
+        "API reference" => "api.md",
     ],
 )
 

docs/src/api/api.md renamed to docs/src/api.md

@@ -2,175 +2,131 @@
 
 ## Interface
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Utils]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Utils]
 Public = false
 ```
 
 ## Argmax2D
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Argmax2D]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Argmax2D]
 Public = false
 ```
 
 ## Argmax
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Argmax]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Argmax]
 Public = false
 ```
 
 ## Dynamic Vehicle Scheduling
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.DynamicVehicleScheduling]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.DynamicVehicleScheduling]
 Public = false
 ```
 
 ## Dynamic Assortment
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.DynamicAssortment]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.DynamicAssortment]
 Public = false
 ```
 
 ## Fixed-size shortest path
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.FixedSizeShortestPath]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.FixedSizeShortestPath]
 Public = false
 ```
 
 ## Portfolio Optimization
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.PortfolioOptimization]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.PortfolioOptimization]
 Public = false
 ```
 
 ## Ranking
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Ranking]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Ranking]
 Public = false
 ```
 
 ## Subset selection
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.SubsetSelection]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.SubsetSelection]
 Public = false
 ```
 
 ## Stochastic Vehicle Scheduling
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.StochasticVehicleScheduling]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.StochasticVehicleScheduling]
 Public = false
 ```
 
 ## Warcraft
 
-### Public
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Warcraft]
 Private = false
 ```
 
-### Private
-
 ```@autodocs
 Modules = [DecisionFocusedLearningBenchmarks.Warcraft]
 Public = false