Skip to content

Refactor/generalization of bellman structure #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
JeffersonYeh wants to merge 23 commits into
base: main
Choose a base branch
from
Draft

Refactor/generalization of bellman structure #97

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6997d71
rename bellman! to expectation!
JeffersonYeh Mar 11, 2026
53ba3ab
rename _bellman_helper! to _expectation_bellman!
JeffersonYeh Mar 11, 2026
910ea1a
rename step! to bellman_update!
JeffersonYeh Mar 11, 2026
f4f8b12
Rename bellman_precomputation, state_bellman, state_action_bellman to…
JeffersonYeh Mar 11, 2026
3e80ebe
Refactor ValueFunction into dedicated file
JeffersonYeh Mar 11, 2026
7209be4
Add dinstinction between StateAction and State value function + add i…
JeffersonYeh Mar 11, 2026
33ebdb0
Fix intermediate value function dimension
JeffersonYeh Mar 12, 2026
511a901
Split expectation and strategy computations for Dense IMDP
JeffersonYeh Mar 12, 2026
3354b50
rename bellman function to expectation
JeffersonYeh Mar 12, 2026
4a3c494
speed performance improvements + minor documentation update
JeffersonYeh Mar 18, 2026
45642f1
Fix IMDP tests
JeffersonYeh Mar 18, 2026
7573146
Merge pull request #98 from Zinoex/decouple-strategy
JeffersonYeh Mar 18, 2026
164846d
init generalizing state update order
JeffersonYeh Mar 18, 2026
d80c6e2
Add custom state sequence function
JeffersonYeh Mar 19, 2026
655cc4b
Reworked sampling_strategy and update_sequence
JeffersonYeh Mar 20, 2026
335127a
sampling strategies selection through algorithms
JeffersonYeh Mar 20, 2026
f8061aa
remove state_expectation!
JeffersonYeh Mar 20, 2026
eccd55b
in ext/ rename _bellman_helper! to _expectation_helper!
JeffersonYeh Mar 23, 2026
0227b66
threaded expectation loop via custom iterator
JeffersonYeh Mar 23, 2026
7ae507a
Rework custom iterators
JeffersonYeh Mar 23, 2026
a37e82b
remove threaded state from sampling function
JeffersonYeh Mar 23, 2026
37214c6
rework given sequence sampling function
JeffersonYeh Mar 23, 2026
1aa3a07
Merge pull request #99 from Zinoex/qvalue-sampling-func
JeffersonYeh Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ext/cuda/bellman/dense.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
function IntervalMDP._bellman_helper!(
function IntervalMDP._expectation_helper!(
workspace::CuDenseOMaxWorkspace,
strategy_cache::IntervalMDP.AbstractStrategyCache,
Vres::AbstractVector{Tv},
Expand Down
2 changes: 1 addition & 1 deletion ext/cuda/bellman/factored.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
function IntervalMDP._bellman_helper!(
function IntervalMDP._expectation_helper!(
workspace::CuFactoredOMaxWorkspace,
strategy_cache::IntervalMDP.AbstractStrategyCache,
Vres::AbstractArray{Tv},
Expand Down
2 changes: 1 addition & 1 deletion ext/cuda/bellman/sparse.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
function IntervalMDP._bellman_helper!(
function IntervalMDP._expectation_helper!(
workspace::CuSparseOMaxWorkspace,
strategy_cache::IntervalMDP.AbstractStrategyCache,
Vres::AbstractVector{Tv},
Expand Down
2 changes: 2 additions & 0 deletions src/IntervalMDP.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ include("threading.jl")
include("workspace.jl")
include("strategy_cache.jl")
include("bellman.jl")
include("value.jl")
include("state_sampling.jl")

include("robust_value_iteration.jl")

Expand Down
Loading
Loading