Skip to content

Refactor/generalization of bellman structure#97

Draft
JeffersonYeh wants to merge 23 commits intomainfrom
qvalue
Draft

Refactor/generalization of bellman structure#97
JeffersonYeh wants to merge 23 commits intomainfrom
qvalue

Conversation

@JeffersonYeh
Copy link
Copy Markdown
Collaborator

@JeffersonYeh JeffersonYeh commented Mar 11, 2026 •
edited
Loading

  1. refactors the bellman update process to perform action selection and compute expectation separately. This generalizes the architecture to State-Action value functions
  2. state-action update sequence at each iteration generalized through sampling functions

@JeffersonYeh JeffersonYeh self-assigned this Mar 11, 2026
JeffersonYeh and others added 22 commits March 11, 2026 14:57
@JeffersonYeh
rename _bellman_helper! to _expectation_bellman!
53ba3ab
@JeffersonYeh
rename step! to bellman_update!
910ea1a
@JeffersonYeh
Rename bellman_precomputation, state_bellman, state_action_bellman to…
f4f8b12 
… repsective expectation funcs
@JeffersonYeh
Refactor ValueFunction into dedicated file
3e80ebe
@JeffersonYeh
Add dinstinction between StateAction and State value function + add i…
7209be4 
…ntermediate values
@JeffersonYeh
Fix intermediate value function dimension
33ebdb0
@JeffersonYeh
Split expectation and strategy computations for Dense IMDP
511a901
@JeffersonYeh
rename bellman function to expectation
3354b50
@JeffersonYeh
speed performance improvements + minor documentation update
4a3c494
@JeffersonYeh
Fix IMDP tests
45642f1
@JeffersonYeh
Merge pull request #98 from Zinoex/decouple-strategy
7573146 
Split expectation and strategy computations
@JeffersonYeh
init generalizing state update order
164846d
@JeffersonYeh
Add custom state sequence function
d80c6e2
@JeffersonYeh
Reworked sampling_strategy and update_sequence
655cc4b
@JeffersonYeh
sampling strategies selection through algorithms
335127a
@JeffersonYeh
remove state_expectation!
f8061aa
@JeffersonYeh
in ext/ rename _bellman_helper! to _expectation_helper!
eccd55b
@JeffersonYeh
threaded expectation loop via custom iterator
0227b66
@JeffersonYeh
Rework custom iterators
7ae507a
@JeffersonYeh
remove threaded state from sampling function
a37e82b
@JeffersonYeh
rework given sequence sampling function
37214c6
@JeffersonYeh
Merge pull request #99 from Zinoex/qvalue-sampling-func
1aa3a07 
Generalize beyond exhaustive search
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@JeffersonYeh JeffersonYeh

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@JeffersonYeh