Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
236 commits
Select commit Hold shift + click to select a range
25033d5
added objective function api
rick12000 Jan 24, 2025
4ab8ba9
temporarily remove quantile forest dependancy
rick12000 Jan 28, 2025
ec15a12
fix pd dummy edge case + unit tests
rick12000 Jan 29, 2025
b3c018e
add support for warm starting
rick12000 Feb 2, 2025
c37c6da
fix runtime management + gp hashing
rick12000 Feb 2, 2025
bd8bc99
increase n configs to predict + temp comment out pandas infer + fix o…
rick12000 Feb 2, 2025
0943583
add lasso and knn quantile reg + add max iter
rick12000 Feb 9, 2025
0a5b103
major refactor + add samplers
rick12000 Feb 12, 2025
0c24cb9
update sampling
rick12000 Feb 14, 2025
2c3ff13
update sampler
rick12000 Feb 14, 2025
44c85de
add adaptive intervals
rick12000 Feb 15, 2025
141b47c
refactor sampler interactions + add quantile estimators
rick12000 Feb 17, 2025
9562a61
refactor samplers + add ts for lw
rick12000 Feb 18, 2025
5c296ba
fix bad sampler logic
rick12000 Feb 18, 2025
911f06b
add optimistic thompson sampling
rick12000 Feb 18, 2025
fdc02c5
add tuning methods + runtime tracking
rick12000 Feb 19, 2025
f1d5749
add breach rate tracking
rick12000 Feb 19, 2025
ca7bc12
refactor estimation.py
rick12000 Feb 21, 2025
de5f541
refactor searcher classes
rick12000 Feb 21, 2025
167e24f
add single fit searcher
rick12000 Feb 21, 2025
7d3f918
fixes + track estimator error
rick12000 Feb 22, 2025
a7ad0e1
improve dtaci but still needs fix + fix lw alpha
rick12000 Mar 1, 2025
300a027
fix update interval for q methods
rick12000 Mar 2, 2025
3bbbf81
refactor + partial dtaci fix
rick12000 Mar 2, 2025
10b229f
improve random stating, but not fixed yet
rick12000 Mar 2, 2025
4e17064
wip - updating tests
rick12000 Mar 4, 2025
e75404b
wip - updating tests
rick12000 Mar 4, 2025
b880463
wip - updating tests
rick12000 Mar 4, 2025
49c6c66
spin out locally weighted searcher
rick12000 Mar 4, 2025
ca3bc2e
refactor estimation
rick12000 Mar 5, 2025
5fee160
refactor quantile estimators
rick12000 Mar 5, 2025
b948351
refactor quantile estimators
rick12000 Mar 5, 2025
7117248
add conformalization unit tests
rick12000 Mar 5, 2025
2beaeae
add unit tests
rick12000 Mar 5, 2025
8dcc8f3
update lgbm defaults + knn ball tree
rick12000 Mar 7, 2025
ce9f25a
update qrf and qgbm defaults to speed up
rick12000 Mar 7, 2025
bcb4d6d
update default params
rick12000 Mar 7, 2025
f1de8c2
optimize dict checks + fix estimation params
rick12000 Mar 7, 2025
0949c8b
fix qgbm params
rick12000 Mar 7, 2025
d6a6910
optimize configuration collection
rick12000 Mar 7, 2025
faca453
refactor + add warm start input
rick12000 Mar 8, 2025
3070b1d
commit missing ranges module
rick12000 Mar 8, 2025
98b607f
fix ucb + add pessimistic ucb
rick12000 Mar 8, 2025
6d1be36
fix samplers + bound calculations for all searchers
rick12000 Mar 8, 2025
dc22bd3
fix ts sampling in lw
rick12000 Mar 8, 2025
d1ca333
improve speed
rick12000 Mar 8, 2025
f134fd3
fix single fit point estimator + fix multi fit incrementality
rick12000 Mar 8, 2025
4f1e684
add and update unit tests
rick12000 Mar 8, 2025
272eec0
add quantile lasso back
rick12000 Mar 9, 2025
060d55b
add ensembles
rick12000 Mar 9, 2025
90fd34f
partial fix for ensembling, still needs fixing
rick12000 Mar 9, 2025
9eaa07a
fix all ensemblers + add pe ensemble
rick12000 Mar 10, 2025
da0c5cd
remove log files
rick12000 Mar 10, 2025
f916d19
fixes
rick12000 Mar 11, 2025
3906cbb
fix lw unit testing
rick12000 Mar 11, 2025
4dc23e1
improve point estimator syntax
rick12000 Mar 11, 2025
80f663f
clean ups
rick12000 Mar 11, 2025
3867bc6
fix quantile lasso + change ucb schedules
rick12000 Mar 15, 2025
2dc2bd2
spin out sampling and add unit tests
rick12000 Mar 18, 2025
73556db
refine unit tests for conformalization and sampling + fix tuning for …
rick12000 Mar 20, 2025
7ada028
interim refactor of estimation + fix dtaci
rick12000 Mar 21, 2025
5409a1a
fix dtaci, but still trailing aci performance
rick12000 Mar 22, 2025
3eeec35
refactor estimation frameworks
rick12000 Mar 22, 2025
6098186
interim refactor of estimation frameworks
rick12000 Mar 23, 2025
d66bf10
update single fit quantile ensemble error
rick12000 Mar 23, 2025
5a72e3e
broken, refactoring sampling, estimation
rick12000 Mar 23, 2025
2c5e5b1
refactors, dtaci still needs integration
rick12000 Mar 24, 2025
71a6faa
refactor ensembles, remove option to tune them
rick12000 Mar 25, 2025
0cec819
refactor folder structure
rick12000 Mar 25, 2025
f0961df
add betas for dtaci + refactor tuner
rick12000 Mar 25, 2025
0a82400
update unit tests
rick12000 Mar 26, 2025
c84f0a0
save point, refactor sampling and adaptation
rick12000 Mar 26, 2025
7d8cdb2
save point, refactor acquisition and conformalization
rick12000 Mar 27, 2025
1dbcb80
revisions to conformalization, estimation and quantile estimator + up…
rick12000 Mar 28, 2025
f525d0c
refactor and unit testing of estimation, acquisition, quantile estima…
rick12000 Mar 28, 2025
4551674
clean up ensembling
rick12000 Mar 28, 2025
5a9bbbb
ensembling changes + unit tests + refactors
rick12000 Mar 28, 2025
95266c5
misc refactors
rick12000 Mar 29, 2025
1652a15
misc refactors
rick12000 Mar 29, 2025
1bc982f
refactor tuning
rick12000 Mar 29, 2025
9c78b3e
fixes
rick12000 Mar 30, 2025
eb4a550
fix interval update sign
rick12000 Mar 30, 2025
1dd117f
add linear interpolation of non conform scores
rick12000 Mar 30, 2025
6000005
add ucb schedule + store errors for paper
rick12000 Mar 30, 2025
968c632
add adaptive lbc
rick12000 Apr 1, 2025
6598879
revamp tuning count optimization - wip
rick12000 Apr 1, 2025
6a97269
store incremental best configs
rick12000 Apr 1, 2025
770002b
update tuning schedules
rick12000 Apr 7, 2025
b192b6c
update surrogate tuner + fix estimation and configs
rick12000 Apr 10, 2025
a14809f
add quantile GP + update ensemble configs
rick12000 Apr 12, 2025
9fc3ef1
add quantile GP + update ensemble configs
rick12000 Apr 13, 2025
671c808
add es and ei
rick12000 Apr 15, 2025
9f368e7
fixes to expected improvementand es - in progress
rick12000 Apr 19, 2025
9244a52
reduced run time of unit tests and fixed information gain test failure
rick12000 Apr 20, 2025
a564985
fix reproduce ability and update alphas
rick12000 Apr 21, 2025
277da52
changed kernel
rick12000 Apr 29, 2025
2a7e3e1
parallelize es + fix starting ei
rick12000 May 1, 2025
bba6323
add entropy search strategies
rick12000 May 2, 2025
39c20d5
add max value entropy search
rick12000 May 3, 2025
85b03c8
clean up
rick12000 May 3, 2025
81c379b
improve entropy sampler
rick12000 May 5, 2025
b7c2ef7
added .toml + added resampled hyperparameter search + refactored entr…
rick12000 Jun 7, 2025
cf9878d
update readme
rick12000 Jun 7, 2025
cb4ff05
update readme
rick12000 Jun 7, 2025
99d2a16
misc
rick12000 Jun 21, 2025
89900a1
improve breach tracking + add aci + add code instructions
rick12000 Jun 21, 2025
5a9d28a
fix breach status
rick12000 Jun 21, 2025
6b9d433
fix configuration handling
rick12000 Jun 28, 2025
03e5338
documentation template
rick12000 Jun 29, 2025
e7f0bb6
added leaf based QRF + revised estimators folder, tests and docs
rick12000 Jun 29, 2025
2a0e762
change default params
rick12000 Jun 30, 2025
3e542ea
set up qleaf
rick12000 Jun 30, 2025
90ec788
setup updates
rick12000 Jul 1, 2025
c4b8632
fix min sampes leaf + conformalization review
rick12000 Jul 4, 2025
73812eb
review estimation module
rick12000 Jul 4, 2025
1faf2b3
update pydantic definitions
rick12000 Jul 4, 2025
d99ab94
review acquisition
rick12000 Jul 4, 2025
f9b3613
review adaptation
rick12000 Jul 5, 2025
3351a1f
sampling review - in progress
rick12000 Jul 6, 2025
b725c5e
sampling review completed
rick12000 Jul 6, 2025
149f43e
refactor config manager
rick12000 Jul 6, 2025
5538cde
review tuning + configuration handling
rick12000 Jul 8, 2025
76eb67a
tuning docs + renaming
rick12000 Jul 8, 2025
0697252
set up cicd and other productionization elements
rick12000 Jul 8, 2025
bdcec9a
update upload-artifact version
rick12000 Jul 8, 2025
d09ac2d
update requirements
rick12000 Jul 8, 2025
6c7a64b
drop support for py 3.8
rick12000 Jul 8, 2025
747147a
fix name
rick12000 Jul 8, 2025
a74503f
fix name
rick12000 Jul 8, 2025
2ef05c1
add architecture section
rick12000 Jul 9, 2025
eb8ff91
add bulk of documentation
rick12000 Jul 12, 2025
097327b
change default pre conformal to 32 + doc string
rick12000 Jul 12, 2025
a8e4e31
fix dtaci
rick12000 Jul 13, 2025
449d0d8
misc
rick12000 Jul 16, 2025
5c99a3a
fix cross validation + add Lasso meta
rick12000 Jul 17, 2025
262c6c7
clean up readme
rick12000 Jul 18, 2025
877ad42
update readme
rick12000 Jul 18, 2025
88628cc
update documentation
rick12000 Jul 18, 2025
0c36483
update docs + logo
rick12000 Jul 19, 2025
9327b7f
update docs + logo
rick12000 Jul 20, 2025
3211fe0
fix latex + add calibration scores
rick12000 Jul 21, 2025
93e4ba4
qgbm param change + monotonicity tests
rick12000 Jul 22, 2025
ecd78d1
remove asymmetric variance in ucb for now
rick12000 Jul 22, 2025
975540a
update params
rick12000 Jul 22, 2025
6c52ff9
fix gp kernels + stricter tests for quantile estimators
rick12000 Jul 23, 2025
bbf7d52
update dtaci gammas
rick12000 Jul 23, 2025
2b27ffe
switch to ordinal split + fix bound extraction order
rick12000 Jul 26, 2025
f4bb3c1
bound none fix
rick12000 Jul 26, 2025
084657e
add trial entry
rick12000 Jul 27, 2025
6564a0e
alignment changes - in progress
rick12000 Jul 29, 2025
c7a7317
update params
rick12000 Jul 30, 2025
eb7517a
update params
rick12000 Jul 30, 2025
25925ad
fix ensemble loss + add new strats
rick12000 Aug 1, 2025
15a6087
update params
rick12000 Aug 2, 2025
b994e01
add estimation tests
rick12000 Aug 6, 2025
86ec50b
fix ensembles
rick12000 Aug 8, 2025
4aa1039
precommit fixes
rick12000 Aug 8, 2025
f3ca80c
enhance conformalization + update tests + remove ES
rick12000 Aug 9, 2025
66a0955
clean up unused methods + remove bayesian tuner
rick12000 Aug 11, 2025
1d57603
misc fixes
rick12000 Aug 12, 2025
72b1d98
fix calibration and normalization
rick12000 Aug 13, 2025
a0f7fcf
fix alpha + add integration test
rick12000 Aug 15, 2025
25f50d8
fix alpha in cqr
rick12000 Aug 15, 2025
15bd731
remove lgbm support
rick12000 Aug 16, 2025
79bb95b
fix tests
rick12000 Aug 16, 2025
03ceebb
added log scale support to intrange
rick12000 Aug 16, 2025
63efb4f
add unit tests
rick12000 Aug 18, 2025
6cc9ad8
fix entropy sampler
rick12000 Aug 18, 2025
0bf5277
fix ensemble from point to pinball stack
rick12000 Aug 19, 2025
5de88a8
switch to sobol + fix GP + change default params for cv and estimators
rick12000 Aug 21, 2025
4c5e640
misc
rick12000 Aug 21, 2025
80f1186
misc
rick12000 Aug 23, 2025
d3102a0
gp improvements
rick12000 Aug 28, 2025
4cc5f50
speed up configuration handling
rick12000 Aug 29, 2025
107e5c6
improve point estimation for ucb and ost + fix ucb interval width
rick12000 Aug 29, 2025
2c49eda
add cursor rule
rick12000 Aug 30, 2025
ff9d4f4
fix mves cython
rick12000 Aug 30, 2025
e4ca38b
fix cython build + revamp CI
rick12000 Aug 31, 2025
e0a1ba5
add linguist flag
rick12000 Aug 31, 2025
9d875cb
temp - check ci prod flow
rick12000 Aug 31, 2025
c8882a4
fix wheels
rick12000 Aug 31, 2025
2df2fb4
wheel fixes
rick12000 Aug 31, 2025
4ac0f2e
wheel fixes
rick12000 Aug 31, 2025
ab531b7
wheel fixes
rick12000 Aug 31, 2025
98a87ef
wheel fixes
rick12000 Aug 31, 2025
c21b47b
enhance pypi tests
rick12000 Aug 31, 2025
8201a0e
ci fix
rick12000 Aug 31, 2025
59fecb8
update cython handling
rick12000 Aug 31, 2025
fb4c739
bump version
rick12000 Aug 31, 2025
512359a
fix mves runtime + cicd fix attempt
rick12000 Aug 31, 2025
b9103ed
build from pyx not c
rick12000 Aug 31, 2025
dee1193
fix unicode issues with emojis
rick12000 Aug 31, 2025
58d782d
update setup
rick12000 Aug 31, 2025
048e5d6
bump version
rick12000 Aug 31, 2025
5ce7019
fix setup
rick12000 Aug 31, 2025
e614731
fix ci
rick12000 Aug 31, 2025
565c73a
fix ci
rick12000 Aug 31, 2025
1143cd7
fix ci
rick12000 Aug 31, 2025
524c4cb
fix name collision
rick12000 Aug 31, 2025
0d48984
fix ci
rick12000 Aug 31, 2025
7e09e7b
switch from cv to cv+ and remove locally weighted methods
rick12000 Sep 3, 2025
03ae568
remove entropy sampler + update docs and cicd
rick12000 Sep 3, 2025
6ef6a88
fix ci
rick12000 Sep 3, 2025
7382a17
fix ci
rick12000 Sep 3, 2025
bcd967a
fix ci
rick12000 Sep 3, 2025
620a9ab
update readme badges
rick12000 Sep 4, 2025
4d4b42a
update readme badges
rick12000 Sep 4, 2025
75b4b1d
update readme
rick12000 Sep 4, 2025
6316315
update readme badges
rick12000 Sep 4, 2025
90fd7ca
improve runtime efficiency of conformalizing steps + misc
rick12000 Sep 6, 2025
fb6b7c4
change knn default
rick12000 Sep 10, 2025
2119cbd
update defaults@
rick12000 Sep 13, 2025
4f0ab2c
remove training frequency param + fix docs
rick12000 Sep 14, 2025
9770321
pre commit fixes
rick12000 Sep 14, 2025
ac8742c
fix docs
rick12000 Sep 14, 2025
96b865f
fix docs
rick12000 Sep 14, 2025
9d651fc
fix docs
rick12000 Sep 14, 2025
263308a
improve docs
rick12000 Sep 14, 2025
3f6e8f9
hide toc tree elements
rick12000 Sep 14, 2025
93b1173
upgrade styling + misc fixes
rick12000 Sep 14, 2025
e090135
fix rst underline
rick12000 Sep 14, 2025
0758d97
fix logo hover
rick12000 Sep 14, 2025
827d5c4
bump version for test
rick12000 Sep 14, 2025
cbc26ce
fix docs
rick12000 Sep 14, 2025
35c78de
change code block color
rick12000 Sep 14, 2025
d6e0932
update docs and doc strings
rick12000 Sep 14, 2025
24e4541
finalize docs and readme
rick12000 Sep 14, 2025
1bea863
add results to readme
rick12000 Sep 21, 2025
08bb510
update ci tests
rick12000 Sep 21, 2025
a4a471a
update docsite + reinstate production cicd
rick12000 Sep 22, 2025
9913af8
change release template
rick12000 Sep 22, 2025
f56b187
update readme and docs with paper
rick12000 Sep 23, 2025
0d78215
undo unintended cicd changes
rick12000 Sep 23, 2025
6e675bc
rename vars, remove docstrings
rick12000 Sep 23, 2025
0111004
improve tuning tests + refactor conformalization module + finalize CI
rick12000 Sep 23, 2025
9b38d5b
update tuning tests + misc
rick12000 Sep 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .cursor/rules/conda-environment.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
description:
globs:
alwaysApply: true
---

To run anything in the terminal or console, ALWAYS:
1. Run: conda activate confopt_env
2. Run: pip install .
3. Run your command
25 changes: 25 additions & 0 deletions .cursor/rules/python-best-practice-instructions.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
description:
globs:
alwaysApply: true
---
# Coding Style Guidelines

- Adopt the DRY principle. If code is repeated in multiple places, it should be functionalized and called in those places.
- Make all inputs explicit. Avoid relying on state or shared context unless encapsulated.
- Avoid implicit behavior (e.g., mutation of input lists, in place dataframe modification).
- Variable names should be descriptive and reduced to the shortest possible length.
- If a function returns multiple types, refactor. Don't return Union[str, dict, None].
- Use pytest.mark.parametrize for testing functions with categorical input values.
- If mocking in unit testing is required, mock external APIs and I/O only.
- No print() statements anywhere in the code. Use logging instead.
- No single-letter variable names unless in mathematical contexts or loops.
- No hard coded values. Use constants or configuration files.
- Don't use early returns in if else statements.
- Don't create classes if functions are sufficient.
- Keep modules small and focused (under 500 lines).
- Comments should explain why, not what. Keep comments under 10% of code.
- Don't write docstrings.
- Don't rely on default values for function arguments.
- Avoid *args or **kwargs unless absolutely necessary.
- Use pydantic models for configuration values.
10 changes: 10 additions & 0 deletions .cursor/rules/software-engineering-best-practices.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
description:
globs:
alwaysApply: true
---
- Always comply with DRY and SOLID principles.
- Use as little code as is necessary to carry out the desired functionality, do not over-engineer or over-validate your code.
- Write easily testable and maintainable code.
- Maximize separation of concerns.
- Consider how your changes will affect the wider codebase, think several dependancies ahead.
8 changes: 8 additions & 0 deletions .cursor/rules/system-permissions.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
description:
globs:
alwaysApply: true
---
- NEVER commit any changes.
- NEVER revert commits or affect the commit history.
- NEVER push commits or pull from remote, or interact at all with remote branches.
31 changes: 31 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Coding Style Guidelines

- Adopt the DRY principle. If code is repeated in multiple places, it should be functionalized and called in those places.
- Make all inputs explicit. Avoid relying on state or shared context unless encapsulated.
- Avoid implicit behavior (e.g., mutation of input lists, in place dataframe modification).
- Variable names should be descriptive and reduced to the shortest possible length.
- If a function returns multiple types, refactor. Don't return Union[str, dict, None].
- Use pytest.mark.parametrize for testing functions with categorical input values.
- If mocking in unit testing is required, mock external APIs and I/O only.
- No print() statements anywhere in the code. Use logging instead.
- No single-letter variable names unless in mathematical contexts or loops.
- No hard coded values. Use constants or configuration files.
- Don't use early returns in if else statements.
- Don't create classes if functions are sufficient.
- Keep modules small and focused (under 500 lines).
- Comments should explain why, not what. Keep comments under 10% of code.
- Don't write docstrings.
- Don't rely on default values for function arguments.
- Avoid *args or **kwargs unless absolutely necessary.
- Use pydantic models for configuration values.
- Always comply with DRY and SOLID principles.
- Use as little code as is necessary to carry out the desired functionality, do not over-engineer or over-validate your code.
- Write easily testable and maintainable code.
- Maximize separation of concerns.
- Consider how your changes will affect the wider codebase, think several dependancies ahead.


To run anything in the terminal or console, ALWAYS:
1. Run: conda activate confopt_env
2. Run: pip install .
3. Run your command
95 changes: 95 additions & 0 deletions .github/documentation-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Quantile Estimation Module Documentation Template

This template provides a step-by-step guide and example prompt for documenting any Python module (e.g., `quantile_estimation.py`) in a style consistent with best practices for technical and developer documentation.

---

## 1. Docstring Requirements

Add or update detailed and informative Google-style docstrings following these guidelines:

### Module-level docstring:
- Brief description of the module's purpose and core functionality
- Key methodological approaches or architectural patterns used
- Integration context within the broader framework
- Focus on salient aspects, avoid trivial descriptions
- Do not add any type hints in the doc strings.

---

## 2. Documentation File Requirements

Create a comprehensive `.rst` documentation file in `docs/developer/components/[module_name].rst` with:

### Structure Example:

```
[Module Name] Module
===================

Overview
--------
[Brief description and key features]

Key Features
------------
[Bullet points of main capabilities]

Architecture
------------
[Class hierarchy, design patterns, architectural decisions]

[Methodology/Algorithm Sections]
-------------------------------
[Detailed explanations of key approaches, mathematical foundations where relevant]

Usage Examples
--------------
[Practical code examples showing common usage patterns]

Performance Considerations
-------------------------
[Computational complexity, scaling considerations, best practices]

Integration Points
-----------------
[How this module connects with other framework components]

Common Pitfalls
---------------
[Common mistakes and how to avoid them]

See Also
--------
[Cross-references to related modules]
```

### Content requirements:
- Technical depth appropriate for developers
- Mathematical foundations with LaTeX equations where relevant (cite relevant papers if mainstream, do not hallucinate, if unsure do not cite any)
- Practical usage examples with actual code
- Performance and scalability guidance
- Integration context within the framework
- Best practices and common pitfalls
- Cross-references to related components

---

## 3. Index Update

Update `docs/developer/components/index.rst` to include the new module documentation in the appropriate section.

---

## 5. Best Practices
- Documentation should be contextually relevant and technically accurate
- Focus on methodology and implementation details that matter to developers
- Provide both theoretical understanding and practical guidance
- Ensure consistency with existing documentation style and organization
- Make it easy for both newcomers and experienced developers to understand and use the module

---

## 6. Example Output (for quantile_estimation.py)

See the current `quantile_estimation.py` for a fully documented example, and `docs/developer/components/quantile_estimation.rst` for a comprehensive documentation file.
22 changes: 22 additions & 0 deletions .github/testing-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Coding Style Guidelines

- Use pytest for all testing, use unittest for mocking.
- Use pytest.mark.parametrize for testing functions with categorical input values:
- For literals, you should automatically cycle through all possible Literal values in your parametrization.
- For ordinal categories or discrete inputs (eg. n_observations, n_recommendations, etc.) pick some sensible ranges (eg. 0 if allowed, or minimum otherwise, then a sensible every day value, say 10, then a very large value, if it's not computationally expensive, say 1000).
- Mock external APIs and I/O only, do not use mocking as a crutch to abstract away components who's behaviour you need to test.
- Use fixtures to store toy data, mocked objects or any other object you plan to reference in the main tests, particularly if it will be used more than once. If the toy data is small and specific to the one test it's called in, it's ok to define it inside the test function.
- Never define nested functions (function def is inside another function) unless explicitly required because of scope (eg. nested generator builders).
- Avoid defining helper functions at the top of a test module, tests should be simple and mostly check existing methods' outputs. Very complex tests may require helper functions, but this should be limited.
- ALL fixtures need to be defined in the tests/conftest.py file, NEVER define them directly in a test module.
- Do not test initialization of classes. Do not use asserts that just check if an attribute of a class exists, or is equal to what you just defined it as, these are bloated tests that accomplish little, but add maintenance cost.
- If you're testing a function or method that returns a shaped object, always check the shape (should it be the same as the input's? Should it be different? Should it be a specific size based on the inputs you passed to the function? etc. based on these questions formulate asserts that check those shape aspects)
- Test the intent behind a function or method, not form or attributes. Read through the function or method carefully, understand its goals and approach, then write meaningful tests that check quality of outputs relative to intent.
- Do not add strings after asserts, eg. do NOT do this:
assert len(final_alphas) == len(initial_alphas), "Alpha count should remain consistent"
after any assert statement, it should just be assert len(final_alphas) == len(initial_alphas)
- Keep comments to a minimum, comments should just explain more obscure asserts or tests.
- Each unit test should be a function, functions should not be grouped in testing classes and should not have self attributes.
- When testing mathematical functions, understand the derivations and test assumptions and outputs given mathematical constraints and theory.
- Do not write excessive amounts of tests, focus on the most important aspects of each function.
- Avoid lenghty code repetition. If multiple tests share the same set ups or fixture processing but only differ in asserts, join them in a single test and add comments before each assert.
Loading
Loading