Skip to content
Merged
Show file tree
Hide file tree
Changes from 100 commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
df859c9
updated xml parser; hefs loading tests WIP
samlamont Apr 15, 2025
5ca1765
update null field checks, revert previous temp changes
samlamont Apr 16, 2025
4315c59
remove previous xml parsing func
samlamont Apr 16, 2025
f69f5bf
remove unneeded funcs
samlamont Apr 16, 2025
8beb68a
include timezone offset
samlamont Apr 16, 2025
599cfc1
parallelize timeseries loading from directory; xml parsing update
samlamont Apr 16, 2025
6fda974
add max_workers argument
samlamont Apr 16, 2025
4c5e11e
climatology WIP
samlamont Apr 17, 2025
52dbb39
Update unique column set fields
samlamont Apr 17, 2025
1a374cb
Update null column checks
samlamont Apr 17, 2025
0c80c86
update test, add member to field check
samlamont Apr 17, 2025
4cbec06
update null column check
samlamont Apr 21, 2025
5880a2d
update max cpu count
samlamont Apr 21, 2025
6190bc8
Adds persist_dataframe argument for loading timeseries
samlamont Apr 22, 2025
119ab33
Calculate reference forecast WIP
samlamont Apr 23, 2025
288e384
Aggregate timeseries in cache according to file size
samlamont Apr 25, 2025
547d9a2
Adding calculate_climatology() and create_refernce_forecast() methods…
samlamont Apr 28, 2025
2388e0d
Add drop_duplicates test
samlamont Apr 28, 2025
ac206d7
change function name
samlamont Apr 30, 2025
f64f1e8
fix function references
samlamont Apr 30, 2025
0e271b6
Update concurrent convert_timeseries logic
samlamont May 6, 2025
af54d27
set default persist_dataframe to false
samlamont May 7, 2025
a7a1eaf
Merge branch '424-parsing-xml-fews-timeseries-is-slow' into 420-add-s…
samlamont May 7, 2025
c3c9c0a
WIP backup
samlamont May 10, 2025
ff33c80
Fix ensemble pivot preprocessor
samlamont May 11, 2025
0c35190
Ref forecast updates
samlamont May 11, 2025
09adf68
fillna WIP
samlamont May 12, 2025
0ace90a
Validate dynamic overwrite
samlamont May 12, 2025
a2b7641
WIP
samlamont May 13, 2025
47c3fd3
adds spark config.
mgdenno May 14, 2025
9fe5a74
notebook update
samlamont May 15, 2025
6240894
fix generate_weights_file() doc string
samlamont May 16, 2025
089c6a1
add drop_duplicates argument; remove null column checks
samlamont May 19, 2025
016b16d
update write mode
samlamont May 19, 2025
19d533e
Merge branch '424-parsing-xml-fews-timeseries-is-slow' into 420-add-s…
samlamont May 19, 2025
0d33190
rolling window WIP
samlamont May 19, 2025
80121b8
validate secondary before writing
samlamont May 22, 2025
bad68c5
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jun 3, 2025
083ff40
wip
samlamont Jun 5, 2025
af5c1e8
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jun 16, 2025
a921573
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jul 1, 2025
8244eca
add ability to filter using timedelta
samlamont Jul 1, 2025
713932c
test update
samlamont Jul 1, 2025
2131670
load timeseries from in-memory dataframe
samlamont Jul 1, 2025
81ce9cd
load_dataframe to base_table
samlamont Jul 2, 2025
d20fcd5
override load_dataframe() for locations table; doc updates
samlamont Jul 7, 2025
4af9bd8
refactoring wip
samlamont Jul 7, 2025
e28bb0e
secondary timeseries table updates
samlamont Jul 7, 2025
dcf0815
more refactoring
samlamont Jul 8, 2025
fe4d5a5
timeseries table updates
samlamont Jul 8, 2025
c9087ed
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jul 10, 2025
9e102aa
set driver memory dynamically
samlamont Jul 10, 2025
26e7eb3
docstring update
samlamont Jul 10, 2025
50bd06d
redundant _load_table() calls
samlamont Jul 10, 2025
d1de2d6
fix climatology bug and hour_of_year for leap years
samlamont Jul 10, 2025
086ff69
fix hour of year
samlamont Jul 10, 2025
f9d1b24
updates
samlamont Jul 11, 2025
d9e5834
Optimize probability transpose func; refactoring
samlamont Jul 14, 2025
a92f5a9
optimize
samlamont Jul 14, 2025
d98267d
logger message
samlamont Jul 14, 2025
1adb6cd
add skill score to metric post-processing
samlamont Jul 15, 2025
fa9fca9
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jul 15, 2025
8911516
update ensemble metric test
samlamont Jul 15, 2025
d686a35
fix bug on _load_dataframe()
samlamont Jul 15, 2025
d5719ca
include reference_time
samlamont Jul 15, 2025
f1b5dbd
fix clone from s3 test
samlamont Jul 15, 2025
9b22821
fix read from s3 tests
samlamont Jul 16, 2025
b1875d4
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Jul 21, 2025
c767b37
update skill score func and metrics post processor
samlamont Jul 23, 2025
32633dc
return dataframe, update test
samlamont Jul 23, 2025
651525e
remove load_dataframe method from base_table
samlamont Aug 4, 2025
7ee27e2
include setuptools for python 3.12 support
samlamont Aug 4, 2025
8785256
post process docstring update
samlamont Aug 4, 2025
736fe20
Added docstrings
samlamont Aug 4, 2025
f59de35
update doc string
samlamont Aug 5, 2025
a07035f
add notebook to playground
samlamont Aug 6, 2025
ded93e5
climatology and ref fcst tests
samlamont Aug 6, 2025
c9ebc97
add default column names
samlamont Aug 6, 2025
159090c
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
mgdenno Aug 11, 2025
3d72c04
fix merge issue
mgdenno Aug 12, 2025
b9b70a2
fix filter formatter and test to work with int leadtimes
mgdenno Aug 12, 2025
8970bc2
reorganize tests
samlamont Aug 14, 2025
8c66eed
add tests/data to sys path for debugger
samlamont Aug 14, 2025
6e5c243
test filtering interval strings
samlamont Aug 14, 2025
1ef41b0
hook up component class
samlamont Aug 14, 2025
864b6ba
timeseries generator WIP
samlamont Aug 15, 2025
4854ec8
timeseries generators wip
samlamont Aug 15, 2025
5cf585c
wip
samlamont Aug 16, 2025
d1441f2
refactor wip
samlamont Aug 18, 2025
2813907
benchmark forecast updates
samlamont Aug 18, 2025
bc1a71f
generator updates wip
samlamont Aug 19, 2025
8ebb58a
update test
samlamont Aug 19, 2025
6b29358
line wrap
samlamont Aug 19, 2025
092ea7f
add hvplot to dev group
samlamont Aug 19, 2025
2d7f788
update example notebook, fill nans
samlamont Aug 19, 2025
de187ab
fix tests; pivot_by_member for single member
samlamont Aug 20, 2025
7a90567
update
samlamont Aug 20, 2025
c1034db
fix test
samlamont Aug 20, 2025
aac77a7
trigger workflow on ready for review
samlamont Aug 20, 2025
5e981b8
drop nan values
samlamont Aug 21, 2025
3630adc
remove unnecessary classes
samlamont Aug 22, 2025
36e62f1
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Aug 22, 2025
dd60afd
uncomment test
samlamont Aug 22, 2025
bd4cafb
add docstrings
samlamont Aug 22, 2025
e1dad27
evaluation table filters
samlamont Aug 23, 2025
d4667e1
check for table_name, line wraps
samlamont Aug 25, 2025
7bbe34c
fix test
samlamont Aug 25, 2025
2383794
fix example notebook
samlamont Aug 25, 2025
a8f7a32
Merge branch 'main' into 420-add-skill-score-for-ensemble-metrics
samlamont Aug 25, 2025
fb280fb
fix test import
samlamont Aug 25, 2025
8ac8607
rename workflows
samlamont Aug 26, 2025
dc92aca
fix add udfs test
samlamont Aug 26, 2025
3468ce1
trigger tests
samlamont Aug 26, 2025
539cf7c
reset workflow trigger
samlamont Aug 26, 2025
3c18c45
update doc strings for date string format
samlamont Aug 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Python package

on:
pull_request:
types: [opened, reopened]
types: [opened, reopened, ready_for_review]
# on:
# push:
# branches: [475-update-python-dependencies]
Expand Down
72 changes: 66 additions & 6 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import nox_poetry


@nox_poetry.session() # for local testing, ex: python=["3.12", "3.13"]
@nox_poetry.session()
def all_tests(session):
"""Run the test suite using pytest."""
session.install("pytest", ".") # for coverage report include: "pytest-cov"
Expand All @@ -14,11 +14,71 @@ def all_tests(session):
)


@nox_poetry.session() # python=["3.12", "3.13"]
def single_test(session):
"""Run a single test using pytest."""
@nox_poetry.session()
def evaluations(session):
"""Run tests related to evaluations."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/test_clone_from_s3.py"
)
"tests/evaluations"
)


@nox_poetry.session()
def fetch(session):
"""Run tests related to fetching."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/fetch"
)


@nox_poetry.session()
def generate(session):
"""Run tests related to generating timeseries."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/generate"
)


@nox_poetry.session()
def load(session):
"""Run tests related to loading."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/load"
)


@nox_poetry.session()
def query(session):
"""Run tests related to querying."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/query"
)


@nox_poetry.session()
def s3_utils(session):
"""Run tests related to S3 utilities."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/s3_utils"
)


@nox_poetry.session()
def visualization(session):
"""Run tests related to visualization."""
session.install("pytest", ".")
session.run(
"pytest",
"tests/visualization"
)
Loading