SFMS Wind Speed & Direction by brettedw · Pull Request #5177 · bcgov/wps

brettedw · 2026-03-02T23:17:44Z

Adds Wind Speed and Wind Direction to SFMS Daily Actuals
Adds COG generation to FWI Interpolations (only done for "Calculations" before. ie. not a re-interpolation Monday in the early fire season)
Includes a refactor of the fwi processor. The reason for this is that wind speed was temporarily faked in the FFMC calculation before. Now that we have actual wind speed, the weather inputs are different for FFMC than they are for DMC/DC (this is made a bit more confusing by the CFFDRS library, which includes things like RH in DC calculations, even though it isn't actually used in the calculation) link
- This made me think about calculating ISI/BUI/FWI which have no weather inputs, only other FWI inputs
- ISI/BUI/FWI aren't being calculated as part of this PR, but I included their "Calculators" along with the refactor
closes SFMS: Wind Interpolation #5104

This reverts commit 38872d3.

codecov · 2026-03-03T01:00:26Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.93%. Comparing base (69eae2f) to head (ab28745).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5177      +/-   ##
==========================================
+ Coverage   68.90%   68.93%   +0.03%     
==========================================
  Files         392      392              
  Lines       16373    16390      +17     
  Branches     1846     1846              
==========================================
+ Hits        11281    11298      +17     
  Misses       4508     4508              
  Partials      584      584

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

conbrad · 2026-03-04T00:32:31Z

backend/packages/wps-sfms/src/wps_sfms/processors/wind.py

+        return direction
+
+    def interpolate(
+        self, source: StationWindVectorSource, reference_raster_path: str


This'll work in Python but technically this source type defines get_uv_interpolation_data and doesn't match the Interpolator contract signature. It'd break if we accidentally passed another source type in here. Wonder if there's a better way to structure the types...

Thanks, I overlooked that. I'll see what I can come up with, without adding too much complexity there

Made some changes in d30d575. Hopefully that doesn't complicate things too much

I think just not forcing the contract would be a bit simpler. With theTypeVar("SourceT") it has no constraints or bound so a type checker won't enforce that subclasses actually use a source type that satisfies any contract.

backend/packages/wps-sfms/src/wps_sfms/processors/idw.py

conbrad · 2026-03-04T00:42:56Z

backend/packages/wps-sfms/src/wps_sfms/interpolation/source.py

+    def get_uv_interpolation_data(
+        self,
+    ) -> Tuple[NDArray[np.float32], NDArray[np.float32], NDArray[np.float32], NDArray[np.float32]]:
+        valid = [


We might also want to include lat and lons in the valid check

@dgboss @conbrad Looking at this brought up a thought. When we save_sfms_run, we're storing all the station codes from the actuals. But in the processing of each parameter, we're filtering these actuals based on "valid" data. I'm thinking in that case, we may still not really know which stations were used for interpolating data. What do you guys think?

In this example - 289 stations saved to db, 288 stations used for temp/rh/precip/wind speed, 287 stations used for wind direction

2026-03-05 11:00:49,783 - __main__ - INFO - Starting SFMS daily actuals for 2025-05-13 2026-03-05 11:00:50,039 - __main__ - INFO - Using reference raster: /vsis3/gpdqha/sfms/static/fuel/2025/fbp2025_v2.tif 2026-03-05 11:00:51,421 - wps_sfms.processors.idw - INFO - Starting interpolation, output: sfms_ng/actual/2025/05/13/temperature_20250513.tif 2026-03-05 11:00:53,153 - wps_sfms.processors.temperature - INFO - Interpolating temperature for raster grid (778 x 683) 2026-03-05 11:00:53,154 - wps_sfms.processors.idw - INFO - Processing 229171 valid pixels (skipping 302203 NoData pixels) 2026-03-05 11:00:53,154 - wps_sfms.processors.idw - INFO - Running batch temperature IDW interpolation for 229171 pixels and 288 stations 2026-03-05 11:00:59,296 - wps_sfms.interpolation.common - INFO - Interpolation complete: 2026-03-05 11:00:59,296 - wps_sfms.interpolation.common - INFO - Total pixels: 531374 2026-03-05 11:00:59,296 - wps_sfms.interpolation.common - INFO - Successfully interpolated: 229171 (43.1%) 2026-03-05 11:00:59,296 - wps_sfms.interpolation.common - INFO - Failed interpolation (no stations in range): 0 (0.0%) 2026-03-05 11:00:59,296 - wps_sfms.interpolation.common - INFO - Skipped (NoData): 302203 (56.9%) 2026-03-05 11:00:59,927 - wps_sfms.processors.idw - INFO - Interpolation complete: sfms_ng/actual/2025/05/13/temperature_20250513.tif 2026-03-05 11:00:59,927 - __main__ - INFO - Temperature interpolation raster: sfms_ng/actual/2025/05/13/temperature_20250513.tif 2026-03-05 11:00:59,927 - wps_shared.db.crud.sfms_run - INFO - temperature_interpolation completed successfully -- time elapsed 0 hours, 0 minutes, 8.00 seconds 2026-03-05 11:01:00,099 - wps_sfms.processors.idw - INFO - Starting interpolation, output: sfms_ng/actual/2025/05/13/relative_humidity_20250513.tif 2026-03-05 11:01:00,603 - wps_sfms.processors.relative_humidity - INFO - Interpolating dew point for RH raster grid (778 x 683) 2026-03-05 11:01:00,603 - wps_sfms.processors.idw - INFO - Processing 229171 valid pixels (skipping 302203 NoData pixels) 2026-03-05 11:01:00,603 - wps_sfms.processors.idw - INFO - Running batch dew point IDW interpolation for 229171 pixels and 288 stations 2026-03-05 11:01:06,639 - wps_sfms.interpolation.common - INFO - Interpolation complete: 2026-03-05 11:01:06,639 - wps_sfms.interpolation.common - INFO - Total pixels: 531374 2026-03-05 11:01:06,639 - wps_sfms.interpolation.common - INFO - Successfully interpolated: 229171 (43.1%) 2026-03-05 11:01:06,639 - wps_sfms.interpolation.common - INFO - Failed interpolation (no stations in range): 0 (0.0%) 2026-03-05 11:01:06,639 - wps_sfms.interpolation.common - INFO - Skipped (NoData): 302203 (56.9%) 2026-03-05 11:01:06,988 - wps_sfms.processors.idw - INFO - Interpolation complete: sfms_ng/actual/2025/05/13/relative_humidity_20250513.tif 2026-03-05 11:01:06,988 - __main__ - INFO - RH interpolation raster: sfms_ng/actual/2025/05/13/relative_humidity_20250513.tif 2026-03-05 11:01:06,988 - wps_shared.db.crud.sfms_run - INFO - rh_interpolation completed successfully -- time elapsed 0 hours, 0 minutes, 6.00 seconds 2026-03-05 11:01:06,998 - wps_sfms.processors.idw - INFO - Starting interpolation, output: sfms_ng/actual/2025/05/13/wind_speed_20250513.tif 2026-03-05 11:01:06,998 - wps_sfms.interpolation.idw - INFO - Starting interpolation for 288 stations 2026-03-05 11:01:07,428 - wps_sfms.interpolation.idw - INFO - Interpolating for raster grid (778 x 683) 2026-03-05 11:01:07,428 - wps_sfms.interpolation.idw - INFO - Processing 229171 valid pixels (skipping 302203 NoData pixels) 2026-03-05 11:01:07,428 - wps_sfms.interpolation.idw - INFO - Running batch IDW interpolation for 229171 pixels and 288 stations 2026-03-05 11:01:13,544 - wps_sfms.interpolation.common - INFO - Interpolation complete: 2026-03-05 11:01:13,544 - wps_sfms.interpolation.common - INFO - Total pixels: 531374 2026-03-05 11:01:13,544 - wps_sfms.interpolation.common - INFO - Successfully interpolated: 229171 (43.1%) 2026-03-05 11:01:13,544 - wps_sfms.interpolation.common - INFO - Failed interpolation (no stations in range): 0 (0.0%) 2026-03-05 11:01:13,544 - wps_sfms.interpolation.common - INFO - Skipped (NoData): 302203 (56.9%) 2026-03-05 11:01:13,849 - wps_sfms.processors.idw - INFO - Interpolation complete: sfms_ng/actual/2025/05/13/wind_speed_20250513.tif 2026-03-05 11:01:13,849 - __main__ - INFO - Wind speed interpolation raster: sfms_ng/actual/2025/05/13/wind_speed_20250513.tif 2026-03-05 11:01:13,849 - wps_shared.db.crud.sfms_run - INFO - wind_speed_interpolation completed successfully -- time elapsed 0 hours, 0 minutes, 6.00 seconds 2026-03-05 11:01:13,852 - wps_sfms.processors.idw - INFO - Starting interpolation, output: sfms_ng/actual/2025/05/13/wind_direction_20250513.tif 2026-03-05 11:01:14,133 - wps_sfms.processors.wind - INFO - Interpolating wind direction for raster grid (778 x 683) 2026-03-05 11:01:14,133 - wps_sfms.processors.idw - INFO - Processing 229171 valid pixels (skipping 302203 NoData pixels) 2026-03-05 11:01:14,133 - wps_sfms.processors.idw - INFO - Running batch wind-u component IDW interpolation for 229171 pixels and 287 stations 2026-03-05 11:01:20,113 - wps_sfms.processors.idw - INFO - Processing 229171 valid pixels (skipping 302203 NoData pixels) 2026-03-05 11:01:20,113 - wps_sfms.processors.idw - INFO - Running batch wind-v component IDW interpolation for 229171 pixels and 287 stations 2026-03-05 11:01:26,205 - wps_sfms.interpolation.common - INFO - Interpolation complete: 2026-03-05 11:01:26,205 - wps_sfms.interpolation.common - INFO - Total pixels: 531374 2026-03-05 11:01:26,205 - wps_sfms.interpolation.common - INFO - Successfully interpolated: 229171 (43.1%) 2026-03-05 11:01:26,205 - wps_sfms.interpolation.common - INFO - Failed interpolation (no stations in range): 0 (0.0%) 2026-03-05 11:01:26,205 - wps_sfms.interpolation.common - INFO - Skipped (NoData): 302203 (56.9%)```

Oh nice catch, I wonder if we should store the stations in object storage and just have a path to that object in each parameter record.

Ya I'll make another ticket to address something like that

backend/packages/wps-sfms/src/wps_sfms/tests/test_wind_interpolation.py

backend/packages/wps-sfms/src/wps_sfms/tests/test_source.py

backend/packages/wps-sfms/src/wps_sfms/tests/test_wind_interpolation.py

backend/packages/wps-sfms/src/wps_sfms/processors/fwi.py

backend/packages/wps-sfms/src/wps_sfms/interpolation/source.py

conbrad · 2026-03-05T21:52:23Z

backend/packages/wps-sfms/src/wps_sfms/processors/fwi.py

+                *(index_keys_by_param[param] for param in index_params),
+            ]
+
+            with input_dataset_context(input_keys) as input_datasets:


Suggested change

with input_dataset_context(input_keys) as input_datasets:

with input_dataset_context(input_keys) as input_datasets:

n_weather = len(weather_params)

weather_datasets: WeatherDatasetMap = dict(zip(weather_params, input_datasets[:n_weather]))

index_datasets: IndexDatasetMap = dict(zip(index_params, input_datasets[n_weather:]))

With this we can remove the cast because input_datasets is already List[WPSDataset] per the MultiDatasetContext type alias. The positional dependency is now explicit — the slice boundary n_weather is the
same value used to build the key list, so they can't drift.

This could also be pulled out into a method or class and return a FWIDatasets if this change is adopted:
https://github.com/bcgov/wps/pull/5177/changes#r2892595196
Should also have it's own unit tests because I think this setup relies on ordering being correct.

What do you think about removing the cast but keeping the iterator? In my mind it's more explicit to get the weather and index params this way than by slicing. Open to thoughts there
ab28745

Sure sounds good, I am still a bit concerned about the assumption that parameters and datasets follow the same ordering though.

backend/packages/wps-sfms/src/wps_sfms/processors/fwi.py

sonarqubecloud · 2026-03-05T23:06:21Z

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

dgboss · 2026-03-05T23:19:00Z

backend/packages/wps-api/src/app/jobs/sfms_daily_actuals.py

            s3_key = await processor.process(s3_client, fuel_raster_path, _source, output_key)
-            logger.info("%s interpolation raster: %s", _job_name.value, s3_key)
+
+            cog_key = raster_addresser.get_cog_key(output_key)


Nice addition!

dgboss · 2026-03-05T23:41:52Z

backend/packages/wps-api/src/app/tests/jobs/test_sfms_daily_actuals.py

            await run_sfms_daily_actuals(target_date)

        mock_dependencies.temp_processor.process.assert_called_once()
+        mock_dependencies.wind_speed_processor.process.assert_called_once()


Should the wind_direction_processor also have been called?

brettedw added 6 commits March 2, 2026 15:15

init

3fa0540

diif & histo title

8064bcc

use threshold for wind

5c7ff5a

wind speed interpolator

759692e

no sonar?

38872d3

Revert "no sonar?"

297919a

This reverts commit 38872d3.

brettedw added 7 commits March 3, 2026 08:40

plotter

03c51d2

docstring

c7e6847

tests

510de21

test comment

6317d53

fwi refactor for wind speed + ISI/BUI/FWI

0929811

COG for interpolation

39367df

comments

9e71fd7

conbrad reviewed Mar 4, 2026

View reviewed changes

brettedw added 5 commits March 4, 2026 15:11

roundtrip test

8b5b0f3

add wind uv tests

ac2c4ef

test u/v fail

b9f5584

BaseInterpolator

d30d575

sfms logging

b24a612

brettedw requested review from conbrad and dgboss March 5, 2026 21:08

conbrad reviewed Mar 5, 2026

View reviewed changes

brettedw added 3 commits March 5, 2026 14:14

validity lat/long alignment

60e7faf

FWIDatasets

b1b46c7

no cast

ab28745

brettedw requested a review from conbrad March 5, 2026 23:08

dgboss reviewed Mar 5, 2026

View reviewed changes

-            with input_dataset_context(input_keys) as input_datasets:
+            with input_dataset_context(input_keys) as input_datasets:
+                n_weather = len(weather_params)
+                weather_datasets: WeatherDatasetMap = dict(zip(weather_params, input_datasets[:n_weather]))
+                index_datasets: IndexDatasetMap = dict(zip(index_params, input_datasets[n_weather:]))

Conversation

brettedw commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Links:

Uh oh!

codecov bot commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sonarqubecloud bot commented Mar 5, 2026

Quality Gate failed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

brettedw commented Mar 2, 2026 •

edited

Loading

codecov bot commented Mar 3, 2026 •

edited

Loading