Add spatial hashing #1169

fluidnumerics-joe · 2025-02-24T16:50:21Z

Overview

This PR adds the SpatialHash class in the neighbors.py module. This class is used to encapsulate the necessary information and methods for conducting unstructured grid searches using spatial hashes. As part of the initialization of the SpatialHash object, an associated unstructured grid is used to construct a uniformly spaced structured grid, called the "hash grid", and relate cell indices in the hash grid to elements in the unstructured grid. This hash table is a critical element of the spatial hash grid search as it is used to create a short-list of elements to search within.

The SpatialHash.query method returns the element id that a point (or element ids that a list of points) reside within alonside the barycentric coordinates. I've included helper methods to compute the barycentric coordinates of a point in relation to a given unstructured grid face (it works for convex faces n>=3 vertices). The barycentric coordinates are used to determine if a point is inside or outside a face; if all the barycentric coordinates are between [0,1], then a point is inside a face.

Currently, only spherical coordinates (in radians or degrees) are supported for this search.

Expected Usage

import uxarray as ux
grid_path = "../../test/meshfiles/fesom/soufflet-netcdf/grid.nc"
uxgrid = ux.open_grid(grid_path)

uxds = ux.open_dataset(grid_path, data_path)

# Construct the `SpatialHash` object
spatial_hash = uxgrid.get_spatial_hash(
    reconstruct="False",
)

# Query the `SpatialHash` for the face id and the barycentric coordinates.
face_ids, bcoords = spatial_hash.query([0.9, 1.8])

PR Checklist

General

An issue is linked created and linked
Add appropriate labels
Filled out Overview and Expected Usage (if applicable) sections

Testing

Adequate tests are created if there is new functionality
Tests cover all possible logical paths in your function
Tests are not too basic (such as simply calling a function and nothing else)

Documentation

Docstrings have been added to all new functions
Docstrings have updated with any function changes
Internal functions have a preceding underscore (_) and have been added to docs/internal_api/index.rst
User functions have been added to docs/user_api/index.rst

Examples

Any new notebook examples added to docs/examples/ folder
Clear the output of all cells before committing
New notebook files added to docs/examples.rst toctree
New notebook files added to new entry in docs/gallery.yml with appropriate thumbnail photo in docs/_static/thumbnails/

I attempted to stay close to the KD-Tree and BallTree API. However, the purpose of the SpatialHash class is to help locate the faces that a list of coordinates lie within. Because of this, the intent of the `query` function for the SpatialHash is a bit different than that for the KD and Ball trees. Additional support routines are provided for assessing whether a point is inside or outside a polygon. This is done by calculating the barycentric coordinates of a point in a convex polygon. The method is defined so that the sum of the barycentric coordinates is exactly one. Because of this, if any of the barycentric coordinates are negative, we immediately know that a point is outside the polygon. All coordinate calculations are done using (lon,lat) in radians for the SpatialHash class. Cartesian coordinates are not supported in this commit.

review-notebook-app · 2025-02-24T16:50:28Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

philipc2 · 2025-02-26T17:29:16Z

Hi @fluidnumerics-joe

This looks great so far! May you add a test_spatial_hasing.py module with a few tests and run pre-commit? Let me know if you have any questions.

philipc2

Can you add the spatial-hashing notebook here? (both in the outline and the toc tree at the bottom)

I'm thinking between "Subsetting" and "Cross-Sections?" would be a good spot.

https://github.com/UXARRAY/uxarray/blob/main/docs/userguide.rst?plain=1

fluidnumerics-joe · 2025-02-26T21:32:49Z

Hi @fluidnumerics-joe

This looks great so far! May you add a test_spatial_hasing.py module with a few tests and run pre-commit? Let me know if you have any questions.

Yes, I'll take care of both of these.

* Ensure that the lon bounds are sorted from low to high. This is needed to make sure that the index looping for filling the hashtable actually executes. * All barycentric coordinates are calculated directly. Rather than calculating the last coordinate to ensure they sum to one, we now compute all coordinates. The coordinates are calculated now to align with the indexing of the node ID's so that np.sum(bcoords*nodes) returns the barycentric interpolation estimate of point. If the point is within the element, then the barycentric interpolation estimate matches the input coordinate (to machine precision). I've added this as a check.

fluidnumerics-joe · 2025-02-28T13:56:27Z

@philipc2 - I've added tests for the spatial hashing, fixed formatting, and added spatial hashing to the toc in the userguide. Let me know if there's anything else you'd like to see updated

philipc2

Looks great! A few small comments attached.

I'm going to play around with this today.

Thanks for putting this together so quickly!

uxarray/grid/grid.py

uxarray/grid/neighbors.py

uxarray/grid/grid.py

Co-authored-by: Philip Chmielowiec <[email protected]>

philipc2

Some leftover print statements.

uxarray/grid/neighbors.py

test/test_spatial_hashing.py

uxarray/grid/neighbors.py

philipc2 · 2025-03-03T17:06:13Z

I'm having some issues with the soufflet-netcdf/grid.nc grid (seems to be crashing the docs too).

It stalls for multiple minutes when trying to construct the tree. This was one of the grids that gave me some trouble in #1013

philipc2

Looks like one of the Github suggestions broke the pre-commit. Can you run it again?

Also, please restart and clear the outputs of the notebook, they will be rendered via our CI. Other than that all looks good.

fluidnumerics-joe · 2025-03-04T21:33:46Z

Looks like one of the Github suggestions broke the pre-commit. Can you run it again?

Also, please restart and clear the outputs of the notebook, they will be rendered via our CI. Other than that all looks good.

I re-ran pre-commit run --all-files and cleared python notebook output.

philipc2 · 2025-03-04T21:35:56Z

Looks like one of the Github suggestions broke the pre-commit. Can you run it again?
Also, please restart and clear the outputs of the notebook, they will be rendered via our CI. Other than that all looks good.

I re-ran pre-commit run --all-files and cleared python notebook output.

Oops, look's like its complaining about a merge conflict.

uxarray/grid/neighbors.py

Instead, we're favoring a check of the predicted coordinates using barycentric interpolation

uxarray/grid/neighbors.py

Co-authored-by: Philip Chmielowiec <[email protected]>

philipc2

Thanks for this contribution! Other than the comments I left earlier, it looks good to me.

aaronzedwick

This looks great! I will make use of the barycentric weights in one of my current PR's, so that is really helpful to me. Thanks for your contribution!

@Philip2c

The denominator in the barycentric weights calculation (Weischell weights) is clipped to be no smaller than `ERROR_TOLERANCE` from the uxarray.constants module. This change is made to fix an issue, reported anectdotally by @Philip2c in UXARRAY#1169

When the denominator is clipped, rather than the individual triangle areas, the Weischell weights are incorrect for the case when a query point is on a face vertex. I've added tests for query points on edges and vertices

philipc2 · 2025-03-10T19:53:14Z

@fluidnumerics-joe

Turns out the issue with the 30km MPAS grid was the edge_node_distances were represented in meters, instead of radians, which is inconsistent with our Grid.edge_node_distances definition. The same behavior can be see in the meshfiles\mpas\QU\oQU480.231010.nc grid, but the result is still correct. This was causing the hash cell size to be much larger than exepcted.

It also appears we have a typo in our Grid.edge_node_distances definition. The unit should be radians instead of degrees here

uxarray/uxarray/grid/grid.py

Lines 1386 to 1394 in d9f091f

    
               @property 
        
               def edge_node_distances(self): 
        
                   """Distances between the two nodes that surround each edge in degrees. 
        
                   Dimensions ``(n_edge, )`` 
        
                   """ 
        
                   if "edge_node_distances" not in self._ds: 
        
                       _populate_edge_node_distances(self) 
        
                   return self._ds["edge_node_distances"]

I'll raise an issue and get this fixed. Glad to see that the issue wasn't anything with your implementation.

fluidnumerics-joe · 2025-03-10T23:11:44Z

🥳 - thanks @philipc2 and @aaronzedwick for your feedback. Without it, I would have not seen some of the cases that we ended up building tests for. This is great to see pushed across the finish line.

Feel free to reach out on any future issues related to the spatial hashing. I'm happy to chip in.

rajeeja · 2025-03-11T00:01:27Z

@fluidnumerics-joe

Thanks for this work and hoping to see few more features in future :)

fluidnumerics-joe and others added 4 commits February 18, 2025 16:38

Add get_spatial_hash to grid api

3093d48

Add example usage in spatial hashing docs

46dcdc6

Merge branch 'UXARRAY:main' into main

77c47c6

philipc2 assigned fluidnumerics-joe Feb 24, 2025

philipc2 added the new feature New feature or request label Feb 24, 2025

philipc2 reviewed Feb 26, 2025

View reviewed changes

fluidnumerics-joe added 2 commits February 26, 2025 16:31

Add spatialhashing to userguide

63048b5

Merge branch 'main' of github.com:FluidNumerics/uxarray

3831793

fluidnumerics-joe and others added 8 commits February 26, 2025 17:02

Add basic tests

b79bc8b

Fix call to get_spatialhash

86d2d5f

Merge branch 'main' into main

b7f3ca3

Fix formatting

8d6e47e

Merge branch 'main' of github.com:FluidNumerics/uxarray

72c1ed6

Prepend attributes intended for internal use with _

5a59918

Fix docstrings for tests; add fesom test with particle list

95cba52

philipc2 requested changes Feb 28, 2025

View reviewed changes

uxarray/grid/grid.py Outdated Show resolved Hide resolved

uxarray/grid/neighbors.py Outdated Show resolved Hide resolved

fluidnumerics-joe added 2 commits March 3, 2025 08:56

Add docstring for get_spatialhash

7afca52

remove empty notes

d599276

philipc2 requested changes Mar 3, 2025

View reviewed changes

uxarray/grid/grid.py Outdated Show resolved Hide resolved

uxarray/grid/grid.py Outdated Show resolved Hide resolved

fluidnumerics-joe and others added 3 commits March 3, 2025 09:16

Update uxarray/grid/grid.py

52f1728

Co-authored-by: Philip Chmielowiec <[email protected]>

Fix naming in test

dfa4f77

Merge branch 'main' of github.com:FluidNumerics/uxarray

4eac186

philipc2 requested changes Mar 3, 2025

View reviewed changes

uxarray/grid/neighbors.py Outdated Show resolved Hide resolved

uxarray/grid/neighbors.py Outdated Show resolved Hide resolved

test/test_spatial_hashing.py Outdated Show resolved Hide resolved

uxarray/grid/neighbors.py Outdated Show resolved Hide resolved

fluidnumerics-joe added 5 commits March 4, 2025 14:57

Retain output in notebook. Make requested formatting changes

fc59591

Merge branch 'main' of github.com:FluidNumerics/uxarray

7ec453d

Fix formatting

8db36e6

Change to smaller/simpler 4xhex grid

fb4bbf8

Add get_spatial_hash to the Grid/Methods section

b655a46

philipc2 requested changes Mar 4, 2025

View reviewed changes

Clear outputs from notebook.

37034ae

aaronzedwick reviewed Mar 4, 2025

View reviewed changes

uxarray/grid/neighbors.py Show resolved Hide resolved

fluidnumerics-joe added 2 commits March 4, 2025 16:36

Merge remote-tracking branch 'upstream/main'

63e8b29

Fix docstring; we're not computing signed area any longer.

e7ab031

Instead, we're favoring a check of the predicted coordinates using barycentric interpolation

philipc2 requested changes Mar 5, 2025

View reviewed changes

uxarray/grid/neighbors.py Show resolved Hide resolved

uxarray/grid/neighbors.py Show resolved Hide resolved

uxarray/grid/neighbors.py Show resolved Hide resolved

uxarray/grid/neighbors.py Show resolved Hide resolved

fluidnumerics-joe and others added 4 commits March 7, 2025 11:49

Update uxarray/grid/neighbors.py

b318f40

Co-authored-by: Philip Chmielowiec <[email protected]>

Update uxarray/grid/neighbors.py

305b8e3

Co-authored-by: Philip Chmielowiec <[email protected]>

Add query optimization suggested by @philipc2

5fcd039

Remove extra return statement

61fcc1a

philipc2 approved these changes Mar 7, 2025

View reviewed changes

aaronzedwick self-requested a review March 10, 2025 16:35

aaronzedwick approved these changes Mar 10, 2025

View reviewed changes

philipc2 mentioned this pull request Mar 10, 2025

[Release]: v2024.03.0 #1171

Closed

6 tasks

fluidnumerics-joe and others added 3 commits March 10, 2025 14:02

Clip triangle areas rather than denominator

b3c9a07

When the denominator is clipped, rather than the individual triangle areas, the Weischell weights are incorrect for the case when a query point is on a face vertex. I've added tests for query points on edges and vertices

Merge branch 'main' into main

5acf05e

philipc2 mentioned this pull request Mar 10, 2025

MPAS Edge & Face Distances Parsed in Meters #1178

Closed

Merge branch 'main' into main

56b0c85

philipc2 merged commit 2154ea8 into UXARRAY:main Mar 10, 2025
20 checks passed

Add spatial hashing #1169

Add spatial hashing #1169

Uh oh!

Conversation

fluidnumerics-joe commented Feb 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Expected Usage

PR Checklist

Uh oh!

review-notebook-app bot commented Feb 24, 2025

Uh oh!

philipc2 commented Feb 26, 2025

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

fluidnumerics-joe commented Feb 26, 2025

Uh oh!

fluidnumerics-joe commented Feb 28, 2025

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

philipc2 commented Mar 3, 2025

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

fluidnumerics-joe commented Mar 4, 2025

Uh oh!

philipc2 commented Mar 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

aaronzedwick left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

philipc2 commented Mar 10, 2025

Uh oh!

Uh oh!

fluidnumerics-joe commented Mar 10, 2025

Uh oh!

rajeeja commented Mar 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fluidnumerics-joe commented Feb 24, 2025 •

edited

Loading

aaronzedwick left a comment •

edited

Loading