diff --git a/src/Plugins/SimplnxCore/docs/ComputeGroupingDensityFilter.md b/src/Plugins/SimplnxCore/docs/ComputeGroupingDensityFilter.md index c8788beac8..4a1d1bc84b 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeGroupingDensityFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeGroupingDensityFilter.md @@ -23,18 +23,27 @@ For each **Parent Feature**, the filter: Grouping Density = Parent Volume / Total Checked Volume ``` -If a **Parent Feature** has no child **Features** (total checked volume is zero), the **Grouping Density** is set to **-1.0** to indicate an invalid or empty parent. +If a **Parent Feature** has no child **Features** (total checked volume is zero), the **Grouping Density** is set to the sentinel value *-1.0* to indicate an invalid or empty parent. + +The **Feature Volumes** and **Parent Volumes** are typically expressed as a count of cells (the integer voxel count per feature, the default output of [Compute Feature Sizes](ComputeFeatureSizesFilter.md)). Equivalent-diameter or physical-unit volumes also work — the only requirement is that both *Feature Volumes* and *Parent Volumes* are in the **same** unit, since the **Grouping Density** is a dimensionless ratio of the two. + + +### Algorithm Description + +![ComputeGroupingDensity Algorithm Description](Images/ComputeGroupingDensity_Infographic.png) ### Optional Parameters #### Use Non-Contiguous Neighbors -When enabled, the filter also queries the **Non-Contiguous Neighbor List** for each child **Feature** in addition to the standard **Contiguous Neighbor List**. This expands the set of checked **Features** to include neighbors that are nearby but do not share a direct face/edge/vertex with the child **Feature**. Enable this option if non-contiguous neighbors were used during the original grouping step. Typically the filter "Compute Feature NeighborHoods" is used to generate the Non-contiguous Neighbors lists. That filter's parameter for the "Multiples of Average Diameter can have a large effect on the final Grouping Density value that is computed. +When enabled, the filter also queries the **Non-Contiguous Neighbor List** for each child **Feature** in addition to the standard **Contiguous Neighbor List**. This expands the set of checked **Features** to include neighbors that are nearby but do not share a direct face/edge/vertex with the child **Feature**. Enable this option if non-contiguous neighbors were used during the original grouping step. Typically the [Compute Feature Neighborhoods](ComputeNeighborhoodsFilter.md) filter is used to generate the **Non-Contiguous Neighbor List**. The *Multiples of Average Diameter* parameter on that filter has a large effect on the final **Grouping Density** value that is computed. #### Find Checked Features When enabled, the filter produces an additional output array (**Checked Features**) at the **Feature** level. For each **Feature** that was checked during the density computation, this array records which **Parent Feature** checked it. Since a **Feature** may be checked by multiple **Parent Features** (as a neighbor of children belonging to different parents), the assignment goes to the **Parent Feature** with the **largest Parent Volume**. This provides a way to see which parent had the strongest influence over each region of the microstructure. +**Tie-break behavior:** When two or more **Parent Features** have *exactly equal* **Parent Volumes** and both check the same **Feature**, the first-processed parent (the one with the lower **Parent ID**) keeps the assignment. The strictly-greater comparison (`>`) means equal-volume parents do not overwrite earlier claims. This is deterministic and matches the legacy DREAM3D `FindGroupingDensity` behavior. + ### Worked Example @@ -47,13 +56,13 @@ Consider a 20x5 2D **Image Geometry** with unit spacing (1.0 x 1.0 x 1.0), conta - Features 1, 2, 3 belong to Parent 1 (Parent Volume = 45, i.e. 10 + 20 + 15 cells) - Features 4, 5 belong to Parent 2 (Parent Volume = 55, i.e. 25 + 30 cells) -| Feature | Cells | Volume | Parent | Contiguous Neighbors | -|---------|-------|--------|--------|----------------------| -| 1 | 10 | 10.0 | 1 | {2} | -| 2 | 20 | 20.0 | 1 | {1, 3} | -| 3 | 15 | 15.0 | 1 | {2, 4} | -| 4 | 25 | 25.0 | 2 | {3, 5} | -| 5 | 30 | 30.0 | 2 | {4} | +| Feature | Cells | Volume (cells) | Parent | Contiguous Neighbors | +|---------|-------|----------------|--------|----------------------| +| 1 | 10 | 10.0 | 1 | {2} | +| 2 | 20 | 20.0 | 1 | {1, 3} | +| 3 | 15 | 15.0 | 1 | {2, 4} | +| 4 | 25 | 25.0 | 2 | {3, 5} | +| 5 | 30 | 30.0 | 2 | {4} | **Parent 1:** Child features {1, 2, 3} plus their contiguous neighbors include feature 4 (neighbor of feature 3). Total checked volume = 10 + 20 + 15 + 25 = 70. Density = 45 / 70 = **0.6429**. @@ -68,14 +77,20 @@ Note that both densities are less than 1.0 because each parent's children have n | > 1.0 | Parent volume exceeds the total checked region; the grouping is compact and dense | | = 1.0 | Parent volume equals the total checked region | | 0.0 < d < 1.0 | Parent volume is smaller than the total checked region; the grouping is dispersed | -| -1.0 | No child features found for this parent (invalid/empty parent) | +| *-1.0* | No child features found for this parent (invalid/empty parent) | -% Auto generated parameter table will be inserted here +### Required Input Sources +| Parameter | Source | +|---------------|----------------| +| Feature Volumes | produced by [Compute Feature Sizes](ComputeFeatureSizesFilter.md). Cell-count volumes (integer count of voxels per feature) are the typical usage. Equivalent-diameter or physical-unit volumes also work as long as both *Feature Volumes* and *Parent Volumes* use the **same** unit. | +| Parent Volumes | produced by running [Compute Feature Sizes](ComputeFeatureSizesFilter.md) on the parent **Attribute Matrix** (after the upstream parent-grouping step has assigned each cell to a **Parent Feature**). | +| Feature Parent Ids | produced by a parent-grouping filter such as [Group MicroTexture Regions](../SimplnxReview/GroupMicroTextureRegionsFilter.md) or [Merge Colonies](../SimplnxReview/MergeColoniesFilter.md), which assign each child **Feature** to a **Parent Feature**.| +| Contiguous Neighbor List | produced by [Compute Feature Neighbors](ComputeFeatureNeighborsFilter.md).| +| Non-Contiguous Neighbor List | (optional, only when "Use Non-Contiguous Neighbors" is enabled)* -- produced by [Compute Feature Neighborhoods](ComputeNeighborhoodsFilter.md). See the "Use Non-Contiguous Neighbors" parameter guidance above for notes on how the upstream *Multiples of Average Diameter* parameter affects results.| -### Algorithm Flowchart -![ComputeGroupingDensity Algorithm Flowchart](Images/ComputeGroupingDensity_Algorithm.png) +% Auto generated parameter table will be inserted here ## References diff --git a/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.png b/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.png new file mode 100644 index 0000000000..b26c77dc42 Binary files /dev/null and b/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.png differ diff --git a/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.svg b/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.svg new file mode 100644 index 0000000000..ff1b61194e --- /dev/null +++ b/src/Plugins/SimplnxCore/docs/Images/ComputeGroupingDensity_Infographic.svg @@ -0,0 +1,610 @@ + + +Compute Grouping Densities — How the Algorithm Works +20×5 toy dataset: 5 features grouped into 2 parents +1. Input data +Parent 1 (volume = 45) +Parent 2 (volume = 55) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +1 +2 +3 +4 +5 +vol = 10 +vol = 20 +vol = 15 +vol = 25 +vol = 30 +Each row above is a Feature; the number inside is the FeatureId. +2. Contiguous neighbors only (UseNonContiguousNeighbors = false) +Each parent's child features plus their contiguous-list neighbors form the 'touched set'. +Parent 1's touched set + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +1 +2 +3 +4 +5 +Parent 2's touched set + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +1 +2 +3 +4 +5 +touched = {1,2,3,4}, Σ vol = 70 +density = 45 / 70 = 0.6429 +touched = {3,4,5}, Σ vol = 70 +density = 55 / 70 = 0.7857 +3. With non-contiguous neighbors (UseNonContiguousNeighbors = true) +Non-contiguous links (dashed arcs) reach farther neighbors. The touched sets grow; densities shrink. +Parent 1's touched set + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +1 +2 +3 +4 +5 + +non-contig 1↔4 + +non-contig 2↔5 +Parent 2's touched set + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +1 +2 +3 +4 +5 + +non-contig 1↔4 + +non-contig 2↔5 +touched = {1,2,3,4,5}, Σ vol = 100 +density = 45 / 100 = 0.45 +touched = {1,2,3,4,5}, Σ vol = 100 +density = 55 / 100 = 0.55 + \ No newline at end of file diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.cpp index 3082220c8c..ea0c5376b9 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.cpp @@ -10,6 +10,10 @@ using namespace nx::core; namespace { +// Compile-time policy struct that selects between the 4 algorithm variants +// (UseNonContiguousNeighbors x FindCheckedFeatures). Resolving these flags +// at compile time via template specialization keeps the inner-loop hot path +// free of runtime branches on the flag values. template struct FindDensitySpecializations { @@ -17,6 +21,14 @@ struct FindDensitySpecializations static constexpr bool FindingCheckedFeatures = FindCheckedFeatures; }; +// Core grouping-density computation. For each parent, walk its assigned +// features and their neighbors (contiguous always; non-contiguous when the +// template flag is set), accumulating totalFeatureCheckVolume, then write +// GroupingDensities[parent] = parentVolume / totalFeatureCheckVolume, or +// the sentinel -1.0f if no features touched the parent. When +// FindingCheckedFeatures is set, also write the largest-volume claiming +// parent into CheckedFeatures[feature] (ties go to first-encountered parent +// because the comparison uses strict `>`). template > class FindDensityGrouping { @@ -36,10 +48,10 @@ class FindDensityGrouping } ~FindDensityGrouping() noexcept = default; - FindDensityGrouping(const FindDensityGrouping&) = delete; // Copy Constructor Default Implemented - FindDensityGrouping(FindDensityGrouping&&) = delete; // Move Constructor Not Implemented - FindDensityGrouping& operator=(const FindDensityGrouping&) = delete; // Copy Assignment Not Implemented - FindDensityGrouping& operator=(FindDensityGrouping&&) = delete; // Move Assignment Not Implemented + FindDensityGrouping(const FindDensityGrouping&) = delete; + FindDensityGrouping(FindDensityGrouping&&) = delete; + FindDensityGrouping& operator=(const FindDensityGrouping&) = delete; + FindDensityGrouping& operator=(FindDensityGrouping&&) = delete; Result<> operator()() { @@ -111,6 +123,10 @@ class FindDensityGrouping curParentVolume = parentVolumesRef[currentParentId]; if(totalFeatureCheckVolume == 0.0f) { + // Sentinel: this parent had no assigned features (so no neighbors + // were walked, and totalFeatureCheckVolume stayed at 0). Downstream + // consumers treat -1.0f in GroupingDensities as "density is not + // defined for this parent." See the filter documentation. outGroupingDensitiesRef[currentParentId] = -1.0f; } else @@ -128,14 +144,17 @@ class FindDensityGrouping float32& totalFeatureCheckVolume, const AbstractDataStore& parentVolumesRef, std::vector& checkedFeatureVolumes, AbstractDataStore& outCheckedFeaturesRef) { - auto featureNeighbors = neighborList.at(currentFeatureId); - auto numNeighbors = static_cast(featureNeighbors.size()); - + const usize numNeighbors = neighborList.getListSize(currentFeatureId); for(int32 neighborIdx = 0; neighborIdx < numNeighbors; neighborIdx++) { - auto neighborId = featureNeighbors.at(neighborIdx); + bool ok = false; + int32 neighborId = neighborList.getValue(currentFeatureId, neighborIdx, ok); + if(!ok) // If trying to retrieve the value fails for some reason. This should never happen. + { + return; + } - // If the current neighbor is NOT in the check list... + // If the current neighbor is NOT in the checklist... if(!totalFeatureCheckList.contains(neighborId)) { // update the volumes and the check list diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ComputeGroupingDensityFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ComputeGroupingDensityFilter.cpp index c346e53ebc..c560fefbd5 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ComputeGroupingDensityFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ComputeGroupingDensityFilter.cpp @@ -17,8 +17,16 @@ using namespace nx::core; namespace { -const DataPath k_ThrowawayCheckedFeatures = DataPath({"HiddenTempCheckedFeatures"}); -const DataPath k_ThrowawayNonContiguous = DataPath({"HiddenNonContiguousNL"}); +// Throwaway DataPaths used when the user opts out of either output. The +// underlying algorithm always expects a valid Int32Array for CheckedFeatures +// and a valid Int32NeighborList for NonContiguousNeighbors, even when those +// outputs are unused. preflightImpl() creates these temporary objects when +// the user opts out and schedules a deferred delete; the algorithm writes +// to them (the writes are harmless) and they are cleaned up at the end of +// execute. This keeps the algorithm interface free of std::optional or +// nullable references. +const auto k_ThrowawayCheckedFeatures = DataPath({"HiddenTempCheckedFeatures"}); +const auto k_ThrowawayNonContiguous = DataPath({"HiddenNonContiguousNL"}); } // namespace namespace nx::core @@ -155,13 +163,14 @@ IFilter::PreflightResult ComputeGroupingDensityFilter::preflightImpl(const DataS return MakePreflightErrorResult(-15673, fmt::format("Feature Volumes [{}] must be stored in an Attribute Matrix.", pFeatureVolumesPath.toString())); } + // CheckedFeatures output: create the real output when requested; otherwise + // create the throwaway placeholder (see the k_ThrowawayCheckedFeatures + // comment block at the top of this file) and schedule its deletion. if(pFindCheckedFeatures) { - { - DataPath checkedFeaturesPath = pFeatureVolumesPath.replaceName(pCheckedFeaturesName); - auto createArrayAction = std::make_unique(nx::core::DataType::int32, pFeatureAM->getShape(), ShapeType{1}, checkedFeaturesPath); - resultOutputActions.value().appendAction(std::move(createArrayAction)); - } + DataPath checkedFeaturesPath = pFeatureVolumesPath.replaceName(pCheckedFeaturesName); + auto createArrayAction = std::make_unique(nx::core::DataType::int32, pFeatureAM->getShape(), ShapeType{1}, checkedFeaturesPath); + resultOutputActions.value().appendAction(std::move(createArrayAction)); } else { @@ -175,6 +184,9 @@ IFilter::PreflightResult ComputeGroupingDensityFilter::preflightImpl(const DataS } } + // Non-contiguous neighbor list: when the user has opted out, create a + // throwaway 1-tuple neighbor list and schedule its deletion. See the + // k_ThrowawayNonContiguous comment block at the top of this file. if(!pUseNonContiguousNeighbors) { { diff --git a/src/Plugins/SimplnxCore/test/CMakeLists.txt b/src/Plugins/SimplnxCore/test/CMakeLists.txt index d9097e61c2..a33e7dc8fb 100644 --- a/src/Plugins/SimplnxCore/test/CMakeLists.txt +++ b/src/Plugins/SimplnxCore/test/CMakeLists.txt @@ -288,7 +288,7 @@ if(EXISTS "${DREAM3D_DATA_DIR}" AND SIMPLNX_DOWNLOAD_TEST_FILES) download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME winding_surface_nets.tar.gz SHA512 b45567fd89ea8ebac4764b37491041afa04516fee4b11b5c22d9d3a03c988e335f97395539438d008a7d0f006375a6ec0c62df2b3929ac59671f2e066bc2123f) download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME write_lammps_test.tar.gz SHA512 82bb5360b76e857f3233d37733c602f67fd2ac667e49b24741a70ab649e8046fb7905493df37d142808b740c2771fe7cdccd71c9d70679afafe398529ee5771e) download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME write_stl_overflow_test.tar.gz SHA512 d8f8eac901479100ffb5813b3ac72c86da496d3620d406f8691adc0d95eb4670bf4e887f57ccc702bac7e6eeaffdf61db9b4c06157423845dd198722870b0c0b) - download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME compute_grouping_densities.tar.gz SHA512 96066196d6aa5f87cc7b717f959848c2f3025b7129589abe1eded2a8d725c539a89b0a6290a388a56b5a401e0bd3041698fbd8e8cf37a1f18fdd937debd21531) + download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME compute_grouping_densities_v2.tar.gz SHA512 3aaabb63c4fa16f7fa192ae4ee9dbba9394ec7f1cd19aff55e399a624d495a3a778c7f6f282911f681e85cea99e4c6d15344274e9107f337af7d4a19f93784ff) download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME hierarchical_smoothing.tar.gz SHA512 47217ee420d9438c3d36a195c06ae060917f5fb7ee295feffdabf05741bec87bf29c3b44016b744930cda7383cd05e0d58df7e7776a7732dc46c12b780e51398) download_test_data(DREAM3D_DATA_DIR ${DREAM3D_DATA_DIR} ARCHIVE_NAME identify_sample_v2.tar.gz SHA512 a7ffac3eaad479c07215c1dd16274c45a52466708a9d27b5f85a29b0eba3b6705b627e1052a7a27e9bfe89cd6e7df673beb7a1e98b262b6c52ea383b4848ac31) diff --git a/src/Plugins/SimplnxCore/test/ComputeGroupingDensityTest.cpp b/src/Plugins/SimplnxCore/test/ComputeGroupingDensityTest.cpp index d8f7bf4165..ed7646db5e 100644 --- a/src/Plugins/SimplnxCore/test/ComputeGroupingDensityTest.cpp +++ b/src/Plugins/SimplnxCore/test/ComputeGroupingDensityTest.cpp @@ -21,419 +21,368 @@ using namespace nx::core; namespace { -// DataStructure path constants +// ============================================================================= +// Exemplar archive paths (compute_grouping_densities_v2.tar.gz) +// ============================================================================= +const std::string k_TestDataDirName = "compute_grouping_densities_v2"; +const fs::path k_TestDataDir = fs::path(unit_test::k_TestFilesDir.view()) / k_TestDataDirName; +const fs::path k_InputFile = k_TestDataDir / "data" / "compute_grouping_density_inputs.dream3d"; +const fs::path k_ExemplarFile = k_TestDataDir / "output_simplnx" / "simplnx_compute_grouping_density_ab.dream3d"; + +// ============================================================================= +// DataPath constants for the v2 input file +// ============================================================================= +const std::string k_DataContainerName = "DataContainer"; +const std::string k_FeatureAMName = "FeatureData"; +const std::string k_ParentAMName = "ParentData"; + +const auto k_VolumesPath = DataPath({k_DataContainerName, k_FeatureAMName, "Volumes"}); +const auto k_ParentIdsPath = DataPath({k_DataContainerName, k_FeatureAMName, "ParentIds"}); +const auto k_ContiguousNLPath = DataPath({k_DataContainerName, k_FeatureAMName, "ContiguousNeighborList"}); +const auto k_NonContiguousNLPath = DataPath({k_DataContainerName, k_FeatureAMName, "NonContiguousNeighborList"}); +const auto k_ParentVolumesPath = DataPath({k_DataContainerName, k_ParentAMName, "ParentVolumes"}); + +// Output array names (placed by the filter into the same AMs as the inputs) +const std::string k_ComputedGroupingDensitiesName = "ComputedGroupingDensities"; +const std::string k_ComputedCheckedFeaturesName = "ComputedCheckedFeatures"; +const auto k_ComputedGroupingDensitiesPath = DataPath({k_DataContainerName, k_ParentAMName, k_ComputedGroupingDensitiesName}); +const auto k_ComputedCheckedFeaturesPath = DataPath({k_DataContainerName, k_FeatureAMName, k_ComputedCheckedFeaturesName}); + +// Constants used by the inline preflight-error and edge-case tests. const std::string k_ImageGeomName = "ImageGeom"; -const std::string k_FeatureAMName = "CellFeatureData"; -const std::string k_ParentAMName = "ParentFeatureData"; -const std::string k_VolumesName = "Volumes"; -const std::string k_ParentIdsName = "ParentIds"; -const std::string k_ContiguousNLName = "ContiguousNeighborList"; -const std::string k_NonContiguousNLName = "NonContiguousNeighborList"; -const std::string k_ParentVolumesName = "Volumes"; -const std::string k_ComputedGroupingDensitiesName = "Computed GroupingDensities"; -const std::string k_CheckedFeaturesName = "CheckedFeatures"; - -const DataPath k_VolumesPath = DataPath({k_ImageGeomName, k_FeatureAMName, k_VolumesName}); -const DataPath k_ParentIdsPath = DataPath({k_ImageGeomName, k_FeatureAMName, k_ParentIdsName}); -const DataPath k_ContiguousNLPath = DataPath({k_ImageGeomName, k_FeatureAMName, k_ContiguousNLName}); -const DataPath k_NonContiguousNLPath = DataPath({k_ImageGeomName, k_FeatureAMName, k_NonContiguousNLName}); -const DataPath k_ParentVolumesPath = DataPath({k_ImageGeomName, k_ParentAMName, k_ParentVolumesName}); -const DataPath k_GroupingDensitiesPath = DataPath({k_ImageGeomName, k_ParentAMName, k_ComputedGroupingDensitiesName}); -const DataPath k_CheckedFeaturesPath = DataPath({k_ImageGeomName, k_FeatureAMName, k_CheckedFeaturesName}); - -// Test data dimensions matching the 20x5 2D Image Geometry: -// 6 features (index 0 = placeholder, features 1-5) -// 3 parents (index 0 = placeholder, parents 1-2) -// Features 1,2,3 -> Parent 1 (volume = 10+20+15 = 45) -// Features 4,5 -> Parent 2 (volume = 25+30 = 55) constexpr usize k_NumFeatures = 6; constexpr usize k_NumParents = 3; +} // namespace -/** - * @brief Builds a DataStructure with all input data needed for the ComputeGroupingDensity filter. - * Optionally includes a non-contiguous neighbor list. - * - * Data matches the 20x5 2D Image Geometry worked example: - * Feature Volumes: [0, 10, 20, 15, 25, 30] - * Parent IDs: [0, 1, 1, 1, 2, 2] - * Parent Volumes: [0, 45, 55] - * Contiguous Neighbors: chain 1-2-3-4-5 - */ -DataStructure createTestDataStructure(bool includeNonContiguousNL) +// ============================================================================= +// Exemplar-based test: exercises all 4 (UseNonContiguousNeighbors, FindCheckedFeatures) +// configurations against pre-validated outputs in compute_grouping_densities_v2.tar.gz. +// +// The v2 exemplar archive was hand-reviewed and signed off by the filter author, +// and the SIMPLNX outputs in it were independently confirmed bit-identical to +// the legacy DREAM3D 6.5.172 `FindGroupingDensity` filter (the pre-SIMPLNX port +// source) — see src/Plugins/SimplnxCore/vv/ComputeGroupingDensityFilter.md and +// src/Plugins/SimplnxCore/vv/deviations/ComputeGroupingDensityFilter.md. +// +// Driving the test from the same exemplar archive used for the V&V comparison +// gives a single source of truth: any future change to either the algorithm or +// the exemplar surfaces here. +// ============================================================================= + +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Exemplar A/B — all 4 configurations", "[SimplnxCore][ComputeGroupingDensityFilter]") { - DataStructure dataStructure; + UnitTest::LoadPlugins(); - // Create ImageGeom (just a container for the AMs) - auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); - imageGeom->setDimensions({1, 1, 1}); + const nx::core::UnitTest::TestFileSentinel testDataSentinel(nx::core::unit_test::k_TestFilesDir, "compute_grouping_densities_v2.tar.gz", k_TestDataDirName); + + // Generate all 4 (UseNonContig, FindCheckedFeatures) combinations. + // The suffix matches the exemplar array naming inside the v2 archive. + auto config = GENERATE(table({ + {false, false, "NC0_CF0"}, + {false, true, "NC0_CF1"}, + {true, false, "NC1_CF0"}, + {true, true, "NC1_CF1"}, + })); + const bool useNonContiguous = std::get<0>(config); + const bool findCheckedFeatures = std::get<1>(config); + const std::string suffix = std::get<2>(config); + + DYNAMIC_SECTION("Config " << suffix << " UseNonContig=" << useNonContiguous << " FindChecked=" << findCheckedFeatures) + { + // Fresh DataStructure per configuration so output paths don't collide + DataStructure dataStructure = UnitTest::LoadDataStructure(k_InputFile); + + ComputeGroupingDensityFilter filter; + Arguments args; + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(k_VolumesPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(k_ContiguousNLPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(useNonContiguous)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(useNonContiguous ? k_NonContiguousNLPath : DataPath{})); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(k_ParentIdsPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(k_ParentVolumesPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(findCheckedFeatures)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any(k_ComputedCheckedFeaturesName)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any(k_ComputedGroupingDensitiesName)); + + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + + auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + // Load the exemplar — it contains all 4 pre-computed output configurations + // as separately-named arrays (e.g., "GroupingDensities_NC0_CF1") + DataStructure exemplarDS = UnitTest::LoadDataStructure(k_ExemplarFile); + + // GroupingDensities is always produced + const DataPath exemplarDensitiesPath = DataPath({k_DataContainerName, k_ParentAMName, "GroupingDensities_" + suffix}); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ComputedGroupingDensitiesPath)); + REQUIRE_NOTHROW(exemplarDS.getDataRefAs(exemplarDensitiesPath)); + const auto& computedDensities = dataStructure.getDataRefAs(k_ComputedGroupingDensitiesPath); + const auto& exemplarDensities = exemplarDS.getDataRefAs(exemplarDensitiesPath); + UnitTest::CompareDataArrays(exemplarDensities, computedDensities); + + // CheckedFeatures is produced only when FindCheckedFeatures==true + if(findCheckedFeatures) + { + const DataPath exemplarCheckedFeaturesPath = DataPath({k_DataContainerName, k_FeatureAMName, "CheckedFeatures_" + suffix}); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ComputedCheckedFeaturesPath)); + REQUIRE_NOTHROW(exemplarDS.getDataRefAs(exemplarCheckedFeaturesPath)); + const auto& computedCheckedFeatures = dataStructure.getDataRefAs(k_ComputedCheckedFeaturesPath); + const auto& exemplarCheckedFeatures = exemplarDS.getDataRefAs(exemplarCheckedFeaturesPath); + UnitTest::CompareDataArrays(exemplarCheckedFeatures, computedCheckedFeatures); + } + + // Class 4 (Invariant) oracle assertions — properties that any valid output must satisfy. + // Independent of the exemplar bit-comparison above; these would catch a future + // regression even if someone "fixed" the exemplar incorrectly to match buggy code. + const auto& invDensities = dataStructure.getDataRefAs(k_ComputedGroupingDensitiesPath); + REQUIRE(invDensities[0] == 0.0f); // placeholder parent never touched + for(usize i = 1; i < invDensities.getNumberOfTuples(); ++i) + { + // density is either positive (parent had at least one assigned feature) + // or exactly the -1.0f sentinel (parent had no assigned features). + REQUIRE((invDensities[i] > 0.0f || invDensities[i] == -1.0f)); + // totalCheckVolume always includes the parent's own features, + // so it is never smaller than ParentVolumes[i] -> density <= 1.0. + REQUIRE(invDensities[i] <= 1.0f); + } + if(findCheckedFeatures) + { + const auto& invChecked = dataStructure.getDataRefAs(k_ComputedCheckedFeaturesPath); + REQUIRE(invChecked[0] == 0); // placeholder feature never claimed + const int32 maxParentId = static_cast(invDensities.getNumberOfTuples()) - 1; + for(usize i = 1; i < invChecked.getNumberOfTuples(); ++i) + { + REQUIRE((invChecked[i] >= 0 && invChecked[i] <= maxParentId)); + } + } - // Feature-level AttributeMatrix (6 tuples: indices 0-5) - auto* featureAM = AttributeMatrix::Create(dataStructure, k_FeatureAMName, {k_NumFeatures}, imageGeom->getId()); + UnitTest::CheckArraysInheritTupleDims(dataStructure); + } +} - // Parent-level AttributeMatrix (3 tuples: indices 0-2) - auto* parentAM = AttributeMatrix::Create(dataStructure, k_ParentAMName, {k_NumParents}, imageGeom->getId()); +// ============================================================================= +// Edge case: a parent with no features assigned -> totalCheckVolume == 0 +// triggers the -1.0f sentinel write at ComputeGroupingDensity.cpp line 114. +// Not exercised by the v2 exemplar (every parent has features). +// ============================================================================= + +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Empty-parent edge case (-1.0f sentinel)", "[SimplnxCore][ComputeGroupingDensityFilter]") +{ + UnitTest::LoadPlugins(); + + // 3 features (indices 1, 2 carry data; 0 is the SIMPL placeholder). + // 3 parents (index 1 has features assigned; index 2 has NONE). + DataStructure dataStructure; + auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); + imageGeom->setDimensions({1, 1, 1}); - // --- Feature-level arrays --- + auto* featureAM = AttributeMatrix::Create(dataStructure, "FeatureData", {3}, imageGeom->getId()); + auto* parentAM = AttributeMatrix::Create(dataStructure, "ParentData", {3}, imageGeom->getId()); - // Feature Volumes: [0, 10, 20, 15, 25, 30] - auto* featureVolumes = UnitTest::CreateTestDataArray(dataStructure, k_VolumesName, {k_NumFeatures}, {1}, featureAM->getId()); + // Feature volumes: [0, 5, 10] + auto* featureVolumes = UnitTest::CreateTestDataArray(dataStructure, "Volumes", {3}, {1}, featureAM->getId()); auto& featureVolumesRef = featureVolumes->getDataStoreRef(); featureVolumesRef[0] = 0.0f; - featureVolumesRef[1] = 10.0f; - featureVolumesRef[2] = 20.0f; - featureVolumesRef[3] = 15.0f; - featureVolumesRef[4] = 25.0f; - featureVolumesRef[5] = 30.0f; - - // Parent IDs: [0, 1, 1, 1, 2, 2] - auto* parentIds = UnitTest::CreateTestDataArray(dataStructure, k_ParentIdsName, {k_NumFeatures}, {1}, featureAM->getId()); + featureVolumesRef[1] = 5.0f; + featureVolumesRef[2] = 10.0f; + + // All non-placeholder features map to parent 1; parent 2 has no features. + auto* parentIds = UnitTest::CreateTestDataArray(dataStructure, "ParentIds", {3}, {1}, featureAM->getId()); auto& parentIdsRef = parentIds->getDataStoreRef(); parentIdsRef[0] = 0; parentIdsRef[1] = 1; parentIdsRef[2] = 1; - parentIdsRef[3] = 1; - parentIdsRef[4] = 2; - parentIdsRef[5] = 2; - - // Contiguous Neighbor List (chain: 1-2-3-4-5) - // Feature 0: {} - // Feature 1: {2} - // Feature 2: {1, 3} - // Feature 3: {2, 4} - // Feature 4: {3, 5} - // Feature 5: {4} - auto* contiguousNL = NeighborList::Create(dataStructure, k_ContiguousNLName, ShapeType{k_NumFeatures}, featureAM->getId()); - contiguousNL->setList(0, std::make_shared>(std::vector{})); - contiguousNL->setList(1, std::make_shared>(std::vector{2})); - contiguousNL->setList(2, std::make_shared>(std::vector{1, 3})); - contiguousNL->setList(3, std::make_shared>(std::vector{2, 4})); - contiguousNL->setList(4, std::make_shared>(std::vector{3, 5})); - contiguousNL->setList(5, std::make_shared>(std::vector{4})); - - // Non-Contiguous Neighbor List (optional) - // Feature 0: {} - // Feature 1: {4} - // Feature 2: {5} - // Feature 3: {} - // Feature 4: {1} - // Feature 5: {2} - if(includeNonContiguousNL) - { - auto* nonContiguousNL = NeighborList::Create(dataStructure, k_NonContiguousNLName, ShapeType{k_NumFeatures}, featureAM->getId()); - nonContiguousNL->setList(0, std::make_shared>(std::vector{})); - nonContiguousNL->setList(1, std::make_shared>(std::vector{4})); - nonContiguousNL->setList(2, std::make_shared>(std::vector{5})); - nonContiguousNL->setList(3, std::make_shared>(std::vector{})); - nonContiguousNL->setList(4, std::make_shared>(std::vector{1})); - nonContiguousNL->setList(5, std::make_shared>(std::vector{2})); - } - // --- Parent-level arrays --- + // Trivial contiguous neighbor list — empty for every feature. + auto* contiguousNL = NeighborList::Create(dataStructure, "ContigNL", ShapeType{3}, featureAM->getId()); + contiguousNL->setList(0, std::make_shared>(std::vector{})); + contiguousNL->setList(1, std::make_shared>(std::vector{})); + contiguousNL->setList(2, std::make_shared>(std::vector{})); - // Parent Volumes: [0, 45, 55] (sum of child feature cell volumes) - auto* parentVolumes = UnitTest::CreateTestDataArray(dataStructure, k_ParentVolumesName, {k_NumParents}, {1}, parentAM->getId()); + // Parent volumes: parent 1 sums to 15 (5+10); parent 2 is non-zero but + // irrelevant — totalCheckVolume==0 path triggers regardless of ParentVolumes[2]. + auto* parentVolumes = UnitTest::CreateTestDataArray(dataStructure, "ParentVolumes", {3}, {1}, parentAM->getId()); auto& parentVolumesRef = parentVolumes->getDataStoreRef(); parentVolumesRef[0] = 0.0f; - parentVolumesRef[1] = 45.0f; - parentVolumesRef[2] = 55.0f; - - return dataStructure; -} - -/** - * @brief Creates the filter Arguments for the given boolean option combination. - */ -Arguments createFilterArgs(bool useNonContiguous, bool findCheckedFeatures) -{ - Arguments args; - args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(k_VolumesPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(k_ContiguousNLPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(useNonContiguous)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(useNonContiguous ? k_NonContiguousNLPath : DataPath{})); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(k_ParentIdsPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(k_ParentVolumesPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(findCheckedFeatures)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any(k_CheckedFeaturesName)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any(k_ComputedGroupingDensitiesName)); - return args; -} -} // namespace - -// ============================================================================= -// Exemplar-Based Test - Compare against DREAM3D-NX pipeline output -// ============================================================================= - -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Basic Density (contiguous, no checked features)", "[SimplnxReview][ComputeGroupingDensityFilter]") -{ - - const std::string k_GroupingDensitiesName = "GroupingDensities (false, false)"; - const DataPath k_ExemplarGroupingDensitiesPath = DataPath({k_ImageGeomName, k_ParentAMName, k_GroupingDensitiesName}); - - UnitTest::LoadPlugins(); - - const nx::core::UnitTest::TestFileSentinel testDataSentinel(nx::core::unit_test::k_TestFilesDir, "compute_grouping_densities.tar.gz", "compute_grouping_densities"); + parentVolumesRef[1] = 15.0f; + parentVolumesRef[2] = 7.0f; - // Read Exemplar DREAM3D File Filter - auto exemplarFilePath = fs::path(fmt::format("{}/compute_grouping_densities/compute_grouping_densities.dream3d", unit_test::k_TestFilesDir)); - DataStructure dataStructure = UnitTest::LoadDataStructure(exemplarFilePath); + const DataPath volumesPath = DataPath({k_ImageGeomName, "FeatureData", "Volumes"}); + const DataPath parentIdsPath = DataPath({k_ImageGeomName, "FeatureData", "ParentIds"}); + const DataPath contigNLPath = DataPath({k_ImageGeomName, "FeatureData", "ContigNL"}); + const DataPath parentVolumesPath = DataPath({k_ImageGeomName, "ParentData", "ParentVolumes"}); + const DataPath outputDensitiesPath = DataPath({k_ImageGeomName, "ParentData", "GroupingDensities"}); ComputeGroupingDensityFilter filter; Arguments args; - args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(k_VolumesPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(k_ContiguousNLPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(volumesPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(contigNLPath)); args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(false)); args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(DataPath{})); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(k_ParentIdsPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(k_ParentVolumesPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(parentIdsPath)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(parentVolumesPath)); args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(false)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any(k_CheckedFeaturesName)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any(k_ComputedGroupingDensitiesName)); - - // Preflight the filter and check result - auto preflightResult = filter.preflight(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any("CheckedFeatures")); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any("GroupingDensities")); auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); - // Compare computed densities against the exemplar from the DREAM3D-NX pipeline - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_GroupingDensitiesPath)); - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ExemplarGroupingDensitiesPath)); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(outputDensitiesPath)); + const auto& densities = dataStructure.getDataRefAs(outputDensitiesPath); - const auto& computedDensities = dataStructure.getDataRefAs(k_GroupingDensitiesPath); - const auto& exemplarDensities = dataStructure.getDataRefAs(k_ExemplarGroupingDensitiesPath); + // Parent 1: totalCheckVolume = 5 + 10 = 15; density = 15/15 = 1.0 + // Parent 2: NO features assigned -> totalCheckVolume == 0 -> -1.0f sentinel + REQUIRE(densities[1] == Approx(1.0f).epsilon(0.0001f)); + REQUIRE(densities[2] == -1.0f); - REQUIRE(computedDensities.getNumberOfTuples() == exemplarDensities.getNumberOfTuples()); - for(usize i = 0; i < computedDensities.getNumberOfTuples(); i++) - { - REQUIRE(computedDensities[i] == Approx(exemplarDensities[i]).epsilon(0.0001f)); - } - - // Verify against hand-calculated values: - // Parent Volumes: [0, 45, 55] - // Parent 1: children {1,2,3}, neighbors add feature 4 - // totalCheckVolume = 10 + 20 + 15 + 25 = 70 - // density = 45 / 70 = 0.642857 - // Parent 2: children {4,5}, neighbors add feature 3 - // totalCheckVolume = 25 + 30 + 15 = 70 - // density = 55 / 70 = 0.785714 - REQUIRE(computedDensities[1] == Approx(45.0f / 70.0f).epsilon(0.0001f)); - REQUIRE(computedDensities[2] == Approx(55.0f / 70.0f).epsilon(0.0001f)); + UnitTest::CheckArraysInheritTupleDims(dataStructure); } // ============================================================================= -// Execution Tests - Exercise all 4 template specializations +// Preflight error tests — one TEST_CASE per error code in preflightImpl(). // ============================================================================= -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Contiguous Only, No Checked Features", "[SimplnxReview][ComputeGroupingDensityFilter]") -{ - UnitTest::LoadPlugins(); - - DataStructure dataStructure = createTestDataStructure(false); - ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(false, false); - - auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); - SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); - - // Parent 1: children {1,2,3}, contiguous neighbors add feature 4 - // totalCheckVolume = 10 + 20 + 15 + 25 = 70 - // density = 45 / 70 = 0.642857 - // Parent 2: children {4,5}, contiguous neighbors add feature 3 - // totalCheckVolume = 25 + 30 + 15 = 70 - // density = 55 / 70 = 0.785714 - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_GroupingDensitiesPath)); - const auto& groupingDensities = dataStructure.getDataRefAs(k_GroupingDensitiesPath); - - REQUIRE(groupingDensities[1] == Approx(45.0f / 70.0f).epsilon(0.0001f)); - REQUIRE(groupingDensities[2] == Approx(55.0f / 70.0f).epsilon(0.0001f)); -} - -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: With Non-Contiguous Neighbors", "[SimplnxReview][ComputeGroupingDensityFilter]") -{ - UnitTest::LoadPlugins(); - - DataStructure dataStructure = createTestDataStructure(true); - ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(true, false); - - auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); - SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); - - // With non-contiguous neighbors, all 5 features get checked for each parent - // Parent 1: totalCheckVolume = 10+20+15+25+30 = 100, density = 45/100 = 0.45 - // Parent 2: totalCheckVolume = 25+30+15+10+20 = 100, density = 55/100 = 0.55 - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_GroupingDensitiesPath)); - const auto& groupingDensities = dataStructure.getDataRefAs(k_GroupingDensitiesPath); - - REQUIRE(groupingDensities[1] == Approx(45.0f / 100.0f).epsilon(0.0001f)); - REQUIRE(groupingDensities[2] == Approx(55.0f / 100.0f).epsilon(0.0001f)); -} - -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: With Checked Features", "[SimplnxReview][ComputeGroupingDensityFilter]") +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Preflight Error - Feature tuple count mismatch (-15671)", "[SimplnxCore][ComputeGroupingDensityFilter]") { UnitTest::LoadPlugins(); - DataStructure dataStructure = createTestDataStructure(false); - ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(false, true); - - auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); - SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + // ParentIds in a different AM with mismatched tuple count + DataStructure dataStructure; + auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); + imageGeom->setDimensions({1, 1, 1}); - // Densities same as contiguous-only case - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_GroupingDensitiesPath)); - const auto& groupingDensities = dataStructure.getDataRefAs(k_GroupingDensitiesPath); - - REQUIRE(groupingDensities[1] == Approx(45.0f / 70.0f).epsilon(0.0001f)); - REQUIRE(groupingDensities[2] == Approx(55.0f / 70.0f).epsilon(0.0001f)); - - // Checked features: each feature is assigned to the parent with the largest volume that checked it - // Parent 1 (vol=45) processes first and checks features {1,2,3,4} - // Parent 2 (vol=55) processes second and checks features {3,4,5} - // Feature 3: checked by Parent 1 (45) then Parent 2 (55 > 45) -> overridden to Parent 2 - // Feature 4: checked by Parent 1 (45) then Parent 2 (55 > 45) -> overridden to Parent 2 - // Feature 5: only checked by Parent 2 - // Expected: [0, 1, 1, 2, 2, 2] - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_CheckedFeaturesPath)); - const auto& checkedFeatures = dataStructure.getDataRefAs(k_CheckedFeaturesPath); - - REQUIRE(checkedFeatures[0] == 0); - REQUIRE(checkedFeatures[1] == 1); - REQUIRE(checkedFeatures[2] == 1); - REQUIRE(checkedFeatures[3] == 2); - REQUIRE(checkedFeatures[4] == 2); - REQUIRE(checkedFeatures[5] == 2); -} + auto* featureAM = AttributeMatrix::Create(dataStructure, "FeatureData", {k_NumFeatures}, imageGeom->getId()); + auto* parentAM = AttributeMatrix::Create(dataStructure, "ParentData", {k_NumParents}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "Volumes", {k_NumFeatures}, {1}, featureAM->getId()); + NeighborList::Create(dataStructure, "ContigNL", ShapeType{k_NumFeatures}, featureAM->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentVolumes", {k_NumParents}, {1}, parentAM->getId()); -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Both Options Enabled", "[SimplnxReview][ComputeGroupingDensityFilter]") -{ - UnitTest::LoadPlugins(); + auto* mismatchAM = AttributeMatrix::Create(dataStructure, "MismatchAM", {10}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentIds", {10}, {1}, mismatchAM->getId()); - DataStructure dataStructure = createTestDataStructure(true); ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(true, true); - - auto executeResult = filter.execute(dataStructure, args, nullptr, IFilter::MessageHandler{[](const IFilter::Message& message) { fmt::print("{}\n", message.message); }}); - SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + Arguments args; + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "Volumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "ContigNL"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(DataPath{})); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(DataPath({k_ImageGeomName, "MismatchAM", "ParentIds"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(DataPath({k_ImageGeomName, "ParentData", "ParentVolumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any("CheckedFeatures")); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any("GroupingDensities")); - // With non-contiguous neighbors, all features get checked by both parents - // Parent 1: density = 45/100 = 0.45 - // Parent 2: density = 55/100 = 0.55 - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_GroupingDensitiesPath)); - const auto& groupingDensities = dataStructure.getDataRefAs(k_GroupingDensitiesPath); - - REQUIRE(groupingDensities[1] == Approx(45.0f / 100.0f).epsilon(0.0001f)); - REQUIRE(groupingDensities[2] == Approx(55.0f / 100.0f).epsilon(0.0001f)); - - // Parent 1 (vol=45) checks ALL features {1,2,3,4,5} via non-contiguous links - // Parent 2 (vol=55) also checks ALL features, and 55 > 45 so all get overridden - // Expected: [0, 2, 2, 2, 2, 2] - REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_CheckedFeaturesPath)); - const auto& checkedFeatures = dataStructure.getDataRefAs(k_CheckedFeaturesPath); - - REQUIRE(checkedFeatures[0] == 0); - REQUIRE(checkedFeatures[1] == 2); - REQUIRE(checkedFeatures[2] == 2); - REQUIRE(checkedFeatures[3] == 2); - REQUIRE(checkedFeatures[4] == 2); - REQUIRE(checkedFeatures[5] == 2); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); } -// ============================================================================= -// Preflight Error Tests -// ============================================================================= - -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Preflight Error - Feature tuple count mismatch", "[SimplnxReview][ComputeGroupingDensityFilter]") +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Preflight Error - NonContiguousNL tuple count mismatch (-15672)", "[SimplnxCore][ComputeGroupingDensityFilter]") { UnitTest::LoadPlugins(); - // Build a DataStructure where ParentIds has a different tuple count than Volumes + // Feature-level arrays are all 6 tuples; NonContiguousNL is 4 tuples in a different AM DataStructure dataStructure; auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); imageGeom->setDimensions({1, 1, 1}); - auto* featureAM = AttributeMatrix::Create(dataStructure, k_FeatureAMName, {k_NumFeatures}, imageGeom->getId()); - auto* parentAM = AttributeMatrix::Create(dataStructure, k_ParentAMName, {k_NumParents}, imageGeom->getId()); - - // Volumes with 6 tuples - UnitTest::CreateTestDataArray(dataStructure, k_VolumesName, {k_NumFeatures}, {1}, featureAM->getId()); - // Contiguous NL with 6 tuples - NeighborList::Create(dataStructure, k_ContiguousNLName, ShapeType{k_NumFeatures}, featureAM->getId()); - // Parent Volumes - UnitTest::CreateTestDataArray(dataStructure, k_ParentVolumesName, {k_NumParents}, {1}, parentAM->getId()); - - // ParentIds in a DIFFERENT AM with a different tuple count (mismatch!) - auto* mismatchAM = AttributeMatrix::Create(dataStructure, "MismatchAM", {10}, imageGeom->getId()); - UnitTest::CreateTestDataArray(dataStructure, k_ParentIdsName, {10}, {1}, mismatchAM->getId()); + auto* featureAM = AttributeMatrix::Create(dataStructure, "FeatureData", {k_NumFeatures}, imageGeom->getId()); + auto* parentAM = AttributeMatrix::Create(dataStructure, "ParentData", {k_NumParents}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "Volumes", {k_NumFeatures}, {1}, featureAM->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentIds", {k_NumFeatures}, {1}, featureAM->getId()); + NeighborList::Create(dataStructure, "ContigNL", ShapeType{k_NumFeatures}, featureAM->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentVolumes", {k_NumParents}, {1}, parentAM->getId()); - DataPath mismatchParentIdsPath = DataPath({k_ImageGeomName, "MismatchAM", k_ParentIdsName}); + auto* mismatchAM = AttributeMatrix::Create(dataStructure, "MismatchAM", {4}, imageGeom->getId()); + NeighborList::Create(dataStructure, "NonContigNL", ShapeType{4}, mismatchAM->getId()); ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(false, false); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(mismatchParentIdsPath)); + Arguments args; + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "Volumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "ContigNL"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(true)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "MismatchAM", "NonContigNL"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "ParentIds"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(DataPath({k_ImageGeomName, "ParentData", "ParentVolumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any("CheckedFeatures")); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any("GroupingDensities")); auto preflightResult = filter.preflight(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); } -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Preflight Error - Volumes not in AttributeMatrix", "[SimplnxReview][ComputeGroupingDensityFilter]") +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Preflight Error - Volumes not in AttributeMatrix (-15673)", "[SimplnxCore][ComputeGroupingDensityFilter]") { UnitTest::LoadPlugins(); - // Build a DataStructure where Volumes is NOT inside an AttributeMatrix + // Volumes placed directly under ImageGeom (no AttributeMatrix parent) DataStructure dataStructure; auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); imageGeom->setDimensions({1, 1, 1}); - // Create volumes directly under the ImageGeom (not in an AM) - UnitTest::CreateTestDataArray(dataStructure, k_VolumesName, {k_NumFeatures}, {1}, imageGeom->getId()); - UnitTest::CreateTestDataArray(dataStructure, k_ParentIdsName, {k_NumFeatures}, {1}, imageGeom->getId()); - NeighborList::Create(dataStructure, k_ContiguousNLName, ShapeType{k_NumFeatures}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "Volumes", {k_NumFeatures}, {1}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentIds", {k_NumFeatures}, {1}, imageGeom->getId()); + NeighborList::Create(dataStructure, "ContigNL", ShapeType{k_NumFeatures}, imageGeom->getId()); - auto* parentAM = AttributeMatrix::Create(dataStructure, k_ParentAMName, {k_NumParents}, imageGeom->getId()); - UnitTest::CreateTestDataArray(dataStructure, k_ParentVolumesName, {k_NumParents}, {1}, parentAM->getId()); - - DataPath volumesNoAMPath = DataPath({k_ImageGeomName, k_VolumesName}); - DataPath parentIdsNoAMPath = DataPath({k_ImageGeomName, k_ParentIdsName}); - DataPath contiguousNLNoAMPath = DataPath({k_ImageGeomName, k_ContiguousNLName}); + auto* parentAM = AttributeMatrix::Create(dataStructure, "ParentData", {k_NumParents}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentVolumes", {k_NumParents}, {1}, parentAM->getId()); ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(false, false); - args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(volumesNoAMPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(parentIdsNoAMPath)); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(contiguousNLNoAMPath)); + Arguments args; + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "Volumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "ContigNL"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(DataPath{})); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(DataPath({k_ImageGeomName, "ParentIds"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(DataPath({k_ImageGeomName, "ParentData", "ParentVolumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any("CheckedFeatures")); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any("GroupingDensities")); auto preflightResult = filter.preflight(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); } -TEST_CASE("SimplnxReview::ComputeGroupingDensityFilter: Preflight Error - Parent Volumes not in AttributeMatrix", "[SimplnxReview][ComputeGroupingDensityFilter]") +TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: Preflight Error - Parent Volumes not in AttributeMatrix (-15670)", "[SimplnxCore][ComputeGroupingDensityFilter]") { UnitTest::LoadPlugins(); + // ParentVolumes placed directly under ImageGeom (no AM parent) DataStructure dataStructure; auto* imageGeom = ImageGeom::Create(dataStructure, k_ImageGeomName); imageGeom->setDimensions({1, 1, 1}); - auto* featureAM = AttributeMatrix::Create(dataStructure, k_FeatureAMName, {k_NumFeatures}, imageGeom->getId()); - UnitTest::CreateTestDataArray(dataStructure, k_VolumesName, {k_NumFeatures}, {1}, featureAM->getId()); - UnitTest::CreateTestDataArray(dataStructure, k_ParentIdsName, {k_NumFeatures}, {1}, featureAM->getId()); - NeighborList::Create(dataStructure, k_ContiguousNLName, ShapeType{k_NumFeatures}, featureAM->getId()); - - // Parent Volumes directly under ImageGeom (not in AM) - UnitTest::CreateTestDataArray(dataStructure, k_ParentVolumesName, {k_NumParents}, {1}, imageGeom->getId()); + auto* featureAM = AttributeMatrix::Create(dataStructure, "FeatureData", {k_NumFeatures}, imageGeom->getId()); + UnitTest::CreateTestDataArray(dataStructure, "Volumes", {k_NumFeatures}, {1}, featureAM->getId()); + UnitTest::CreateTestDataArray(dataStructure, "ParentIds", {k_NumFeatures}, {1}, featureAM->getId()); + NeighborList::Create(dataStructure, "ContigNL", ShapeType{k_NumFeatures}, featureAM->getId()); - DataPath parentVolumesNoAMPath = DataPath({k_ImageGeomName, k_ParentVolumesName}); + UnitTest::CreateTestDataArray(dataStructure, "ParentVolumes", {k_NumParents}, {1}, imageGeom->getId()); ComputeGroupingDensityFilter filter; - Arguments args = createFilterArgs(false, false); - args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(parentVolumesNoAMPath)); + Arguments args; + args.insertOrAssign(ComputeGroupingDensityFilter::k_FeatureVolumesArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "Volumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ContiguousNeighborListArrayPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "ContigNL"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_UseNonContiguousNeighbors_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_NonContiguousNeighborListArrayPath_Key, std::make_any(DataPath{})); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentIdsPath_Key, std::make_any(DataPath({k_ImageGeomName, "FeatureData", "ParentIds"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_ParentVolumesPath_Key, std::make_any(DataPath({k_ImageGeomName, "ParentVolumes"}))); + args.insertOrAssign(ComputeGroupingDensityFilter::k_FindCheckedFeatures_Key, std::make_any(false)); + args.insertOrAssign(ComputeGroupingDensityFilter::k_CheckedFeaturesName_Key, std::make_any("CheckedFeatures")); + args.insertOrAssign(ComputeGroupingDensityFilter::k_GroupingDensitiesName_Key, std::make_any("GroupingDensities")); auto preflightResult = filter.preflight(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); } +// ============================================================================= +// SIMPL JSON backwards-compatibility — verifies FromSIMPLJson() correctly +// translates the SIMPL 6.5 filter parameter keys to the simplnx Arguments. +// ============================================================================= + TEST_CASE("SimplnxCore::ComputeGroupingDensityFilter: SIMPL Backwards Compatibility", "[SimplnxCore][ComputeGroupingDensityFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/vv/ComputeGroupingDensityFilter.md b/src/Plugins/SimplnxCore/vv/ComputeGroupingDensityFilter.md new file mode 100644 index 0000000000..34384da8b2 --- /dev/null +++ b/src/Plugins/SimplnxCore/vv/ComputeGroupingDensityFilter.md @@ -0,0 +1,147 @@ +# V&V Report: ComputeGroupingDensityFilter + +| | | +|---|---| +| Plugin | SimplnxCore | +| SIMPLNX UUID | ff46afcf-de32-4f37-98bc-8f0fd4b3c122 | +| DREAM3D 6.5.171 equivalent | `FindGroupingDensity` (SIMPL UUID `708be082-8b08-4db2-94be-52781ed4d53d`) — *unreleased pre-simplnx implementation* on `tuks188/DREAM3D` `feature/770_Grouping_Density`; never merged into `v6_5_171`. Served as the port source. | +| Verified commit | ** | +| Status | **COMPLETE — 2026-05-27** | +| Sign-off | Michael Jackson (BlueQuartz Software), 2026-05-27 | + +## Summary + +- This **Filter** computes a **Grouping Density** value for each **Parent Feature** in a hierarchical reconstruction. Hierarchical reconstructions involve more than one level of segmentation, creating a **Feature** to **Parent Feature** relationship (e.g., grains grouped into reconstructed parent grains). +- The filter was verified by generating a small data set that exercises each code path and combination of featureIds. The final calculations were done by hand and then verified by executing the filter. +- The result is that the filter generates the expected output values + + +## Resolution (V&V outcomes — 2026-05-27) + +| Topic | DRAFT-tentative finding | Final confirmed finding | +|---|---|---| +| Algorithm Relationship | Port (with rename `Find→Compute`) | **Port** — confirmed. The SIMPLNX algorithm is a line-by-line translation of the legacy `FindGroupingDensity::execute()` from `tuks188/DREAM3D` `feature/770_Grouping_Density`. Six port-time deltas documented in the V&V report, none change output. | +| Oracle class | Class 1 (Analytical) primary, optionally Class 4 (Invariant) | **Class 1 + Class 4 confirmed.** Hand-derivation embedded in V&V report; invariant predicates added inline in the test (sentinel `-1.0f`, `density ≤ 1.0`, `density > 0 ∨ == -1.0f`, CheckedFeatures range). Second-engineer review skipped — set-union arithmetic on 5-feature toy dataset + bit-identical cross-check against independently-built legacy. | +| Legacy comparison | Not yet run | **Done. No deviations.** All 4 `(UseNonContiguous, FindCheckedFeatures)` configurations produce bit-identical output between SIMPLNX and the locally-rebuilt legacy `FindGroupingDensity` (`/Users/mjackson/DREAM3D-Dev/DREAM3D` 6.5.172 branch with feature-branch sources). See `vv/deviations/ComputeGroupingDensityFilter.md`. | +| `[SimplnxReview]` test-tag bug | Flagged as cleanup-needed | **Fixed.** All 7 tests now use `[SimplnxCore][ComputeGroupingDensityFilter]`. | +| Exemplar archive | `compute_grouping_densities.tar.gz` (v1) — provenance TBD | **Replaced.** v1 archive retired from this filter's tests; new `compute_grouping_densities_v2.tar.gz` published to the GitHub Data_Archive with hand-review sign-off in its inline ReadMe + comparison report. SHA512 wired into `test/CMakeLists.txt`. | +| Test inventory | 8 tests, 4 redundant `(NC, CF)` execution tests + 1 exemplar test | **Restructured to 7 tests.** 5 redundant tests replaced by a single DYNAMIC_SECTION exemplar A/B test covering all 4 configurations. Added: empty-parent sentinel test (Class 4) + preflight error -15672 test (gap closed). Kept: 3 existing preflight error tests + SIMPL backwards-compat test. | +| Deviation entries (`ComputeGroupingDensity-D`) | Placeholder pending comparison | **None.** No deviations observed across all 4 configurations. See `vv/deviations/ComputeGroupingDensityFilter.md` for the comparison method, fixture, SHA512, and migration recommendation. | +| Algorithm review (`review-algorithm`) | Not visible from PR history | **Done.** Code-comment cleanup applied (sentinel documented, deleted-special-members trailing comments removed, throwaway-placeholder pattern explained, FindDensitySpecializations + FindDensityGrouping class docs added). Memory / formatting / overflow concerns mitigated by the engineer. Tie-break behavior added to the user-facing filter doc. | +| Filter documentation (`review-filter-docs`) | "Excellent — empty References section is the one defect" | **Updated.** Added Required Input Sources section with MyST cross-links to upstream producer filters. Stated volume units explicitly. MyST-linked the inline Compute Feature Neighborhoods mention. Italicized the `-1.0` sentinel as a value reference. References + Example Pipelines remain empty (no published paper; no shipping pipeline currently uses this filter). | +| Verification archive (OneDrive) | Not yet created | **Materially captured by the v2 GitHub Data_Archive release** — the v2 tarball contains the input file, all 4 legacy and SIMPLNX outputs, the comparison script + report, the legacy and SIMPLNX pipelines, and the hand-review sign-off ReadMe. OneDrive duplication can be done at SBIR deliverable assembly. | + + +## Algorithm Relationship + + +*Classification:* **Port** + +*Evidence:* The SIMPLNX algorithm at `Algorithms/ComputeGroupingDensity.cpp` (219 lines) is a line-by-line translation of the legacy `FindGroupingDensity::execute()` on `tuks188/DREAM3D` `feature/770_Grouping_Density` (469-line `.cpp` file; ~80-line algorithm body). Identical control flow (nested parent×feature loops + neighbor-list walks), identical sentinel (`-1.0f` when `totalCheckVolume == 0.0f`), identical density formula (`curParentVolume / totalCheckVolume`), and a preserved-from-legacy variable-name lineage (`totalCheckVolume` → `totalFeatureCheckVolume`, `checkedfeaturevolumes` → `checkedFeatureVolumes`). Same SIMPL UUID retained via `SimplnxCoreLegacyUUIDMapping.hpp` + SIMPL conversion fixture at `test/simpl_conversion/6_5/ComputeGroupingDensityFilter.json`. **Important caveat:** the legacy filter was never officially released in any DREAM3D 6.5.x — it lived on an un-merged feature branch on a contributor fork. However, a small number of important customers consumed a custom DREAM3D 6.5.x build that included this filter, and those customers have downstream data dependent on its output. Therefore the policy's diff-explanation purpose **does** apply to this filter, just with a narrower migrant audience than usual. The legacy comparison evidence is captured in `vv/deviations/ComputeGroupingDensityFilter.md` via an A/B run against a local rebuild of the legacy source (`/Users/mjackson/DREAM3D-Dev/DREAM3D` on the 6.5.172 branch with the feature-branch sources pulled in). Verification still requires an independent oracle (see Oracle section). + +*Port-time deltas that do not change output (defensible "Port" rather than "Minor changes"):* + +1. `QVector` totalCheckList (linear `.contains()`, O(n²) per parent) → `std::unordered_set` (O(1) membership, O(n) per parent) — performance, no behavior change. +2. Runtime `if (m_FindCheckedFeatures == true)` in the inner loop → `if constexpr (FindingCheckedFeatures)` template specialization — performance, no behavior change. +3. Runtime `for(k=0; k checkedfeaturevolumes(numfeatures, 0.0f)` always allocated → conditionally allocated only when `FindCheckedFeatures==true` — memory savings, no behavior change (legacy zeros were unread when the flag was false). +5. Added: `m_ShouldCancel` check in the outer parent loop (legacy has no cancel support). +6. Added: `ThrottledMessenger` per-parent progress (legacy emits one terminal "Complete" message). + +*Material PRs since baseline (2025-10-01):* + +- **#1548** — "FILT: Compute Grouping Density filter added." (merge `30c9b1090`, 2026-02-25) — initial port from the legacy feature branch: filter + algorithm (.hpp/.cpp), docs (with worked example + 3 figures), `FromSIMPLJson()` conversion path, legacy UUID map entry, 431-line test file covering all 4 template specializations + 3 preflight error tests, exemplar archive. +- *(excluded — broad refactor)* #1588 — "ENH: SIMPL Backwards Compatibility Test Redesign" (merge `f854bb636`, 2026-04-22) — on the cross-cutting exclusion list; added only the per-filter SIMPL backwards-compat fixture at `test/simpl_conversion/6_5/ComputeGroupingDensityFilter.json`. No algorithm change. + + + +## Oracle + +*Class:* **1 (Analytical)** primary, **4 (Invariant)** companion. + +*Applied (Class 1 — Analytical):* Expected outputs are hand-derived from the input definition without reference to any DREAM3D implementation. For each parent index `i ≥ 1`: + +1. `assigned = {j : ParentIds[j] == i}` +2. `touched = assigned ∪ {nbr : nbr ∈ contiguousNL[j], j ∈ assigned}` (and additionally `∪ {nbr : nbr ∈ nonContiguousNL[j], j ∈ assigned}` when `UseNonContiguousNeighbors == true`) +3. `totalCheckVolume[i] = Σ Volumes[k] for k ∈ touched` +4. `GroupingDensities[i] = ParentVolumes[i] / totalCheckVolume[i]`, or `-1.0f` sentinel when `totalCheckVolume[i] == 0` + +For `CheckedFeatures[k]` (when `FindCheckedFeatures == true`): the parent with the largest `ParentVolumes` among the parents that touched feature `k` (last-writer-wins-on-greater-volume semantics in the algorithm). + +Hand-derivation on the v2 toy dataset (`Volumes = [0,10,20,15,25,30]`, `ParentIds = [0,1,1,1,2,2]`, `ParentVolumes = [0,45,55]`, contiguous chain `1↔2↔3↔4↔5`, non-contiguous pairs `1↔4` and `2↔5`): + +| Config (NC, CF) | Parent 1 touched | Σ Vol | `density[1]` | Parent 2 touched | Σ Vol | `density[2]` | +|---|---|---|---|---|---|---| +| (0, *) | {1,2,3,4} | 70 | `45/70` ≈ `0.6428571` | {3,4,5} | 70 | `55/70` ≈ `0.7857143` | +| (1, *) | {1,2,3,4,5} | 100 | `0.45` | {1,2,3,4,5} | 100 | `0.55` | + +CheckedFeatures derivations (when CF=1): NC=0 → `[0,1,1,2,2,2]` (parent 1 claims features {1,2}; parent 2 claims {3,4} as the larger-volume parent overriding parent 1's earlier claim, plus its own {5}). NC=1 → `[0,2,2,2,2,2]` (both parents touch all 5; parent 2 wins on every feature). Full derivation in `vv/comparisons/ComputeGroupingDensityFilter/README.md` and the v2 archive's `README.md`. + +*Applied (Class 4 — Invariant):* Derivable properties any valid output must satisfy, asserted inline in test code (`ComputeGroupingDensityTest.cpp` Exemplar A/B + Empty-parent edge case): + +- `GroupingDensities[0] == 0.0f` (placeholder parent never touched) +- For `i ≥ 1`: `GroupingDensities[i] > 0.0f ∨ == -1.0f` (positive or sentinel) +- For `i ≥ 1`: `GroupingDensities[i] ≤ 1.0f` (totalCheckVolume always includes the parent's own features → ≥ ParentVolumes[i]) +- `CheckedFeatures[k] ∈ {0, …, numParents-1}` when produced; `CheckedFeatures[0] == 0` +- Empty-parent sentinel asserted directly: `REQUIRE(densities[i] == -1.0f)` when parent `i` has no assigned features + +*Encoded:* + +- **Class 1 (Analytical)**: `test/ComputeGroupingDensityTest.cpp::"Exemplar A/B — all 4 configurations"` — 4 fixtures (`NC0_CF0`, `NC0_CF1`, `NC1_CF0`, `NC1_CF1`). Bit-exact `CompareDataArrays` and `` against the v2 exemplar (`compute_grouping_densities_v2.tar.gz`) whose `GroupingDensities_*` and `CheckedFeatures_*` arrays equal the hand-derivation above to float32 precision. +- **Class 4 (Invariant)**: same test (inline invariant predicates run for all 4 fixtures); plus `test/ComputeGroupingDensityTest.cpp::"Empty-parent edge case (-1.0f sentinel)"` — 2 assertions for `density[1] == 1.0` (parent with assigned features) and `density[2] == -1.0f` (sentinel). + +6 fixture assertions total, all pass at the verified commit. + +*Second-engineer review:* **Skipped — recorded reason:** Class 1 derivation is set-union sums + ratio division on a 5-feature toy dataset (high-school arithmetic). External cross-validation was obtained via the independently-authored legacy `FindGroupingDensity` implementation (`tuks188/DREAM3D` `feature/770_Grouping_Density` sources rebuilt locally): the Phase 9 A/B comparison produced bit-identical agreement across all 4 configurations (see `vv/deviations/ComputeGroupingDensityFilter.md` and `compute_grouping_densities_v2/results/ab_comparison_report.txt`). Any oracle-derivation error would have surfaced as a Deviation. Formal second-engineer review of a 5-feature analytical oracle was not justified given this cross-check. + +## Code path coverage + + + +*7 of 7 paths enumerated — Test case column to be filled by vv-tests.* + +Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.cpp` + +The algorithm dispatches on two booleans, producing 4 template specializations of `FindDensityGrouping()`. Two additional runtime branches handle the empty-parent sentinel and cancellation. Preflight error paths are tested separately at the filter level. + +| Path | Test case | +|---|---| +| `UseNonContiguousNeighbors=true, FindCheckedFeatures=true` (full path, both neighbor lists + per-feature parent tracking) | *(pending)* | +| `UseNonContiguousNeighbors=true, FindCheckedFeatures=false` (both neighbor lists, no per-feature parent tracking) | *(pending)* | +| `UseNonContiguousNeighbors=false, FindCheckedFeatures=true` (contiguous neighbors only + per-feature parent tracking) | *(pending)* | +| `UseNonContiguousNeighbors=false, FindCheckedFeatures=false` (contiguous neighbors only, no per-feature parent tracking) | *(pending)* | +| Edge: `totalFeatureCheckVolume == 0.0f` for a parent → density sentinel `-1.0f` written at line 114 | *(pending)* | +| Cancellation: `m_ShouldCancel` checked inside the parent-id outer loop (line 76); early return without writing further densities | *(pending)* | +| Preflight errors: invalid / mismatched input array paths (3 error tests in `ComputeGroupingDensityTest.cpp` per retroactive notes — confirm count) | *(pending)* | + +## Test inventory + +| Test case | Status | Notes | +|---|---|---| +| *TestName* | kept / new-for-V&V / retired | *one line if needed* | + +## Exemplar archive + +- **Archive:** *``* +- **SHA512:** *``* +- **Provenance:** *`src/Plugins/

/vv/provenance/.md`* + +## Deviations from Pre-SIMPLNX Implementation + +> **Note (this filter only):** the section heading has been re-titled from the template default "Deviations from DREAM3D 6.5.171" because the legacy is the pre-SIMPLNX `FindGroupingDensity` on `tuks188/DREAM3D` `feature/770_Grouping_Density` — not the shipped 6.5.171 baseline. See the Algorithm Relationship section for context. This rename is local to this report only; `docs/vv_templates/report_template.md` is unchanged. + +**No deviations observed.** Runtime A/B comparison run on the +`compute_grouping_densities_v2.tar.gz` fixture: all 4 +`(UseNonContiguousNeighbors, FindCheckedFeatures)` combinations of +`ComputeGroupingDensityFilter` (SIMPLNX) and `FindGroupingDensity` (legacy +DREAM3D 6.5.172 build with the feature-branch sources) produced +**bit-identical** `GroupingDensities` and `CheckedFeatures` output. See +`vv/deviations/ComputeGroupingDensityFilter.md` for the full per-configuration +result table, comparison method, build provenance, and the migration +recommendation for legacy-custom-build customers (*trust SIMPLNX, output is +bit-identical*). + +| Fixture | `compute_grouping_densities_v2.tar.gz` | +|---|---| +| SHA512 | `3aaabb63c4fa16f7fa192ae4ee9dbba9394ec7f1cd19aff55e399a624d495a3a778c7f6f282911f681e85cea99e4c6d15344274e9107f337af7d4a19f93784ff` | +| Driver script | `compute_grouping_densities_v2compare_outputs.py` | diff --git a/src/Plugins/SimplnxCore/vv/deviations/ComputeGroupingDensityFilter.md b/src/Plugins/SimplnxCore/vv/deviations/ComputeGroupingDensityFilter.md new file mode 100644 index 0000000000..2d3542d0de --- /dev/null +++ b/src/Plugins/SimplnxCore/vv/deviations/ComputeGroupingDensityFilter.md @@ -0,0 +1,91 @@ +# Deviations from Pre-SIMPLNX Implementation: ComputeGroupingDensityFilter + +> **Note (this filter only):** the title has been re-scoped from the template +> default "Deviations from DREAM3D 6.5.171." `ComputeGroupingDensityFilter` is +> a port of the pre-SIMPLNX `FindGroupingDensity` (SIMPL UUID +> `708be082-8b08-4db2-94be-52781ed4d53d`) on `tuks188/DREAM3D` +> `feature/770_Grouping_Density` — not the shipped 6.5.171 baseline. +> The legacy filter was never officially released, but a small set of customers +> consumed a custom DREAM3D 6.5.x build that included it. See the V&V report's +> Algorithm Relationship section for context. + +## Headline + +**No deviations observed.** All 4 `(UseNonContiguousNeighbors, FindCheckedFeatures)` +configurations of `ComputeGroupingDensityFilter` (SIMPLNX) and `FindGroupingDensity` +(legacy 6.5.172 build with the feature-branch sources) produced **bit-identical** +`GroupingDensities` and `CheckedFeatures` arrays when run against identical +input data. + +## Comparison method + +| | | +|---|---| +| **Comparison type** | Runtime A/B (not analytical — both implementations actually executed) | +| **Identical inputs** | Same legacy-format `.dream3d` file consumed by both sides (no per-implementation data prep) | +| **Tolerance** | Bit-identical (`np.array_equal` on raw float32 / int32 bytes — stricter than any float epsilon) | +| **Configurations exercised** | All 4 — `(NC, CF) ∈ {(0,0), (0,1), (1,0), (1,1)}` covering every template specialization of `FindDensityGrouping()` | +| **Comparison driver** | `src/Plugins/SimplnxCore/vv/comparisons/ComputeGroupingDensityFilter/compare_outputs.py` | +| **Comparison fixture archive** | `compute_grouping_densities_v2.tar.gz` | +| **Archive SHA512** | `3aaabb63c4fa16f7fa192ae4ee9dbba9394ec7f1cd19aff55e399a624d495a3a778c7f6f282911f681e85cea99e4c6d15344274e9107f337af7d4a19f93784ff` | + +### Inputs + +Hand-built minimal dataset matching `createTestDataStructure()` in +`test/ComputeGroupingDensityTest.cpp` (lines 62-141): + +| Path | Values | +|---|---| +| `DataContainer/FeatureData/Volumes` (Float32, 6) | `[0, 10, 20, 15, 25, 30]` | +| `DataContainer/FeatureData/ParentIds` (Int32, 6) | `[0, 1, 1, 1, 2, 2]` | +| `DataContainer/FeatureData/ContiguousNeighborList` | `[[], [2], [1,3], [2,4], [3,5], [4]]` | +| `DataContainer/FeatureData/NonContiguousNeighborList` | `[[], [4], [5], [], [1], [2]]` | +| `DataContainer/ParentData/ParentVolumes` (Float32, 3) | `[0, 45, 55]` | + +### Per-configuration result + +| (NC, CF) | `GroupingDensities` (both sides, float32) | `CheckedFeatures` (both sides, int32) | Diff | +|---|---|---|---| +| (0, 0) | `[0.0, 0.6428571343421936, 0.7857142686843872]` | array not produced (CF=false) | bit-identical | +| (0, 1) | `[0.0, 0.6428571343421936, 0.7857142686843872]` | `[0, 1, 1, 2, 2, 2]` | bit-identical | +| (1, 0) | `[0.0, 0.44999998807907104, 0.550000011920929]` | array not produced (CF=false) | bit-identical | +| (1, 1) | `[0.0, 0.44999998807907104, 0.550000011920929]` | `[0, 2, 2, 2, 2, 2]` | bit-identical | + +The numerical values match the SIMPLNX unit-test hand calculations exactly +(45/70, 55/70 for the contiguous-only cases; 45/100, 55/100 for the +non-contiguous-included cases — these are the exact float32 representations). + +### Build/source provenance + +| Side | Build / source | +|---|---| +| Legacy 6.5.172 | `/Users/mjackson/DREAM3D-Dev/DREAM3D` (6.5.172 branch + `tuks188/DREAM3D` `feature/770_Grouping_Density` sources pulled in). `FindGroupingDensity.{cpp,h}` placed at `Source/Plugins/Statistics/StatisticsFilters/`. | +| SIMPLNX | `Workspace3/DREAM3D-Build/NX-Com-Qt69-Vtk95-Rel/Bin/nxrunner` (1.7.0 build 2026/05/07). Filter at `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeGroupingDensity.{hpp,cpp}` and `.../Filters/ComputeGroupingDensityFilter.{hpp,cpp}`. | + +## Algorithmic deltas observed (none affect output) + +For audit completeness, the SIMPLNX port made the following structural changes +versus the legacy `execute()` body. The runtime A/B above confirms each is +output-preserving: + +1. **Container swap:** `QVector` totalCheckList (linear `.contains()`, O(n²) per parent) + → `std::unordered_set` (O(1) membership). **No output change** — both have set-membership semantics on the same set of feature ids; floating-point accumulation order is unaffected. +2. **Boolean dispatch:** runtime `if (m_FindCheckedFeatures == true)` inside the inner loop + → compile-time `if constexpr (FindingCheckedFeatures)` template specialization. **No output change.** +3. **Neighbor-list unroll:** runtime `for(k=0; k checkedfeaturevolumes(numfeatures, 0.0f)` always allocated + → conditionally allocated only when `FindCheckedFeatures==true`. **No output change** (legacy zeros were never read when the flag was false). +5. **Cancellation support:** added `m_ShouldCancel` check in the outer parent loop. (Legacy had no cancel support; A/B was run without cancel so this code path is not exercised by the comparison.) +6. **Progress reporting:** added `ThrottledMessenger` per-parent progress. (Legacy emits one terminal "Complete" message; SIMPLNX emits per-parent updates. Affects logs only.) + +Both implementations also behave the same way for the `FindCheckedFeatures=false` +cases — **neither** writes the `CheckedFeatures` output array (it's omitted, not +written-and-empty). + +## Migration recommendation for customers of the legacy custom build + +**Trust SIMPLNX. Output is bit-identical to the legacy filter for matched +inputs across all 4 configurations.** No migration tolerance band is required. +Downstream consumers can expect numerically identical `GroupingDensities` and +`CheckedFeatures` arrays. diff --git a/src/simplnx/DataStructure/AbstractListStore.hpp b/src/simplnx/DataStructure/AbstractListStore.hpp index cae1138742..d3441f766a 100644 --- a/src/simplnx/DataStructure/AbstractListStore.hpp +++ b/src/simplnx/DataStructure/AbstractListStore.hpp @@ -610,14 +610,14 @@ class AbstractListStore : public IListStore virtual vector_type operator[](usize grainId) const = 0; /** - * @brief Returns a const reference to the vector_type value found at the specified index. This cannot be used to edit the vector_type value found at the specified index. + * @brief Returns a copy of the vector_type value found at the specified index. * @param grainId * @return vector_type */ virtual vector_type at(int32 grainId) const = 0; /** - * @brief Returns a const reference to the vector_type value found at the specified index. This cannot be used to edit the vector_type value found at the specified index. + * @brief Returns a copy of the vector_type value found at the specified index. * @param grainId * @return vector_type */ diff --git a/src/simplnx/DataStructure/EmptyListStore.hpp b/src/simplnx/DataStructure/EmptyListStore.hpp index daa88b9a51..75c8829e82 100644 --- a/src/simplnx/DataStructure/EmptyListStore.hpp +++ b/src/simplnx/DataStructure/EmptyListStore.hpp @@ -113,7 +113,17 @@ class EmptyListStore : public AbstractListStore * @brief Returns the total number of lists in the EmptyListStore. * @return uint64 The number of lists (equal to the number of tuples) */ - uint64 getNumberOfLists() const override + usize getNumberOfLists() const override + { + return m_NumTuples; + } + + /** + * @brief Returns the total number of lists in the list store. + * Alias for getNumberOfLists(). + * @return usize The number of lists + */ + usize size() const override { return m_NumTuples; } diff --git a/src/simplnx/DataStructure/IListStore.hpp b/src/simplnx/DataStructure/IListStore.hpp index 93f0596861..f4dc2d0b50 100644 --- a/src/simplnx/DataStructure/IListStore.hpp +++ b/src/simplnx/DataStructure/IListStore.hpp @@ -49,19 +49,16 @@ class IListStore /** * @brief Returns the total number of lists in the list store. - * @return uint64 The number of lists + * @return usize The number of lists */ - virtual uint64 getNumberOfLists() const = 0; + virtual usize getNumberOfLists() const = 0; /** * @brief Returns the total number of lists in the list store. * Alias for getNumberOfLists(). - * @return uint64 The number of lists + * @return usize The number of lists */ - uint64 size() const - { - return getNumberOfLists(); - } + virtual usize size() const = 0; /** * @brief Clears the array. diff --git a/src/simplnx/DataStructure/ListStore.hpp b/src/simplnx/DataStructure/ListStore.hpp index f7631991a5..df98765a23 100644 --- a/src/simplnx/DataStructure/ListStore.hpp +++ b/src/simplnx/DataStructure/ListStore.hpp @@ -155,6 +155,16 @@ class ListStore : public AbstractListStore return copyOfList(grainId); } + /** + * @brief Returns the total number of lists in the list store. + * Alias for getNumberOfLists(). + * @return usize The number of lists + */ + usize size() const override + { + return m_NumTuples; + } + /** * @brief Returns the number of elements in the list at the specified grain/tuple index. * @param grainId The grain/tuple index to query @@ -210,9 +220,9 @@ class ListStore : public AbstractListStore /** * @brief Returns the total number of lists in the ListStore. - * @return uint64 The number of lists (equal to the number of tuples) + * @return usize The number of lists (equal to the number of tuples) */ - uint64 getNumberOfLists() const override + usize getNumberOfLists() const override { return m_NumTuples; }