Add a stable consent hash by Badlapje · Pull Request #1137 · OpenConext/OpenConext-engineblock

Badlapje · 2021-05-19T15:51:05Z

Prior to this commit, there was no stable hashing algorithm used for consent attributes. Casing, order of attributes, ... could all change the hash used to store consent attributes. As such, it couldn't be relied upon.

This commit adds a stable hashing algorithm. It stores all new consent attributes with the new hash, but when retrieving does so checking first the old hash and then the new one.

Edit 4-5-2022 MKodde
After @Badlapje s work was finished on the project, I inherited his work and have finished the work remaining on this ticket.

The consent stabilization feature was implemented in the consent workflows of EB
A database migration was added to allow storing both the unstable and the stable hash.
Tests where added and updated

The QA tests fail on a security check on the JS dependencies. This will be fixed on another PR.

Pivotal ticket: https://www.pivotaltracker.com/story/show/176513931

MKodde

A first review looks very promising! See some improvement suggestions below.

library/EngineBlock/Corto/Model/Consent.php

src/OpenConext/EngineBlock/Service/Consent/ConsentHashService.php

tests/unit/OpenConext/EngineBlock/Service/Consent/ConsentHashServiceTest.php

src/OpenConext/EngineBlock/Service/Consent/ConsentHashService.php

library/EngineBlock/Corto/Model/Consent.php

src/OpenConext/EngineBlockBundle/Authentication/Repository/DbalConsentRepository.php

library/EngineBlock/Corto/Model/Consent.php

database/DoctrineMigrations/Version20220503092351.php

Based on requirements from issue #1721, PR #1137 review threads, and newly added code paths: ConsentIntegrationTest: - test_upgrade_not_applied_when_no_consent_given: upgradeAttributeHashFor does nothing when hasConsentHash returns notGiven() - test_upgrade_continues_gracefully_when_attributes_changed: when updateConsentHash returns false (0 rows, attributes changed since consent was given), upgradeAttributeHashFor must not throw ConsentTest (eb4): - testNullNameIdReturnsNoConsentWithoutCallingRepository: when getNameIdValue() returns null, all three private guards (_storeConsent, _hasStoredConsent, _updateConsent) must return early without touching the repository ConsentVersionTest (new): - isStable/isUnstable/given behaviour for all three states - invalid value throws InvalidArgumentException ConsentHashServiceTest: - test_stable_attribute_hash_attribute_name_casing_normalized: issue #1721 explicitly requires 'Case normalize all attribute names'; verifies keys like 'displayName' and 'DISPLAYNAME' produce the same hash - test_stable_attribute_hash_attribute_name_ordering_normalized: issue #1721 requires 'Sort all attribute names'; verifies key order is stable

kayjoosten · 2026-03-16T09:12:16Z

As we disscused we will make a feature toggle to set the old column to a null value when the new consent hash is set

Based on requirements from issue #1721, PR #1137 review threads, and newly added code paths: ConsentIntegrationTest: - test_upgrade_not_applied_when_no_consent_given: upgradeAttributeHashFor does nothing when hasConsentHash returns notGiven() - test_upgrade_continues_gracefully_when_attributes_changed: when updateConsentHash returns false (0 rows, attributes changed since consent was given), upgradeAttributeHashFor must not throw ConsentTest (eb4): - testNullNameIdReturnsNoConsentWithoutCallingRepository: when getNameIdValue() returns null, all three private guards (_storeConsent, _hasStoredConsent, _updateConsent) must return early without touching the repository ConsentVersionTest (new): - isStable/isUnstable/given behaviour for all three states - invalid value throws InvalidArgumentException ConsentHashServiceTest: - test_stable_attribute_hash_attribute_name_casing_normalized: issue #1721 explicitly requires 'Case normalize all attribute names'; verifies keys like 'displayName' and 'DISPLAYNAME' produce the same hash - test_stable_attribute_hash_attribute_name_ordering_normalized: issue #1721 requires 'Sort all attribute names'; verifies key order is stable

migrations/DoctrineMigrations/Version20260315000001.php

migrations/DoctrineMigrations/Version20260210000000.php

Prior to this change, all consent hashing and persistence logic lived inside the legacy Corto consent model (a God object). This commit extracts a dedicated ConsentHashService that encapsulates both the existing (unstable) hashing algorithm and a new stable hashing algorithm. - Add ConsentHashService with both old and stable hashing methods - Move DB consent queries to the existing DbalConsentRepository, applying naming conventions to query methods - Extract ConsentHashServiceInterface to allow mocking in tests (the concrete class is final) - Wire up service configuration Pivotal ticket: https://www.pivotaltracker.com/story/show/176513931

The old consent hash was sensitive to attribute ordering, case changes, stray empty attributes, and sparse array indexes — causing spurious re-consent prompts. This commit introduces a stable hashing algorithm alongside the existing one, with a migration path that avoids invalidating all existing consent records at once. Key changes: - Add `attribute_stable` VARCHAR(80) nullable column to `consent` table; make existing `attribute` column nullable for future cleanup - Add ConsentVersion value object with three states: stable, unstable, not-given (replaces boolean return from consent-check methods) - Retrieve old-style and new-style attribute hash in a single query instead of two sequential lookups - When existing consent matches only the unstable hash, silently upgrade by writing the stable hash (no re-consent prompt shown) - Newly given consent is stored with the stable hash - Normalize NameID objects before hashing to prevent serialization errors - Only fetch stored consent when consent is enabled (short-circuit optimization) The migration strategy (from the Pivotal story): - Existing consent hashing is left untouched - Consent lookup checks both old and new hash: match on either means consent was given - New consent is stored with the new stable hash - Old consent that matches unstable hash is upgraded in-place Pivotal ticket: https://www.pivotaltracker.com/story/show/176513931

Introduces the `feature_stable_consent_hash_migration` toggle (default: false) to control when the legacy `attribute` column is phased out, and adds edge-case test coverage. Toggle OFF (default): new consent writes both `attribute` and `attribute_stable` so old EB instances still find records during a rolling deploy; hash upgrades leave `attribute` intact. Toggle ON: new consent writes only `attribute_stable` (attribute=NULL) and upgrades null the old column, cleaning it up over time once all instances run the new version. Also: - Add edge-case tests for consent hash versioning - Fix test configuration for consent scenarios - Drop redundant unserialize/serialize deep copies in ConsentHashService (PHP passes arrays by value; a mutation-safety regression test is added to guard against future changes)

* refactor: convert ConsentType to a PHP 8.1 backed enum Replace the ConsentType value class (with its deprecated public constructor since ~2015) with a proper backed enum. - ConsentType is now `enum ConsentType: string` with cases Explicit = 'explicit' and Implicit = 'implicit' - The deprecated public constructor, the explicit()/implicit() named constructors and the equals() method are removed - DbalConsentRepository uses ConsentType::from() to hydrate from DB - Legacy Consent model methods now accept ConsentType and pass ->value into SQL parameter arrays - All call sites updated from TYPE_EXPLICIT/TYPE_IMPLICIT constants and named constructors to enum cases - ConsentTypeTest updated: invalid-string rejection tested via from(), equality via assertSame() on enum singletons

…eanup Encapsulate stable consent hash normalisation in the ConsentAttributes value object. The raw-array + boolean flag signature of getStableAttributesHash scattered the "which strategy" decision across every call boundary. ConsentAttributes with named constructors (withValues / namesOnly) makes the strategy choice once at the call site, removes the boolean from downstream signatures, and centralises all normalisation logic with a single testable output (getCompareValue). Also in this commit: - Use Symfony DI for Consent factory instead of relying on the old Corto service provider singleton (EngineBlock_ApplicationSingleton) - Make empty-attribute stripping explicit: rename sortArrayRecursive to sortRecursive (sort-only), make removeEmptyAttributes recursive so nested empty sub-values are stripped without relying on the sort pass - Add tests pinning the empty-attribute stripping policy: * Attribute going from having values to empty array -> different hash (triggers re-consent: SP was receiving data, now isn't) * Stray empty string in value list -> stripped, same hash as without it (no spurious re-consent) * Zero values (0, 0.0, "0") -> preserved, not stripped - Remove unused parameters and properties - Add CHANGELOG entry for feature_stable_consent_hash_migration - Annotate deprecated code for removal after next release - Expand unit test coverage for consent hashing

johanib · 2026-03-30T12:59:00Z

@kayjoosten @MKodde
I did some checks, cleanup up the code, and the commits. I think this now is ✔️

Can you do a review?

kayjoosten · 2026-03-30T14:09:06Z

src/OpenConext/EngineBlock/Authentication/Value/ConsentVersion.php

+
+use OpenConext\EngineBlock\Assert\Assertion;
+
+final class ConsentVersion


note: this could have been a enum. Don't you agree?

kayjoosten · 2026-03-30T14:12:43Z

migrations/DoctrineMigrations/Version20260315000001.php

+        $this->addSql('ALTER TABLE consent ADD attribute_stable VARCHAR(80) DEFAULT NULL, CHANGE attribute attribute VARCHAR(80) DEFAULT NULL');
+    }
+
+    public function down(Schema $schema): void


Maybe we should remove the down. This looks like trouble when both cols are null

Badlapje added enhancement consent labels May 19, 2021

Badlapje requested a review from MKodde May 19, 2021 15:51

Badlapje changed the title ~~Add a consentHashService~~ Add a stable consent hash May 19, 2021

MKodde requested changes May 20, 2021

View reviewed changes

Badlapje force-pushed the feature/stabilize-consent-hash branch 6 times, most recently from 4e3a85e to dff4b53 Compare May 20, 2021 16:46

Badlapje requested a review from MKodde June 1, 2021 17:07

Badlapje force-pushed the feature/stabilize-consent-hash branch from dff4b53 to 595c767 Compare June 2, 2021 12:38

BackEndTea reviewed Jun 7, 2021

View reviewed changes

src/OpenConext/EngineBlock/Service/Consent/ConsentHashService.php Outdated Show resolved Hide resolved

MKodde force-pushed the feature/stabilize-consent-hash branch 8 times, most recently from c716bf8 to a5dd163 Compare May 4, 2022 10:09

MKodde requested review from VadimSchmitz and thijskh May 4, 2022 10:28

thijskh reviewed May 4, 2022

View reviewed changes

MKodde force-pushed the feature/stabilize-consent-hash branch from 26e8ac1 to 145ebd9 Compare May 5, 2022 11:24

MKodde added 6.8 and removed 6.7 labels May 5, 2022

thijskh removed the 6.8 label Oct 25, 2023

kayjoosten mentioned this pull request Mar 6, 2026

Remove obsolete Consent hashing code #1930

Open

kayjoosten force-pushed the feature/stabilize-consent-hash branch from d30ca90 to b183015 Compare March 9, 2026 09:55

kayjoosten force-pushed the feature/stabilize-consent-hash branch from b183015 to c3faaec Compare March 9, 2026 15:01

kayjoosten force-pushed the feature/stabilize-consent-hash branch 7 times, most recently from b5009cb to 260c436 Compare March 17, 2026 13:48

kayjoosten requested a review from johanib March 23, 2026 09:09

johanib reviewed Mar 26, 2026

View reviewed changes

migrations/DoctrineMigrations/Version20260315000001.php Outdated Show resolved Hide resolved

johanib reviewed Mar 30, 2026

View reviewed changes

migrations/DoctrineMigrations/Version20260210000000.php Outdated Show resolved Hide resolved

johanib force-pushed the feature/stabilize-consent-hash branch from df2737c to 868562f Compare March 30, 2026 12:14

Koen Cornelis and others added 4 commits March 30, 2026 14:46

johanib force-pushed the feature/stabilize-consent-hash branch from 868562f to e52f282 Compare March 30, 2026 12:51

johanib force-pushed the feature/stabilize-consent-hash branch from e52f282 to ed2bcf2 Compare March 30, 2026 12:56

johanib requested review from MKodde and kayjoosten March 30, 2026 12:57

kayjoosten reviewed Mar 30, 2026

View reviewed changes

johanib requested a review from baszoetekouw March 31, 2026 08:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a stable consent hash#1137

Add a stable consent hash#1137
Badlapje wants to merge 5 commits intomainfrom
feature/stabilize-consent-hash

Badlapje commented May 19, 2021 •

edited by MKodde

Loading

Uh oh!

MKodde left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kayjoosten commented Mar 16, 2026

Uh oh!

Uh oh!

Uh oh!

johanib commented Mar 30, 2026

Uh oh!

kayjoosten Mar 30, 2026

Uh oh!

kayjoosten Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants


		use OpenConext\EngineBlock\Assert\Assertion;

		final class ConsentVersion

Conversation

Badlapje commented May 19, 2021 • edited by MKodde Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MKodde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kayjoosten commented Mar 16, 2026

Uh oh!

Uh oh!

Uh oh!

johanib commented Mar 30, 2026

Uh oh!

kayjoosten Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

kayjoosten Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Badlapje commented May 19, 2021 •

edited by MKodde

Loading