Fix: Implement concurrency control for custom property updates in TypeRepository

[aniketkatkar97]

Before:

After:

This pull request introduces thread-safety improvements for adding custom properties to types and adds a corresponding concurrent test to ensure data integrity under concurrent operations. The main focus is to prevent race conditions when multiple threads attempt to add custom properties to the same type simultaneously.

Concurrency control for custom property updates:

• Introduced a ConcurrentHashMap (TYPE_PROPERTY_LOCKS) in TypeRepository.java to provide per-type locking, ensuring that concurrent additions of custom properties to the same type are synchronized and do not cause data loss or corruption. [1] [2]
• Updated the addCustomProperty method in TypeRepository.java to acquire a lock specific to the type's UUID before modifying its custom properties, preventing race conditions during concurrent updates. [1] [2]

Testing and validation:

• Added a new test put_concurrent_customProperty_sameType_allPersisted in TypeResourceTest.java that uses multiple threads to concurrently add custom properties to the same type and verifies that all properties are persisted without errors or data loss. [1] [2]

[gitar-bot]

💡 Quality: Lock map entries not cleaned up on type deletion

The TYPE_PROPERTY_LOCKS ConcurrentHashMap accumulates entries for every type UUID that has addCustomProperty called on it, but entries are never removed — not even when a type is deleted via postDelete. While the number of types in practice is small (O(100s)), this is a minor resource leak that goes against the principle of cleaning up resources.

The postDelete method already exists and handles TypeRegistry cleanup, making it the natural place to also clean up the lock entry.

Suggested fix:

  @Override
  protected void postDelete(Type entity, boolean hardDelete) {
    super.postDelete(entity, hardDelete);
    TypeRegistry.instance().removeType(entity.getName());
    TYPE_PROPERTY_LOCKS.remove(entity.getId());
  }

_{Was this helpful? React with 👍 / 👎}

[gitar-bot]

💡 Edge Case: JVM-level lock won't protect multi-instance deployments

The synchronized block with a ConcurrentHashMap-based lock only provides thread-safety within a single JVM. If OpenMetadata is deployed with multiple application instances (e.g., behind a load balancer), concurrent addCustomProperty requests routed to different instances will still race without coordination.

For single-instance deployments, this fix is correct and sufficient. For multi-instance scenarios, a database-level optimistic locking strategy (e.g., using the entity version/ETag) or a distributed lock would be needed. This may be worth documenting as a known limitation in a code comment or tracking as a follow-up.

_{Was this helpful? React with 👍 / 👎}

[gitar-bot]

🔍 CI failure analysis for 51b0526: All CI failures are unrelated to this PR's TypeRepository changes. New failures: 6 consistent DataProductResourceTest failures (bulk operations, migration, domain assignment) in both maven-sonarcloud-ci and Test Report. Other recurring failures include GitHub Actions auth, OpenSearch connection pools, and Python pytest segfault.

Issue

Multiple CI jobs failed with infrastructure and test issues unrelated to this PR's changes:

• py-checkstyle: GitHub Actions authentication error (previous runs)
• Integration Tests (PostgreSQL + OpenSearch): Connection pool shutdown errors (previous runs)
• Python Integration Tests: pytest process crashed with exit code -11 (previous run)
• maven-sonarcloud-ci: 6 test failures in DataProductResourceTest
• Test Report: 6 test failures in DataProductResourceTest (same failures)

Root Cause

This PR modifies only:

• TypeRepository.java - Added per-type locking for concurrent custom property updates
• TypeResourceIT.java - Added concurrent test for custom property additions

All failures are unrelated to these Type concurrency control changes:

1. Previous Failures (still applicable)

• py-checkstyle: HTTP 401 - GitHub Actions authentication issue
• PostgreSQL + OpenSearch integration tests: Connection pool shutdown errors
• Python integration tests: pytest segmentation fault (exit code -11)

2. maven-sonarcloud-ci Failure (job 64019652202)

6 failures in DataProductResourceTest.java:

• Line 850: testDataProductBulkOutputPorts - expected: <success> but was: <failure>
• Lines 1108, 1066, 1012: testUpdate_dataProduct_200 - Expected count mismatches (expected 1, got 0)
• Line 1732: testDataProductMigrationOutputPortWithDomain - Output port domain mismatch
• Line 904: testDataProductBulkOutputPorts - HTTP 400 error reading response

Result: 7948 tests run, 6 failures, 701 skipped

3. Test Report Failure (job 64045278814)

Same 6 failures in DataProductResourceTest - Identical to maven-sonarcloud-ci failures

Details

All test failures involve DataProduct functionality completely separate from Type metadata:

• Data Product bulk output port operations
• Data Product update assertions
• Data Product migration with domain assignment
• None of these test files or functionality areas touched by PR changes

Pattern: The same 6 DataProductResourceTest failures appear consistently in both maven-sonarcloud-ci and Test Report jobs. These are failures in Data Product functionality, completely separate from the Type metadata repository concurrency control changes introduced by this PR.

The failures suggest:

• Data Product bulk operation logic issues
• Data Product migration logic flakiness
• Data Product domain assignment issues
• These are pre-existing issues in Data Product functionality, not related to Type metadata changes

Summary

All CI failures are unrelated to this PR's Java TypeRepository concurrency control changes:

1. py-checkstyle: GitHub Actions authentication configuration issue
2. PostgreSQL + OpenSearch integration tests: Infrastructure connection pool issues
3. Python integration tests: pytest process segmentation fault
4. maven-sonarcloud-ci + Test Report: 6 consistent failures in DataProductResourceTest (bulk operations, migration, domain assignment)

None of these failures can be addressed by changes to this PR's TypeRepository code. They represent pre-existing issues in other areas of the codebase and CI infrastructure.

Code Review 👍 Approved with suggestions 0 resolved / 2 findings

Solid implementation of per-type JVM-level locking to fix the read-modify-write race condition. Two prior minor findings remain: lock map entries aren't cleaned up on type deletion, and the JVM-level lock doesn't protect multi-instance deployments — both are acknowledged limitations with low practical impact.

💡 Quality: Lock map entries not cleaned up on type deletion

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TypeRepository.java:68-69 📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TypeRepository.java:148-151

The TYPE_PROPERTY_LOCKS ConcurrentHashMap accumulates entries for every type UUID that has addCustomProperty called on it, but entries are never removed — not even when a type is deleted via postDelete. While the number of types in practice is small (O(100s)), this is a minor resource leak that goes against the principle of cleaning up resources.

The postDelete method already exists and handles TypeRegistry cleanup, making it the natural place to also clean up the lock entry.

Suggested fix

  @Override
  protected void postDelete(Type entity, boolean hardDelete) {
    super.postDelete(entity, hardDelete);
    TypeRegistry.instance().removeType(entity.getName());
    TYPE_PROPERTY_LOCKS.remove(entity.getId());
  }

💡 Edge Case: JVM-level lock won't protect multi-instance deployments

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TypeRepository.java:168-169

The synchronized block with a ConcurrentHashMap-based lock only provides thread-safety within a single JVM. If OpenMetadata is deployed with multiple application instances (e.g., behind a load balancer), concurrent addCustomProperty requests routed to different instances will still race without coordination.

For single-instance deployments, this fix is correct and sufficient. For multi-instance scenarios, a database-level optimistic locking strategy (e.g., using the entity version/ETag) or a distributed lock would be needed. This may be worth documenting as a known limitation in a code comment or tracking as a follow-up.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud