Allow counters to be created with same name, provider and source as a deleted one #10223

Pearl1594 · 2025-01-21T02:14:27Z

Description

This PR fixes: #10043

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

…eleted one

codecov · 2025-01-21T02:21:47Z

Codecov Report

❌ Patch coverage is 12.00000% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.94%. Comparing base (b99a030) to head (087244d).
⚠️ Report is 25 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/com/cloud/network/as/dao/CounterDaoImpl.java	0.00%	12 Missing ⚠️
.../api/command/admin/autoscale/CreateCounterCmd.java	0.00%	5 Missing ⚠️
...ava/com/cloud/network/as/AutoScaleManagerImpl.java	37.50%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10223      +/-   ##
============================================
+ Coverage     17.56%   17.94%   +0.38%     
- Complexity    15500    15900     +400     
============================================
  Files          5899     5908       +9     
  Lines        527793   540565   +12772     
  Branches      64479    69702    +5223     
============================================
+ Hits          92714    97026    +4312     
- Misses       424653   432806    +8153     
- Partials      10426    10733     +307

Flag	Coverage Δ
uitests	`3.81% <ø> (+0.21%)`	⬆️
unittests	`19.03% <12.00%> (+0.40%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Pearl1594 · 2025-01-21T04:32:08Z

@blueorangutan package

blueorangutan · 2025-01-21T04:34:04Z

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-01-21T05:24:39Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12137

rohityadavcloud · 2025-01-21T06:27:46Z

@blueorangutan test

blueorangutan · 2025-01-21T06:30:04Z

@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

DaanHoogland

one comment,

Also the issue reported that deleteCounter doesn't work. Is that genuine?

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql

blueorangutan · 2025-01-21T20:01:04Z

[SF] Trillian test result (tid-12142)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 46204 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10223-t12142-kvm-ol8.zip
Smoke tests completed. 133 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql

github-actions · 2025-01-30T16:22:16Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql

…20.sql

Pearl1594 · 2025-02-25T15:40:48Z

@blueorangutan package

weizhouapache · 2025-02-25T15:49:48Z

@Pearl1594
with this PR, can user create a counter with same name, provider and source as non-deleted one ?
(I think it should fail)

Pearl1594 · 2025-02-25T17:04:27Z

@Pearl1594 with this PR, can user create a counter with same name, provider and source as non-deleted one ? (I think it should fail)

@weizhouapache It allows to create a counter with same name provider and source - as long as the value is different. Is that not expected?

github-actions · 2025-09-25T08:09:20Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…create-counters-dup

Pearl1594 · 2025-10-10T15:04:39Z

@blueorangutan package

blueorangutan · 2025-10-10T15:06:04Z

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-10-10T17:13:32Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15406

DaanHoogland · 2025-10-13T06:56:11Z

@blueorangutan test

blueorangutan · 2025-10-13T06:58:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-13T08:18:15Z

[SF] Trillian Build Failed (tid-14641)

weizhouapache · 2025-10-13T08:24:48Z

@blueorangutan test

blueorangutan · 2025-10-13T08:26:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-14T01:52:00Z

[SF] Trillian test result (tid-14643)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 59159 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10223-t14643-kvm-ol8.zip
Smoke tests completed. 148 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_redundant_vpc_site2site_vpn	`Failure`	478.59	test_vpc_vpn.py

Copilot

Pull Request Overview

This PR fixes a bug that prevented counters from being created with the same name, provider and source as a previously deleted counter. The fix involves modifying the database unique constraint to include the "removed" field and updating the counter creation logic to allow recreation of counters with matching properties when the original has been soft-deleted.

Key Changes:

Modified database schema to change unique constraint from (provider, source, value) to (provider, source, value, removed)
Updated counter creation validation to check for existing non-deleted counters only
Refactored counter creation flow to separate creation and response generation

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
AutoScaleManagerImpl.java	Added validation logic and separated counter creation from response generation
schema-42100to42200.sql	Modified database unique constraint to include 'removed' field
cloud.idempotent_drop_unique_key.sql	Added new stored procedure for safely dropping unique keys
CounterDaoImpl.java	Added method to find counters by name, provider and value
CounterDao.java	Added interface method for the new search functionality
CreateCounterCmd.java	Refactored to separate creation and execution phases
AutoScaleService.java	Added getCounter method to service interface

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-14T08:05:55Z

server/src/main/java/com/cloud/network/as/AutoScaleManagerImpl.java

+    @Override
+    @ActionEvent(eventType = EventTypes.EVENT_COUNTER_CREATE, eventDescription = "Creating a counter", async = true)
+    public Counter getCounter(long counterId) {
+        return counterDao.findById(counterId);
+    }


The getCounter method has incorrect event annotation. It should use EVENT_COUNTER_GET or similar since this is a read operation, not a create operation.

Copilot · 2025-10-14T08:05:56Z

server/src/main/java/com/cloud/network/as/AutoScaleManagerImpl.java


+        CounterVO existingCounter = counterDao.findByNameProviderValue(name, value, provider.getName());
+        if (existingCounter != null) {
+            throw new InvalidParameterValueException(String.format("Counter with name %s and value %s already exists. ", name,value));


Missing space before 'value' parameter in the error message format string.

Suggested change

throw new InvalidParameterValueException(String.format("Counter with name %s and value %s already exists. ", name,value));

throw new InvalidParameterValueException(String.format("Counter with name %s and value %s already exists. ", name, value));

rosi-shapeblue

LGTM. The PR fixes the initial bug and introduces correct behavior for counter re-creation.

Verification Checklist

Pre-Fix Behavior – re-creation after deletion not possible FAIL (expected)
Create counter PASS
Delete counter PASS
Recreate same counter after deletion PASS
Duplicate creation with active value blocked PASS
Same name with different value works PASS
Verify listing shows multiple counters PASS
Delete one and recreate again PASS
DB entries properly marked as removed PASS

Detailed Test Results

Pre-Fix Behavior - re-creation after deletion not possible.

(localcloud) 🐱 > list counters name=Linux System CPU - percentage
{
  "count": 1,
  "counter": [
    {
      "id": "4e450db5-a8ba-11f0-86a0-1e00f00003ce",
      "name": "Linux User CPU - percentage - native",
      "provider": "Netscaler",
      "source": "CPU",
      "value": "1.3.6.1.4.1.2021.11.9.1"
    }
  ]
}
(localcloud) 🐱 >
(localcloud) 🐱 > delete counter id=4e450db5-a8ba-11f0-86a0-1e00f00003ce
{
  "success": true
}
(localcloud) 🐱 > list counters name=Linux System CPU - percentage
(localcloud) 🐱 >
(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.10.0"
🙈 Error: (HTTP 530, error code 9999) Entity already exists 
(localcloud) 🐱 >

Verification with Fix

1. Create counter - counter can be created

(localcloud) 🐱 > list counters name=Linux System CPU - percentage
(localcloud) 🐱 >
(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.10.0"
{
  "counter": {
    "id": "20faa221-c51b-43be-96b2-b4fe8037b561",
    "name": "Linux System CPU - percentage",
    "provider": "Netscaler",
    "source": "SNMP",
    "value": "1.3.6.1.4.1.2021.11.10.0"
  }
}
(localcloud) 🐱 > list counters name=Linux System CPU - percentage
{
  "count": 1,
  "counter": [
    {
      "id": "20faa221-c51b-43be-96b2-b4fe8037b561",
      "name": "Linux System CPU - percentage",
      "provider": "Netscaler",
      "source": "SNMP",
      "value": "1.3.6.1.4.1.2021.11.10.0"
    }
  ]
}

2. Delete counter - counter is deleted

(localcloud) 🐱 > delete counter id=20faa221-c51b-43be-96b2-b4fe8037b561
{
  "success": true
}
(localcloud) 🐱 > list counters name=Linux System CPU - percentage
(localcloud) 🐱 >

3. Recreate same counter after delete

(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.10.0"
{
  "counter": {
    "id": "83209340-7e4b-4edc-9791-3d5d24920e85",
    "name": "Linux System CPU - percentage",
    "provider": "Netscaler",
    "source": "SNMP",
    "value": "1.3.6.1.4.1.2021.11.10.0"
  }
}

4. Try duplicate creation - fails

(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.10.0"
🙈 Error: (HTTP 431, error code 4350) Counter with name Linux System CPU - percentage and value 1.3.6.1.4.1.2021.11.10.0 already exists.

5. Create same name with different value - succeeds

(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.12.0"
{
  "counter": {
    "id": "8a96b886-c533-4abb-b491-630014964947",
    "name": "Linux System CPU - percentage",
    "provider": "Netscaler",
    "source": "SNMP",
    "value": "1.3.6.1.4.1.2021.11.12.0"
  }
}

6. Verify listing

(localcloud) 🐱 > list counters name=Linux System CPU - percentage
{
  "count": 2,
  "counter": [
    {
      "id": "8a96b886-c533-4abb-b491-630014964947",
      "name": "Linux System CPU - percentage",
      "provider": "Netscaler",
      "source": "SNMP",
      "value": "1.3.6.1.4.1.2021.11.12.0"
    },
    {
      "id": "83209340-7e4b-4edc-9791-3d5d24920e85",
      "name": "Linux System CPU - percentage",
      "provider": "Netscaler",
      "source": "SNMP",
      "value": "1.3.6.1.4.1.2021.11.10.0"
    }
  ]
}

7. Delete one and recreate it again - recreation succeeds

(localcloud) 🐱 > delete counter id=8a96b886-c533-4abb-b491-630014964947
{
  "success": true
}

(localcloud) 🐱 > create counter name="Linux System CPU - percentage" provider="Netscaler" source="SNMP" value="1.3.6.1.4.1.2021.11.12.0"
{
  "counter": {
    "id": "b5cc257e-9958-4e6a-ad23-66a93936d7a4",
    "name": "Linux System CPU - percentage",
    "provider": "Netscaler",
    "source": "SNMP",
    "value": "1.3.6.1.4.1.2021.11.12.0"
  }
}

8. DB Verification Deleted counters have removed timestamps.; Active counters have NULL in removed.

mysql> select id,name,value,removed from cloud.counter where name='Linux System CPU - percentage';
+-----+-------------------------------+--------------------------+---------------------+
| id  | name                          | value                    | removed             |
+-----+-------------------------------+--------------------------+---------------------+
|   2 | Linux System CPU - percentage | 1.3.6.1.4.1.2021.11.10.0 | 2025-10-14 13:11:08 |
| 121 | Linux System CPU - percentage | 1.3.6.1.4.1.2021.11.10.0 | 2025-10-14 13:52:41 |
| 122 | Linux System CPU - percentage | 1.3.6.1.4.1.2021.11.10.0 | NULL                |
| 123 | Linux System CPU - percentage | 1.3.6.1.4.1.2021.11.12.0 | 2025-10-14 13:54:35 |
| 124 | Linux System CPU - percentage | 1.3.6.1.4.1.2021.11.12.0 | NULL                |
+-----+-------------------------------+--------------------------+---------------------+
5 rows in set (0.00 sec)

sonarqubecloud · 2025-10-15T10:03:04Z

Quality Gate failed

Failed conditions
20.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube Cloud

… deleted one (apache#10223)

Allow counters to be created with same name, provider and source as d…

294696d

…eleted one

Pearl1594 requested review from DaanHoogland and weizhouapache January 21, 2025 02:14

fix api response

fdb9a71

boring-cyborg bot added component:api component:networking labels Jan 21, 2025

DaanHoogland reviewed Jan 21, 2025

View reviewed changes

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql Outdated Show resolved Hide resolved

drop unique key only if it exists

7a1e54d

Pearl1594 requested a review from DaanHoogland January 22, 2025 14:34

weizhouapache reviewed Jan 22, 2025

View reviewed changes

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql Outdated Show resolved Hide resolved

create a procedure

2ace4dd

github-actions bot added the status:has-conflicts label Jan 30, 2025

Merge branch '4.19' into fix-create-counters-dup

96691f6

github-actions bot removed the status:has-conflicts label Jan 30, 2025

DaanHoogland reviewed Jan 30, 2025

View reviewed changes

engine/schema/src/main/resources/META-INF/db/schema-41910to41920.sql Outdated Show resolved Hide resolved

Update engine/schema/src/main/resources/META-INF/db/schema-41910to419…

6b5ece3

…20.sql

DaanHoogland added this to the 4.19.3 milestone Feb 13, 2025

Pearl1594 requested review from DaanHoogland and weizhouapache February 25, 2025 15:40

Merge branch 'main' of https://github.com/apache/cloudstack into fix-…

087244d

…create-counters-dup

github-actions bot removed the status:has-conflicts label Oct 10, 2025

DaanHoogland added the status:needs-testing label Oct 14, 2025

vishesh92 requested a review from Copilot October 14, 2025 08:05

Copilot AI reviewed Oct 14, 2025

View reviewed changes

vishesh92 self-assigned this Oct 14, 2025

DaanHoogland closed this Oct 14, 2025

DaanHoogland reopened this Oct 14, 2025

rosi-shapeblue self-assigned this Oct 14, 2025

vishesh92 removed their assignment Oct 14, 2025

rosi-shapeblue added status:testing and removed status:needs-testing labels Oct 14, 2025

rosi-shapeblue approved these changes Oct 14, 2025

View reviewed changes

rosi-shapeblue removed the status:testing label Oct 14, 2025

DaanHoogland closed this Oct 15, 2025

DaanHoogland reopened this Oct 15, 2025

DaanHoogland merged commit 0e8b0b8 into main Oct 15, 2025
110 of 118 checks passed

DaanHoogland deleted the fix-create-counters-dup branch October 15, 2025 11:06

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Oct 17, 2025

Allow counters to be created with same name, provider and source as a…

24d9dd4

… deleted one (apache#10223)

	throw new InvalidParameterValueException(String.format("Counter with name %s and value %s already exists. ", name,value));
	throw new InvalidParameterValueException(String.format("Counter with name %s and value %s already exists. ", name, value));

Allow counters to be created with same name, provider and source as a deleted one #10223

Allow counters to be created with same name, provider and source as a deleted one #10223

Uh oh!

Conversation

Pearl1594 commented Jan 21, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Jan 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Pearl1594 commented Jan 21, 2025

Uh oh!

blueorangutan commented Jan 21, 2025

Uh oh!

blueorangutan commented Jan 21, 2025

Uh oh!

rohityadavcloud commented Jan 21, 2025

Uh oh!

blueorangutan commented Jan 21, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

blueorangutan commented Jan 21, 2025

Uh oh!

Uh oh!

github-actions bot commented Jan 30, 2025

Uh oh!

Uh oh!

Pearl1594 commented Feb 25, 2025

Uh oh!

weizhouapache commented Feb 25, 2025

Uh oh!

Pearl1594 commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

Pearl1594 commented Oct 10, 2025

Uh oh!

blueorangutan commented Oct 10, 2025

Uh oh!

blueorangutan commented Oct 10, 2025

Uh oh!

DaanHoogland commented Oct 13, 2025

Uh oh!

blueorangutan commented Oct 13, 2025

Uh oh!

blueorangutan commented Oct 13, 2025

Uh oh!

weizhouapache commented Oct 13, 2025

Uh oh!

blueorangutan commented Oct 13, 2025

Uh oh!

blueorangutan commented Oct 14, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes:

Reviewed Changes

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

rosi-shapeblue left a comment

Choose a reason for hiding this comment

Detailed Test Results

Uh oh!

codecov bot commented Jan 21, 2025 •

edited

Loading

Pearl1594 commented Feb 25, 2025 •

edited

Loading