Skip to content

Conversation

@slavkap
Copy link
Contributor

@slavkap slavkap commented Aug 20, 2025

Description

This PR fixes:
regression from #11452

  • When a snapshot has a copy on StorPool primary storage in another zone, but the original snapshot resides on secondary storage, creating a template from the copied snapshot results in the template being created in the first zone.
  • For NFS/Ceph - if the snapshot.backup.to.secondary setting is disabled, and a user creates a volume or template from a snapshot, the snapshot is temporarily backed up to secondary storage during the operation. After the operation, this backup should be deleted. However, the snapshot currently remains on both primary and secondary storage.
  • When a StorPool's snapshot has a backup to a secondary storage, the user can create a volume from that snapshot, but it will be empty. This is caused by Fix for create template from snapshot (for snapshots on primary storage and storage doesn't support create snapshot to template directly) #11452 - it sends a different snapshot object to the StorPool plug-in, and StorPool doesn't know which is the parent snapshot.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

manually tested and still testing it

When a snapshot has a copy on StorPool primary storage in another zone, but the original snapshot resides on secondary storage, creating a template from the copied snapshot results in the template being created in the first zone.
If the snapshot.backup.to.secondary setting is disabled, and a user creates a volume or template from a snapshot, the snapshot is temporarily backed up to secondary storage during the operation. After the operation, this backup should be deleted. However, the snapshot currently remains on both primary and secondary storage.
@nvazquez
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov
Copy link

codecov bot commented Aug 20, 2025

Codecov Report

❌ Patch coverage is 0% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.36%. Comparing base (f2d6356) to head (e01a03d).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...n/java/com/cloud/template/TemplateManagerImpl.java 0.00% 12 Missing ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java 0.00% 7 Missing ⚠️
...org/apache/cloudstack/snapshot/SnapshotHelper.java 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               main   #11490   +/-   ##
=========================================
  Coverage     17.35%   17.36%           
- Complexity    15230    15234    +4     
=========================================
  Files          5886     5886           
  Lines        525685   525680    -5     
  Branches      64159    64159           
=========================================
+ Hits          91247    91261   +14     
+ Misses       424138   424120   -18     
+ Partials      10300    10299    -1     
Flag Coverage Δ
uitests 3.63% <ø> (+<0.01%) ⬆️
unittests 18.40% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14681

@sureshanaparti
Copy link
Contributor

Hi @rp- Can you test this PR with linstor and share your results. Thanks.

@sureshanaparti
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-14089)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50462 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11490-t14089-kvm-ol8.zip
Smoke tests completed. 146 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

Copy link
Contributor

@rp- rp- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create template from snapshot worked, BUT
create volume from snapshot did NOT.

with Linstor

@rp-
Copy link
Contributor

rp- commented Aug 21, 2025

2025-08-21 06:56:18,117 DEBUG [c.c.a.ApiServer] (qtp1438988851-18:[ctx-7efce117, ctx-b0128148]) (logid:619c3a5e) Retrieved cmdEventType from job info: VOLUME.CREATE
2025-08-21 06:56:18,118 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-82:[ctx-aa54f249, job-170]) (logid:1c225a39) Add job-170 into job monitoring
2025-08-21 06:56:18,119 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (qtp1438988851-18:[ctx-7efce117, ctx-b0128148]) (logid:619c3a5e) submit async job-170, details: AsyncJob {"accountId":2,"cmd":"org.apache.cloudstack.api.command.admin.volume.CreateVolumeCmdByAdmin","cmdInfo":"{\"snapshotid\":\"ffd6b8a4-18ec-4e0c-b437-f41353a49d42\",\"sessionkey\":\"6m0yzvxJKeK7qzZvnst3VAYZZSo\",\"httpmethod\":\"POST\",\"ctxAccountId\":\"2\",\"uuid\":\"7b333b33-4c91-42f3-bd0b-6455d8b75360\",\"domainid\":\"563b4c50-7dfd-11f0-a9b9-525400000078\",\"cmdEventType\":\"VOLUME.CREATE\",\"response\":\"json\",\"ctxUserId\":\"2\",\"name\":\"fsdf\",\"zoneid\":\"45e225f9-894b-4e0f-8aef-f576f117052f\",\"ctxStartEventId\":\"393\",\"id\":\"18\",\"ctxDetails\":\"{\\\"interface com.cloud.storage.Snapshot\\\":\\\"ffd6b8a4-18ec-4e0c-b437-f41353a49d42\\\",\\\"interface com.cloud.dc.DataCenter\\\":\\\"45e225f9-894b-4e0f-8aef-f576f117052f\\\",\\\"interface com.cloud.domain.Domain\\\":\\\"563b4c50-7dfd-11f0-a9b9-525400000078\\\",\\\"interface com.cloud.storage.Volume\\\":\\\"7b333b33-4c91-42f3-bd0b-6455d8b75360\\\"}\",\"account\":\"admin\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":170,"initMsid":90520730730616,"instanceId":18,"instanceType":"Volume","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"641fbb02-a925-444a-b49b-b75a27e35705"}
2025-08-21 06:56:18,119 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-82:[ctx-aa54f249, job-170]) (logid:641fbb02) Executing AsyncJob {"accountId":2,"cmd":"org.apache.cloudstack.api.command.admin.volume.CreateVolumeCmdByAdmin","cmdInfo":"{\"snapshotid\":\"ffd6b8a4-18ec-4e0c-b437-f41353a49d42\",\"sessionkey\":\"6m0yzvxJKeK7qzZvnst3VAYZZSo\",\"httpmethod\":\"POST\",\"ctxAccountId\":\"2\",\"uuid\":\"7b333b33-4c91-42f3-bd0b-6455d8b75360\",\"domainid\":\"563b4c50-7dfd-11f0-a9b9-525400000078\",\"cmdEventType\":\"VOLUME.CREATE\",\"response\":\"json\",\"ctxUserId\":\"2\",\"name\":\"fsdf\",\"zoneid\":\"45e225f9-894b-4e0f-8aef-f576f117052f\",\"ctxStartEventId\":\"393\",\"id\":\"18\",\"ctxDetails\":\"{\\\"interface com.cloud.storage.Snapshot\\\":\\\"ffd6b8a4-18ec-4e0c-b437-f41353a49d42\\\",\\\"interface com.cloud.dc.DataCenter\\\":\\\"45e225f9-894b-4e0f-8aef-f576f117052f\\\",\\\"interface com.cloud.domain.Domain\\\":\\\"563b4c50-7dfd-11f0-a9b9-525400000078\\\",\\\"interface com.cloud.storage.Volume\\\":\\\"7b333b33-4c91-42f3-bd0b-6455d8b75360\\\"}\",\"account\":\"admin\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":170,"initMsid":90520730730616,"instanceId":18,"instanceType":"Volume","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"641fbb02-a925-444a-b49b-b75a27e35705"}
2025-08-21 06:56:18,120 DEBUG [c.c.a.ApiServlet] (qtp1438988851-18:[ctx-7efce117, ctx-b0128148]) (logid:619c3a5e) ===END===  192.168.125.1 -- POST  
command=createVolume 
response=json 
zoneid=45e225f9-894b-4e0f-8aef-f576f117052f 
name=fsdf 
snapshotid=ffd6b8a4-18ec-4e0c-b437-f41353a49d42 
domainid=563b4c50-7dfd-11f0-a9b9-525400000078 
account=admin 
sessionkey=6m0yzvxJKeK7qzZvnst3VAYZZSo 

2025-08-21 06:56:18,126 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) LocalStoragePoolAllocator is returning null since the disk profile does not use local storage and bypassStorageTypeCheck is false.
2025-08-21 06:56:18,126 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Could not find any available Storage Pool using Storage Pool Allocator [LocalStoragePoolAllocator].
2025-08-21 06:56:18,126 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Looking for pools in dc [1], pod [1] and cluster [null]. Disabled pools will be ignored.
2025-08-21 06:56:18,127 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found pools [[StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}]] that match with tags [[]].
2025-08-21 06:56:18,128 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Checking if storage pool [StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}] is suitable to disk [DskChr[DATADISK|8589934592|]].
2025-08-21 06:56:18,128 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Volume [Volume {"id":18,"instanceId":null,"name":"fsdf","uuid":"7b333b33-4c91-42f3-bd0b-6455d8b75360","volumeType":"DATADISK"}] is not allocated to any pool. Cannot check compatibility with pool [StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}].
2025-08-21 06:56:18,128 INFO  [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Storage pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} does not supply IOPS capacity, assuming enough capacity
2025-08-21 06:56:18,129 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Checking pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} for storage, totalSize: 257182138368, usedBytes: 9224268800, usedPct: 0.035866677439321475, disable threshold: 0.85
2025-08-21 06:56:18,129 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Destination pool: StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}
2025-08-21 06:56:18,131 DEBUG [c.c.a.ApiServlet] (qtp1438988851-23:[ctx-080d1748]) (logid:3f42bd99) ===START===  192.168.125.1 -- GET  jobId=641fbb02-a925-444a-b49b-b75a27e35705&command=queryAsyncJobResult&response=json&
2025-08-21 06:56:18,132 DEBUG [c.c.a.ApiServlet] (qtp1438988851-23:[ctx-080d1748]) (logid:3f42bd99) Two factor authentication is already verified for the user 2, so skipping
2025-08-21 06:56:18,133 DEBUG [c.c.a.ApiServer] (qtp1438988851-23:[ctx-080d1748, ctx-ff7b89bc]) (logid:3f42bd99) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"8167e0a7-7dfd-11f0-a9b9-525400000078"}]' is allowed to perform API calls: 0.0.0.0/0,::/0
2025-08-21 06:56:18,134 INFO  [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1438988851-23:[ctx-080d1748, ctx-ff7b89bc]) (logid:3f42bd99) Account for user id 8168499c-7dfd-11f0-a9b9-525400000078 is Root Admin or Domain Admin, all APIs are allowed.
2025-08-21 06:56:18,134 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1438988851-23:[ctx-080d1748, ctx-ff7b89bc]) (logid:3f42bd99) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2025-08-21 06:56:18,134 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1438988851-23:[ctx-080d1748, ctx-ff7b89bc]) (logid:3f42bd99) API rate limiting is disabled. We will not use ApiRateLimitService.
2025-08-21 06:56:18,136 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Pool ID for the volume Volume {"id":18,"instanceId":null,"name":"fsdf","uuid":"7b333b33-4c91-42f3-bd0b-6455d8b75360","volumeType":"DATADISK"} is null
2025-08-21 06:56:18,138 DEBUG [c.c.a.ApiServlet] (qtp1438988851-23:[ctx-080d1748, ctx-ff7b89bc]) (logid:3f42bd99) ===END===  192.168.125.1 -- GET  jobId=641fbb02-a925-444a-b49b-b75a27e35705&command=queryAsyncJobResult&response=json&
2025-08-21 06:56:18,138 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found storage pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} of type Linstor with overprovisioning factor 2
2025-08-21 06:56:18,138 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Total over provisioned capacity calculated is 2 * (239.52 GB) 257182138368
2025-08-21 06:56:18,138 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Total capacity of the pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} is (479.04 GB) 514364276736
2025-08-21 06:56:18,139 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Checking pool: StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} for storage allocation , maxSize : (479.04 GB) 514364276736, totalAllocatedSize : (22.65 GB) 24321341440, askingSize : (8.00 GB) 8589934592, allocated disable threshold: 0.85
2025-08-21 06:56:18,139 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found suitable cluster storage pool [StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}] to allocate disk [DskChr[DATADISK|8589934592|]] to it, adding to list.
2025-08-21 06:56:18,140 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) ClusterScopeStoragePoolAllocator is returning [1] suitable storage pools [[StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}]].
2025-08-21 06:56:18,140 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Using volume allocation algorithm random to reorder pools.
2025-08-21 06:56:18,140 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) VM [null] does not have a preferred storage pool or it cannot be used. Volume Orchestrator will use the available Storage Pool [StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}], which was discovered using Storage Pool Allocator [ClusterScopeStoragePoolAllocator].
2025-08-21 06:56:18,140 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found a suitable pool [{"name":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}] to create the volume [{"name":"fsdf","uuid":"7b333b33-4c91-42f3-bd0b-6455d8b75360"}] in.
2025-08-21 06:56:18,142 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Creating volume from snapshot, with dataStore role Primary and on primary storage: false
2025-08-21 06:56:18,144 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found [0] snapshots [[]] that have checkpoints for volume with id [16].
2025-08-21 06:56:18,150 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found [0] snapshots [[]] that have checkpoints for volume with id [18].
2025-08-21 06:56:18,153 DEBUG [o.a.c.s.d.d.LinstorPrimaryDataStoreDriverImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) LinstorPrimaryDataStoreDriverImpl.canCopy: SNAPSHOT -> VOLUME
2025-08-21 06:56:18,153 DEBUG [o.a.c.s.d.d.LinstorPrimaryDataStoreDriverImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) LinstorPrimaryDataStoreDriverImpl.canCopy: SNAPSHOT -> VOLUME
2025-08-21 06:56:18,153 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type VOLUME
2025-08-21 06:56:18,153 DEBUG [o.a.c.s.c.a.StorageCacheRandomAllocator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Can't find staging storage in zone: 1
2025-08-21 06:56:18,154 ERROR [o.a.c.s.m.AncientDataMotionStrategy] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Failed to create volume from test on pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"} java.lang.NullPointerException: Cannot invoke "org.apache.cloudstack.engine.subsystem.api.storage.DataStore.getId()" because "store" is null
	at org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.createCacheObject(StorageCacheManagerImpl.java:230)
	at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cacheSnapshotChain(AncientDataMotionStrategy.java:252)
	at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyVolumeFromSnapshot(AncientDataMotionStrategy.java:287)
	at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:519)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:88)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:117)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromSnapshot(VolumeServiceImpl.java:1683)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.createVolumeFromSnapshot(VolumeOrchestrator.java:622)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy257.createVolumeFromSnapshot(Unknown Source)
	at com.cloud.storage.VolumeApiServiceImpl.createVolumeFromSnapshot(VolumeApiServiceImpl.java:1106)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:1055)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:251)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy283.createVolume(Unknown Source)
	at org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd.execute(CreateVolumeCmd.java:224)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:689)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:637)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2025-08-21 06:56:18,155 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) copy failed com.cloud.utils.exception.CloudRuntimeException: Failed to create volume from test on pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}
	at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyVolumeFromSnapshot(AncientDataMotionStrategy.java:313)
	at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:519)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:88)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:117)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromSnapshot(VolumeServiceImpl.java:1683)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.createVolumeFromSnapshot(VolumeOrchestrator.java:622)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy257.createVolumeFromSnapshot(Unknown Source)
	at com.cloud.storage.VolumeApiServiceImpl.createVolumeFromSnapshot(VolumeApiServiceImpl.java:1106)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:1055)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:251)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy283.createVolume(Unknown Source)
	at org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd.execute(CreateVolumeCmd.java:224)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:689)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:637)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2025-08-21 06:56:18,158 ERROR [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Failed to create volume from snapshot [{"name":"test","uuid":"ffd6b8a4-18ec-4e0c-b437-f41353a49d42"}] due to [com.cloud.utils.exception.CloudRuntimeException: Failed to create volume from test on pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}].
2025-08-21 06:56:18,159 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Found [0] snapshots [[]] that have checkpoints for volume with id [18].
2025-08-21 06:56:18,161 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Updating resource Type = volume count for Account with id = 2 Operation = decreasing Amount = 1
2025-08-21 06:56:18,162 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170, ctx-74db615a]) (logid:641fbb02) Updating resource Type = primary_storage count for Account with id = 2 Operation = decreasing Amount = (8.00 GB) 8589934592
2025-08-21 06:56:18,166 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-82:[ctx-aa54f249, job-170]) (logid:641fbb02) Unexpected exception while executing org.apache.cloudstack.api.command.admin.volume.CreateVolumeCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Failed to create volume: Volume {"id":18,"instanceId":null,"name":"fsdf","uuid":"7b333b33-4c91-42f3-bd0b-6455d8b75360","volumeType":"DATADISK"}
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:1083)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:251)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy283.createVolume(Unknown Source)
	at org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd.execute(CreateVolumeCmd.java:224)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:689)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:637)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to create volume from snapshot [{"name":"test","uuid":"ffd6b8a4-18ec-4e0c-b437-f41353a49d42"}] due to [com.cloud.utils.exception.CloudRuntimeException: Failed to create volume from test on pool StoragePool {"id":1,"name":"Linstor","poolType":"Linstor","uuid":"bfd430ec-5689-4acb-9c35-4b08a4bb9dcd"}].
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.createVolumeFromSnapshot(VolumeOrchestrator.java:630)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy257.createVolumeFromSnapshot(Unknown Source)
	at com.cloud.storage.VolumeApiServiceImpl.createVolumeFromSnapshot(VolumeApiServiceImpl.java:1106)
	at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:1055)
	... 29 more

2025-08-21 06:56:18,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-82:[ctx-aa54f249, job-170]) (logid:641fbb02) Complete async job-170, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to create volume: Volume {"id":18,"instanceId":null,"name":"fsdf","uuid":"7b333b33-4c91-42f3-bd0b-6455d8b75360","volumeType":"DATADISK"}"}

@slavkap
Copy link
Contributor Author

slavkap commented Aug 21, 2025

thanks @rp-, I'm working on it

@slavkap
Copy link
Contributor Author

slavkap commented Aug 21, 2025

@rp-, can you please test this again if you have time?

@sureshanaparti
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14690

@rp-
Copy link
Contributor

rp- commented Aug 21, 2025

@rp-, can you please test this again if you have time?

works now!

@sureshanaparti
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@sureshanaparti
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link
Contributor

@nvazquez nvazquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - have tried on env with 2 zones having cluster wide NFS storages:

Deploy VM on Zone A:

Setting snapshot.backup.to.secondary = false:

  • Take snapshot of VM’s ROOT volume on Zone A
  • Create template from snapshot on Zone A
  • Deploy VM from template on Zone A

Setting snapshot.backup.to.secondary = true:

  • Take snapshot of VM’s ROOT volume on Zone A and Zone B (by adding the additional zone on the Take snapshot wizard)
  • Create template from snapshot on Zone A
  • Create template from snapshot on Zone B
  • Deploy VM from template on Zone A
  • Deploy VM from template on Zone B

@slavkap
Copy link
Contributor Author

slavkap commented Aug 21, 2025

Tested with Ceph
snapshot.backup.to.secondary = true
Create a snapshot from the ROOT volume

  • create a template from the snapshot - passed
  • create a volume from the snapshot - passed

snapshot.backup.to.secondary = false
Create a snapshot from the ROOT volume

  • create a template from the snapshot - passed // The snapshot is backed up to the secondary storage, and after creating the volume, the backup is deleted
  • create a volume from the snapshot - passed // The snapshot is backed up to the secondary storage, and after creating the template, the backup is deleted

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14701

@sureshanaparti
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@sureshanaparti
Copy link
Contributor

LGTM. Verified volume snapshot creation with volumes on NFS primary (backed to NFS image store) & PowerFlex pools, and able to create template & volume from snapshots on NFS image store, create template from snapshot on PowerFlex storage pool.

@sureshanaparti sureshanaparti merged commit 1272b13 into apache:main Aug 21, 2025
24 of 26 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.21.0 Aug 21, 2025
@blueorangutan
Copy link

[SF] Trillian test result (tid-14096)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51731 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11490-t14096-kvm-ol8.zip
Smoke tests completed. 146 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Sep 4, 2025
…ache#11490)

* Fix of create template from snapshot on another zone

When a snapshot has a copy on StorPool primary storage in another zone, but the original snapshot resides on secondary storage, creating a template from the copied snapshot results in the template being created in the first zone.
If the snapshot.backup.to.secondary setting is disabled, and a user creates a volume or template from a snapshot, the snapshot is temporarily backed up to secondary storage during the operation. After the operation, this backup should be deleted. However, the snapshot currently remains on both primary and secondary storage.

* update snapshot info depending on the data store role
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants