CSTACKEX-25: ONTAP Primary storage pool creation #5

sandeeplocharla · 2025-10-13T16:18:33Z

Description

This PR...

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

rajiv-jain-netapp · 2025-10-15T05:19:41Z

plugins/storage/volume/ontap/pom.xml

    </parent>
    <properties>
        <spring-cloud.version>2021.0.7</spring-cloud.version>
+        <spring-boot.version>2.7.10</spring-boot.version>


remove this and below commented ones

Taken care of

rajiv-jain-netapp · 2025-10-15T05:22:18Z

.../ontap/src/main/java/org/apache/cloudstack/storage/feign/model/request/VolumeRequestDTO.java

    private String name;
    private List<AggregateDTO> aggregates;
    private SvmDTO svm;
+    private Integer size;


Make it of type Long. And please mention the comment stating "this value would always be in bytes".

rajiv-jain-netapp · 2025-10-15T05:30:21Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

-
-        return null;
-
+        String url = dsInfos.get("url").toString();


add logger with input map values

rajiv-jain-netapp · 2025-10-15T05:35:12Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+
+        // Additional details requested for ONTAP primary storage pool creation
+        @SuppressWarnings("unchecked")
+        Map<String, String> details = (Map<String, String>)dsInfos.get("details");


We have to check for empty and null both.

We have to throw exceptions in case of failed validations.

Added where its necessary

rajiv-jain-netapp · 2025-10-15T05:51:02Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

    public boolean attachZone(DataStore dataStore, ZoneScope scope, Hypervisor.HypervisorType hypervisorType) {
-        return false;
+        List<HostVO> hostsToConnect = new ArrayList<>();
+        Hypervisor.HypervisorType[] hypervisorTypes = {Hypervisor.HypervisorType.XenServer, Hypervisor.HypervisorType.VMware, Hypervisor.HypervisorType.KVM};


Please limit support for the KVM hypervisor for now. I think that the remaining would be done in future releases.

...olume/ontap/src/main/java/org/apache/cloudstack/storage/provider/StorageProviderManager.java

github-actions · 2025-10-16T09:26:15Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

sandeeplocharla · 2025-10-17T05:30:24Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+            }
+        }
+
+        if (storagePoolName == null || storagePoolName.isEmpty()) {


add check for space

sandeeplocharla · 2025-10-17T05:36:31Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+        // TODO: scheme could be 'custom' in our case and we might have to ask 'protocol' separately to the user
+        if (scheme.equalsIgnoreCase(Constants.NFS)) {
+            parameters.setType(Storage.StoragePoolType.NetworkFilesystem);
+        } else if (scheme.equalsIgnoreCase(Constants.ISCSI)) {


TODO:// Compare it with custom
and specific protocol field should be included on UI and validated here

sandeeplocharla · 2025-10-17T05:38:29Z

...ins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/model/OntapStorage.java

+        Username = username;
+        Password = password;
+        ManagementLIF = managementLIF;
+        OntapStorage.SVM = SVM;


this should just be 'svm'

sandeeplocharla · 2025-10-17T05:39:09Z

...orage/volume/ontap/src/main/java/org/apache/cloudstack/storage/feign/model/OntapStorage.java

Add this in feign/model

sandeeplocharla · 2025-10-17T05:40:45Z

...olume/ontap/src/main/java/org/apache/cloudstack/storage/provider/StorageProviderManager.java

+import org.springframework.stereotype.Component;
+
+@Component
+public class StorageProviderManager {


Change the name to StorageProviderFactory

sandeeplocharla · 2025-10-17T05:44:12Z

...olume/ontap/src/main/java/org/apache/cloudstack/storage/provider/StorageProviderManager.java

+    public StorageProviderManager(OntapStorage ontapStorage) {
+        String protocol = ontapStorage.getProtocol();
+        s_logger.info("Initializing StorageProviderManager with protocol: " + protocol);
+        if (protocol.equalsIgnoreCase(Constants.NFS)) {


change this to switch and add isDisaggregated flag to OntapStoragr, add the respective condition here

sandeeplocharla · 2025-10-17T05:46:26Z

...torage/volume/ontap/src/main/java/org/apache/cloudstack/storage/service/StorageStrategy.java

+        try {
+            // Call the SVM API to check if the SVM exists
+            Svm svm = null;
+            URI url = URI.create(Constants.HTTPS + storage.getManagementLIF() + Constants.GETSVMs);


create a common method to Utility.java for uri creation

sandeeplocharla · 2025-10-17T05:57:10Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+                details.get(Constants.MANAGEMENTLIF), details.get(Constants.SVMNAME), scheme); //TODO: Here the passing 'scheme' might need a re-look
+        StorageProviderManager storageProviderManager = new StorageProviderManager(ontapStorage);
+        StorageStrategy storageStrategy = storageProviderManager.getStrategy();
+        boolean isValid = storageStrategy.connect();


put this in try catch

sandeeplocharla · 2025-10-17T06:00:06Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+        boolean isValid = storageStrategy.connect();
+        if (isValid) {
+//            String volumeName = storagePoolName + "_vol"; //TODO: Figure out a better naming convention
+            storageStrategy.createVolume(storagePoolName, Long.parseLong((details.get("size")))); // TODO: size should be in bytes, so see if conversion is needed


try catch block

sandeeplocharla · 2025-10-17T06:14:06Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+    public boolean attachCluster(DataStore dataStore, ClusterScope scope) {
+        logger.debug("In attachCluster for ONTAP primary storage");
+        PrimaryDataStoreInfo primarystore = (PrimaryDataStoreInfo)dataStore;
+        List<HostVO> hostsToConnect = _resourceMgr.getEligibleUpAndEnabledHostsInClusterForStorageConnection(primarystore);


Add check to see if the host supports iscsi, because the scheme could be 'custom'

sandeeplocharla · 2025-10-17T06:22:09Z

...ap/src/main/java/org/apache/cloudstack/storage/lifecycle/OntapPrimaryDatastoreLifecycle.java

+        for (HostVO host : hostsToConnect) {
+            // TODO: Fetch the host IQN and add to the initiator group on ONTAP cluster
+            try {
+                _storageMgr.connectHostToSharedPool(host, dataStore.getId());


Read about connectHostsToPool and decide on which one to use

rajiv-jain-netapp

please exclude the files which are taken from other PR and repost.

…e#11639)

…gorithm (apache#11751) * Consider Instance in Starting state as well for allocation algorithm * use IN instead of OR statement

…e#11735)

…rotocols for password reset (apache#11228)

…n during mgmt server / agent start (apache#11722)

…11787)

This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands. We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script. Added global settings: systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature. console.proxy.vm.userdata - UUID of the User data to be used for Console Proxy secstorage.vm.userdata - UUID of the User data to be used for Secondary Storage VM virtual.router.userdata - UUID of the User data to be used for Virtual Routers

* Return details of the storage pool in the response including url, and update capacityBytes and capacityIops if applicable while creating storage pool * Added capacitybytes parameter to the storage pool response in sync with the capacityiops response parameter and createStoragePool cmd request parameter (existing disksizetotal parameter in the storage pool response can be deprecated) * Don't keep url in details * Persist the capacityBytes and capacityIops in the storage_pool_details table while creating storage pool as well, for consistency - as these are updated with during update storage pool * rebase with main fixes

…pache#11488) Addresses apache#11483 Signed-off-by: Abhishek Kumar <[email protected]>

Co-authored-by: Henrique Sato <[email protected]>

…ls during zone deployment (apache#11760)

…ough Script) (apache#11801)

apache#11773) * storage: change storage pool to Up state when cancel storage migration * Update 11773: connect host to shared pool after cancelling storage migration * Update 11773: update db only * Update 11773: skip capacity update for storpool

… added EOF fixes

… added EOF fixes + correcting license header

…POJOs" This reverts commit fe0f752.

…POJOs" This reverts commit 28faca1.

* CSTACKEX-29 Cluster, SVM and Aggr Feign Client * CSTACKEX-29 Change the endpoint method name in feign client * CSTACKEX-29 Make the alignment proper * CSTACKEX-29 Added License Info * CSTACKEX-29 Resolve Review Comments * CSTACKEX-29 Remove Component Annotation from datastoredriverclass * CSTACKEX-29 Resolve Style check issues * CSTACKEX-29 Resolve ALL Style issues * CSTACKEX-29 Resolve Precommits Issues * CSTACKEX-29 Added Method comments and change the ontap response class name --------- Co-authored-by: Gupta, Surya <[email protected]>

* CSTACKEX-31 NAS and Job Feign Client and POJOs * CSTACKEX-31 Fixed Checks Issues * CSTACKEX-31 Resolve Review Comments * CSTACKEX-31 Resolve Review Comments * CSTACKEX-31 Resolve Review Comments * CSTACKEX-31 Added Aggr and size to volume model * CSTACKEX-31 Change the export policy endpoint path * CSTACKEX-31 Fixed check styles --------- Co-authored-by: Gupta, Surya <[email protected]>

…ommits. # This is the 1st commit message: CSTACKEX-25: Basic class structure # This is the commit message #2: Add PrimaryStoragePool base code # This is the commit message #3: CSTACKEX-25: Create Volume code basic code added # This is the commit message #4: CSTACKEX-25: additional logic for Primary storage pool creation # This is the commit message #5: CSTACKEX-29 Cluster, SVM and Aggr Feign Client # This is the commit message #6: CSTACKEX-29 Added License Info # This is the commit message #7: CSTACKEX-29 Resolve Review Comments # This is the commit message #8: CSTACKEX-29 Resolve Style check issues � This is the commit message #9: CSTACKEX-29 Resolve Style check issues � This is the commit message #10: CSTACKEX-29 Resolve Precommits Issues # This is the commit message #11: CSTACKEX-29 Resolve Precommits Issues

Add PrimaryStoragePool base code CSTACKEX-25: Create Volume code basic code added CSTACKEX-25: additional logic for Primary storage pool creation CSTACKEX-29 Cluster, SVM and Aggr Feign Client CSTACKEX-29 Added License Info CSTACKEX-29 Resolve Review Comments CSTACKEX-29 Resolve Style check issues � This is the commit message #9: CSTACKEX-29 Resolve Style check issues � This is the commit message #10: CSTACKEX-29 Resolve Precommits Issues CSTACKEX-29 Resolve Precommits Issues CSTACKEX-25: PrimaryStoragePool create workflow almost done CSTACKEX-25: PrimaryStoragePool create workflow almost done CSTACKEX-25: Added license string to new file

…ents

sandeeplocharla self-assigned this Oct 13, 2025

sandeeplocharla force-pushed the feature/CSTACKEX-25 branch from 994aac7 to 11693f3 Compare October 14, 2025 09:42

rajiv-jain-netapp self-requested a review October 15, 2025 04:45

sandeeplocharla requested review from piyush5netapp and suryag1201 October 15, 2025 04:45

rajiv-jain-netapp reviewed Oct 15, 2025

View reviewed changes

github-actions bot added the status:has-conflicts label Oct 16, 2025

sandeeplocharla changed the title ~~CSTACKEX-25: Basic class structure~~ CSTACKEX-25: ONTAP Primary storage pool creation Oct 17, 2025

sandeeplocharla requested a review from rajiv-jain-netapp October 17, 2025 03:47

sandeeplocharla commented Oct 17, 2025

View reviewed changes

rajiv-jain-netapp reviewed Oct 17, 2025

View reviewed changes

abh1sar and others added 8 commits October 21, 2025 08:15

UI support for extraconfig in deploy and update instance (apache#11719)

c7b0005

CKS: generate a random UUID as password of CKS user in project (apach…

c822028

…e#11639)

server: Consider Instance in Starting state as well for allocation al…

1f436b6

…gorithm (apache#11751) * Consider Instance in Starting state as well for allocation algorithm * use IN instead of OR statement

CKS: fix CKS creation on an existing Shared and Routed network (apach…

23dbe3c

…e#11735)

VMware: match nic mac for ip address fetch (apache#10641)

6da9414

server: add user.password.reset.smtp.useStartTLS and enabledSecurityP…

9e46ee1

…rotocols for password reset (apache#11228)

server: do not enable the disabled local storage(s) on host connectio…

77247b1

…n during mgmt server / agent start (apache#11722)

linstor: use sparse/discard qemu-img convert on thin devices (apache#…

1fc1a5d

…11787)

vishesh92 and others added 27 commits October 21, 2025 08:15

refactor: remove use of term entry-point from extensions code base (a…

4a9b4a4

…pache#11488) Addresses apache#11483 Signed-off-by: Abhishek Kumar <[email protected]>

Add Hypervisor default as cache mode for disk offerings (apache#10282)

39ef8f6

Co-authored-by: Henrique Sato <[email protected]>

systemvmtemplate: Bump Debian version to 12.12.0 (apache#11778)

572c520

UI: Deal with crosssite api call after login (apache#10533)

9b8d822

UI: Fix primary storage for datastore cluster and retain traffic labe…

1b6436d

…ls during zone deployment (apache#11760)

Sanitize the rbd file cmd parameter logs during qemu-img convert (thr…

2392bc8

…ough Script) (apache#11801)

Initial primary storage pool plugin skeleton

80a3c0b

Initial primary storage pool plugin skeleton - added license string

db371e2

Initial primary storage pool plugin skeleton - added license string +…

9bb758a

… added EOF fixes

Initial primary storage pool plugin skeleton - added license string +…

25fb255

… added EOF fixes + correcting license header

Feignconfiguration and volume feignClient along with desired POJOs

6d933e8

Feignconfiguration and volume feignClient along with desired POJOs

f07a444

Revert "Feignconfiguration and volume feignClient along with desired …

43867e6

…POJOs" This reverts commit fe0f752.

Revert "Feignconfiguration and volume feignClient along with desired …

4f57cd2

…POJOs" This reverts commit 28faca1.

Feignconfiguration and volume feignClient along with desired POJOs

bd4a23a

Feignconfiguration and volume feignClient along with desired POJOs

1bb907f

CSTACKEX-28: added copyright comment in the logger configuration

3327c15

CSTACKEX-28: added newline in the end of the file

35ccaab

CSTACKEX-28 - incorporated review comments

b4c2f30

CSTACKEX-25: Added code for job polling and addressed the review comm…

0be388b

…ents

sandeeplocharla force-pushed the feature/CSTACKEX-25 branch from 9072bc3 to 0be388b Compare October 21, 2025 02:46

sandeeplocharla closed this Oct 22, 2025

CSTACKEX-25: ONTAP Primary storage pool creation #5

CSTACKEX-25: ONTAP Primary storage pool creation #5

Uh oh!

Conversation

sandeeplocharla commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Oct 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sandeeplocharla Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rajiv-jain-netapp left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

sandeeplocharla commented Oct 13, 2025 •

edited

Loading

sandeeplocharla Oct 17, 2025 •

edited

Loading