Skip to content

Conversation

@vishesh92
Copy link
Member

@vishesh92 vishesh92 commented Aug 21, 2024

Description

This PR fixes #8417 by allowing the user to register an ISO by uploading from local.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@codecov
Copy link

codecov bot commented Aug 21, 2024

Codecov Report

❌ Patch coverage is 9.77444% with 120 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.35%. Comparing base (3d6ec29) to head (1abe54e).
⚠️ Report is 129 commits behind head on main.

Files with missing lines Patch % Lines
...bernetes/version/KubernetesVersionManagerImpl.java 23.63% 41 Missing and 1 partial ⚠️
...tUploadParamsForKubernetesSupportedVersionCmd.java 0.00% 36 Missing ⚠️
...che/cloudstack/api/AbstractGetUploadParamsCmd.java 0.00% 21 Missing ⚠️
...api/command/user/iso/GetUploadParamsForIsoCmd.java 0.00% 18 Missing ⚠️
...oudstack/api/response/GetUploadParamsResponse.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9561      +/-   ##
============================================
- Coverage     17.36%   17.35%   -0.01%     
- Complexity    15237    15238       +1     
============================================
  Files          5888     5889       +1     
  Lines        525741   525898     +157     
  Branches      64164    64174      +10     
============================================
+ Hits          91274    91277       +3     
- Misses       424167   424320     +153     
- Partials      10300    10301       +1     
Flag Coverage Δ
uitests 3.62% <ø> (-0.01%) ⬇️
unittests 18.39% <9.77%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vishesh92 vishesh92 force-pushed the k8s-version-add-iso-id branch from 158acb9 to 364f4ce Compare August 21, 2024 09:58
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10728

@FelipeM525
Copy link
Contributor

@vishesh92 would you mind adding some images to the description?

@shwstppr
Copy link
Contributor

@vishesh92 will this create access issues? Currently, k8s ISOs are registered as public and are accessible to all. But now will it allow linking any existing ISO which can be seen by root admin.
Also, earlier such ISOs were owned by system account to prevent direct deletion. Is there a check to prevent deletion if it is linked to a k8s version?

@vishesh92
Copy link
Member Author

@vishesh92 will this create access issues? Currently, k8s ISOs are registered as public and are accessible to all. But now will it allow linking any existing ISO which can be seen by root admin. Also, earlier such ISOs were owned by system account to prevent direct deletion. Is there a check to prevent deletion if it is linked to a k8s version?

I can add checks on the backend to ensure that the ISO has the required configuration settings. But it might be confusing for the end user.

Let me revisit the approach and directly allow uploading of ISO for k8s.

@vishesh92 vishesh92 force-pushed the k8s-version-add-iso-id branch from 364f4ce to be1539f Compare August 22, 2024 10:47
@vishesh92 vishesh92 changed the title Provide option to use existing ISOs for kubernetes supported version Allow uploading of ISO for creating kubernetes supported versions Aug 22, 2024
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 10747

@vishesh92 vishesh92 force-pushed the k8s-version-add-iso-id branch from be1539f to 6d378b5 Compare August 23, 2024 06:34
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@weizhouapache
Copy link
Member

@vishesh92
To be frank, I think this is too complicated.
We could add an API to transform a regular ISO (registered or uploaded from local) to a CKS ISO, it will be simple and cover mose cases (e.g. Update an existing ISO to support CKS)

@vishesh92
Copy link
Member Author

@vishesh92 To be frank, I think this is too complicated. We could add an API to transform a regular ISO (registered or uploaded from local) to a CKS ISO, it will be simple and cover mose cases (e.g. Update an existing ISO to support CKS)

@weizhouapache My initial PR was for linking an existing ISO with a CKS Supported version (main...shapeblue:cloudstack:k8s-version-add-iso-id-backup). @shwstppr had some concerns regarding the iso's permissions. So, I made the implementation similar to how it is there right now for templates & ISOs.

@weizhouapache
Copy link
Member

weizhouapache commented Aug 23, 2024

@vishesh92 To be frank, I think this is too complicated. We could add an API to transform a regular ISO (registered or uploaded from local) to a CKS ISO, it will be simple and cover mose cases (e.g. Update an existing ISO to support CKS)

@weizhouapache My initial PR was for linking an existing ISO with a CKS Supported version (main...shapeblue:cloudstack:k8s-version-add-iso-id-backup). @shwstppr had some concerns regarding the iso's permissions. So, I made the implementation similar to how it is there right now for templates & ISOs.

we can add some restrictions

  • only root admin can do (same as add k8s version API)
  • the ISO will be changed to the same properties as k8s ISO (e.g. owner is system account, all users can access)

I had a quick look at your initial PR, it looks ok to me

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10772

@vishesh92
Copy link
Member Author

@vishesh92 To be frank, I think this is too complicated. We could add an API to transform a regular ISO (registered or uploaded from local) to a CKS ISO, it will be simple and cover mose cases (e.g. Update an existing ISO to support CKS)

@weizhouapache My initial PR was for linking an existing ISO with a CKS Supported version (main...shapeblue:cloudstack:k8s-version-add-iso-id-backup). @shwstppr had some concerns regarding the iso's permissions. So, I made the implementation similar to how it is there right now for templates & ISOs.

we can add some restrictions

* only root admin can do (same as add k8s version API)

* the ISO will be changed to the same properties as k8s ISO (e.g. owner is system account, all users can access)

I had a quick look at your initial PR, it looks ok to me

@weizhouapache I am not sure about changing the properties of the existing ISO. I can maybe create a copy of the ISO with required ownership and properties. Do you think that would be a better approach?

@github-actions
Copy link

github-actions bot commented Sep 6, 2024

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@JoaoJandre JoaoJandre removed this from the 4.20.0.0 milestone Sep 10, 2024
@harikrishna-patnala harikrishna-patnala changed the base branch from 4.20 to main September 4, 2025 04:46
@harikrishna-patnala
Copy link
Contributor

harikrishna-patnala commented Sep 4, 2025

moved to 4.22.0 milestone. cc @harikrishna-patnala @vishesh92

@vishesh92 can you please change the base branch to main

I've changed it, please check the conflicts

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 14844

@vishesh92
Copy link
Member Author

@blueorangutan test keepEnv

@blueorangutan
Copy link

@vishesh92 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@vishesh92
Copy link
Member Author

@vishesh92

Failed iso upload remains in allocated state

Step to reproduce the issue

1. Navaigte  to images > kubetnetes iso > Upload a k8s iso

2. For some reason the job fails

3. The kubernetes iso state will be in Allocated state

Screenshot 2025-03-18 at 12 12 19 PM

4. Navaigte to images >iso  the k8s iso will be in "Not ready" state

Screenshot 2025-03-18 at 12 12 31 PM

5. Navaigte  to images > kubetnetes iso

Later on it goes into a failed state after "upload.operation.timeout)"

Screenshot 2025-03-18 at 12 31 59 PM

While images > Iso > the state will be still in Not ready

In case of failures

When end user executed the API call list kubernetessupportedversions

can you change the behaviour of isostate from

Allocated >>failed

to

Notready to failed

(localcloud) 🐱 > list kubernetessupportedversions filter=isoname,isostate { "count": 4, "kubernetessupportedversion": [ { "isoname": "v1.31.1-Kubernetes-Binaries-ISO", "isostate": "Failed" },

@kiranchavala Can you try again? You will need to ensure certs are valid for the SSVM or an exception is added.

@vishesh92
Copy link
Member Author

@blueorangutan test

@vishesh92
Copy link
Member Author

@vishesh92
Failed iso upload remains in allocated state
Step to reproduce the issue

1. Navaigte  to images > kubetnetes iso > Upload a k8s iso

2. For some reason the job fails

3. The kubernetes iso state will be in Allocated state

Screenshot 2025-03-18 at 12 12 19 PM

4. Navaigte to images >iso  the k8s iso will be in "Not ready" state

Screenshot 2025-03-18 at 12 12 31 PM

5. Navaigte  to images > kubetnetes iso

Later on it goes into a failed state after "upload.operation.timeout)"
Screenshot 2025-03-18 at 12 31 59 PM
While images > Iso > the state will be still in Not ready
In case of failures
When end user executed the API call list kubernetessupportedversions
can you change the behaviour of isostate from
Allocated >>failed
to
Notready to failed
(localcloud) 🐱 > list kubernetessupportedversions filter=isoname,isostate { "count": 4, "kubernetessupportedversion": [ { "isoname": "v1.31.1-Kubernetes-Binaries-ISO", "isostate": "Failed" },

@kiranchavala Can you try again? You will need to ensure certs are valid for the SSVM or an exception is added in the browser.

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian Build Failed (tid-14244)

@blueorangutan
Copy link

[SF] Trillian test result (tid-14250)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51965 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9561-t14250-kvm-ol8.zip
Smoke tests completed. 146 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@rosi-shapeblue
Copy link
Collaborator

1. Valid ISO upload:

  • Upload ISO from local → progress bar works → Allocated → Ready
  • K8s supported version is created and visible in UI
  • DB reflects ISO as Active
  • Cleanup works correctly.
Screenshot from 2025-10-13 11-41-06 Screenshot from 2025-10-13 11-41-10 Screenshot from 2025-10-13 11-42-18
mysql> SELECT id, name, state FROM vm_template WHERE name LIKE '%Kubernetes-Binaries-ISO%';
+-----+-------------------------------------------+--------+
| id  | name                                      | state  |
+-----+-------------------------------------------+--------+
| 211 | test-local-upload-Kubernetes-Binaries-ISO | Active |
+-----+-------------------------------------------+--------+
1 row in set (0.00 sec)

(localcloud) 🐱 > list kubernetessupportedversions filter=name,isoname,isostate
{
  "count": 1,
  "kubernetessupportedversion": [
    {
      "isoname": "test-local-upload-Kubernetes-Binaries-ISO",
      "isostate": "Ready",
      "name": "test-local-upload"
    }
  ]
}

2. Invalid ISO / format upload:

image image image

3. Permission & Role Testing:

image
  • with permissions (NOTE: only admin users should be allowed to execute such uploads; the modal is not functional, despite it might be present in some cases where the user was granted such permissions)
image
  • without permissions:
image

4. URL Path Upload

via UI / API

(admin) 🐱 > addKubernetesSupportedVersion semanticversion=1.33.1 url=https://download.cloudstack.org/cks/setup-v1.33.1-calico-aarch64.iso mincpunumber=2 minmemory=2048 zoneid=139d7f30-ab8e-4dbb-9f67-f23f4de8241c
{
  "kubernetessupportedversion": {
    "arch": "x86_64",
    "created": "2025-10-13T10:10:30+0000",
    "directdownload": false,
    "id": "ea635f13-f475-4db1-972d-d7951668fa23",
    "isoid": "e9444a71-fc81-4cb1-ba2d-b21c9a24044a",
    "isoname": "v1.33.1-ref-trl-9660-k-Mol8-rositsa-kyuchukova-Kubernetes-Binaries-ISO",
    "isostate": "Creating",
    "mincpunumber": 2,
    "minmemory": 2048,
    "name": "v1.33.1-ref-trl-9660-k-Mol8-rositsa-kyuchukova",
    "semanticversion": "1.33.1",
    "state": "Enabled",
    "supportsautoscaling": true,
    "supportsha": true,
    "zoneid": "139d7f30-ab8e-4dbb-9f67-f23f4de8241c",
    "zonename": "ref-trl-9660-k-Mol8-rositsa-kyuchukova"
  }
}
(admin) 🐱 >  

image
  • invalid path
image
(admin) 🐱 > delete kubernetessupportedversion id=c5d50d38-c1e3-48ae-8459-0cfee9a7a118
{
  "success": true
}

Copy link
Collaborator

@rosi-shapeblue rosi-shapeblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Test Summary - Kubernetes ISO Upload (Tested for both: Local & URL)

Local upload:

Upload modal works as expected.
Progress bar shows correctly during upload.
ISO transitions: Allocated → Ready.
K8s supported version is created and listed in UI.
DB reflects template as Active.
Cleanup (delete kubernetessupportedversion) works as expected.

Invalid ISO format:

Upload rejected with clear error message.
No version created or stuck entries.

RBAC:

Only Admin can successfully upload ISO.
For non-admin, modal may appear but action is blocked (modal non-functional).
Permission checks behave correctly with custom roles.

URL upload:

Valid URL starts background download on SSVM, progresses normally.
Invalid URL might get stuck in Creating state -> SSVM restart triggers timeout/error (404), making stuck entries deletable.

Deletion via API/CMK works as expected (both via UI and API).

@DaanHoogland DaanHoogland merged commit 0ca267f into apache:main Oct 13, 2025
25 of 26 checks passed
@DaanHoogland DaanHoogland deleted the k8s-version-add-iso-id branch October 13, 2025 10:51
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide an option to upload a kubernetes ISO from Local