List view
We have several automations/pipelines around ALBS and other services that are using some developers' personal access tokens. This is not right. Every pipeline/automation that needs a token to work must use an integration-specific, non-personal token. Some affected services are: * ALBS * Github * Gitea In order for this to happen, we need to: * Collect all the automations that we have around and the tokens they use * Create users/tokens according to the different automations in the different affected services indicated above (some tokens could be shared depending on its purpose) * Update all the automations with their tokens
No due date## Motivation Currently, the AlmaLinux images provided in GCP are not built by us, but by Google. We've been working with the GCP team to change it, and finally, looks like we are going to be able to buid them ourselves with the condition that we also provide SBOMs of the images. We don't have in place anything that automatically generates SBOMs of anything, and we need to make it happen. ## SBOM structure We got the [SBOM of the centos stream image in GCP](https://drive.google.com/file/d/1_ignE_lSGfbWAkKcHubgxiRI4Y8cg6a9/view?usp=sharing) and it contains the following: 1. Usual spdx document info 2. Packages with refs from: * RPMDB - var/lib/rpm/rpmdb.sqlite * python packages manifest files * go modules information 4. Files * Looks like everything in file system * Includes: file path, checksum and license * checksums are all ZERO-ed (SHA1) - tell them about it 5. hasExtractedLicensingInfos * licenses, which are expressed by name, licenseId and extractedText 7. Relationships - file / pkg relationships ## Implementation draft plan We have SBOM data of all our packages stored in immudb, and we can grab it directly by knowing the sha256sum of a package. So yes, it shouldn't be too difficult to make it happen, but we need: 1. A utility/script that: * Assuming that everything is coming from our RPM repos, compile the info accordingly. * If not everything is coming from RPM repos, then we need to do some extra work to figure out things and implement them * Compiles the final SBOM document using the python-spdx lib 2. Figure out where and how this script must run. Ideally, this should be part of the pipeline responsible for the image creation. In fact, we should probably take into account that other cloud providers might start asking for those, so we need to ensure that the utility that we create for this purpose can also be reused in the future. Pain radar: * License matching (Need to take a closer look) * Things not coming from rpm repos, like Python/Go specific stuff - is that part of their gcp-specific images?
Overdue by 1 month(s)•Due by December 1, 2025•5/5 issues closedFix tests in the ALTS (AlmaLinux test system) Make adjustments in ansible playbook for dev environments Make automation to ensure it’s in a working state Speed up repo sync and initial bootstrapping (Also required for dev environments) Investigate which part of ALBS causes the slowness
No due date•0/5 issues closed* Update the guides * Move it to https://wiki.almalinux.org/#contributions-to-the-wiki
No due date•3/6 issues closed- No due date•0/1 issues closed
Red hat is working around evolving errata as explained in [The future of Red Hat security data](https://www.redhat.com/en/blog/future-red-hat-security-data). To summarize: * OVAL v1 was deprecated in mid-2023 * OVAL v2 will continue until the end of 2024 - but el8 and el9 ovals will continue receiving advisories until EOL * CVRF was replaced with CSAF (VEX formatted files), see [this blog post](https://www.redhat.com/en/blog/csaf-vex-documents-now-generally-available) * Introduction of VEX formatted files per CVE, see [this blog post](https://www.redhat.com/en/blog/vulnerability-exploitability-exchange-vex-beta-files-now-available) Our (short) tentative plan, and assuming that we're not generating CSAFs for now, we need to: * Start pulling information from Red Hat errata website * Since OVAL cacher is passing away, our new service needs to ensure that: * The payload data is sent over to API in the same (or similar) way * The DB/API should not need any adjustments (only `errata_type` if we're also bringing non-security advisories) * If we need to pull things from both sides (website and security data API), all the logic must live in the service * Given that we'll no longer getting OVALs, generate required OVAL-specific data before sending over to web_server. Take special attention to "modular erratas" * When we're sending everything right from the new service, and before OVAL is finally deprecated, we must ensure that inserting erratas from both OVAL cacher and the new service results in the same data in DB. This way, we ensure that all our errata data is as consistent as it is now, this is: db, pulp, resulting errata generated data (updateinfo, OVAL, errata.json, etc) When security advisories processing is already working, we also want to start pulling non-security advisories from red hat errata website. This involves: * Adding a switch in public-errata-parser so non-security erratas are also processed and sent to web server * Adding a `is_security` field to `NewErrataParser` * Non-security erratas should be treated exactly the same as security ones, with the exception that non-security erratas won't be included in AlmaLinux OVAL. We can make use of the `is_security` field to filter non-security records
Overdue by 1 year(s)•Due by December 31, 2024•15/17 issues closedWe want achieve following: 1. There are exists tests to validate 60% of code (by classes) in all components Tests should reflect readiness for production environment. Essential functions need to be covered. 2. Tests are run over PR and again over master after merge There should be an environment and automation for it that allow any engineer to run tests against his PR, without a need to wait for someone with access rights. The same tests should be run regularly or after changes against master brunch of codebase.
No due date•23/23 issues closed- Add ability to configure testing: - The repository where tests reside; - The directory where tests are located in the repository (or some rules|regexes); - Pre-test hook (optional); - Environment variables (necessary + custom if some test needs it); - Some order in which tests should be executed (a list of files, or everything in the folder, etc.); - Test execution timeouts (on package basis, with some meaningful defaults if nothing is provided by user); - General requirements: - Detect if VM didn't respond in some time via SSH, drop it, report it as a failure (required); - Be able to reboot VM if test posted a specialized string in its output (like ">>>VMREBOOT<<<"); - Possibly add the ability to configure a need for reboot on the testing configuration phase (optional, not necessary in the initial implementation); - DO NOT USE paramiko/fabric for SSH commands execution (required); - Any abnormalities during the test execution should result a VM termination/deletion and report to main node as test failure with as much explanation as possible (required); - Cache the test repositories in mirror mode to reduce cloning times for each particular environment (required); - Clean up package manager cache before installing anything on the system (required); - Do not expect the environment to have any tools installed (ansible, bats, testinfra, etc.) (optional); - Do not expect the environment to have a repository with tests on it (do some kind of copy to the remote environment) (required); - Be able to run test scripts in detached mode on the remote environment unless a particular runner does not require such approach (provisioners like Ansible, Chef, Puppet, etc.) (required); - Timeouts for tests execution on per-script basis (or per-runner, TBD) (required); - Report back the environment details the tests were running in (VM image name, installed packages, network details; - Add specialized runners for different tests (file formats): - Single command execution on the remote environment; - Shell script executor (with ability to run in detached mode); - BATS executor (with ability to run in detached mode); - Ansible playbooks executor; - Ability to easily add new executors if needed (with ability to run in detached mode); - Dynamic quota allocation for test tasks - Quota managed by user group - Director for overquoting that helps utilize full system while still allowing new urgent builds to be tested - Visualize queues and quotas for users in UI to address lack of transparency
No due date•15/17 issues closedWe need to adapt our packages building flow to all the recent changes in AlmaLinux development. That means adding different automated solutions to get the sources, create security advisories, test the changes, etc. This milestone will contain all the tasks that are related to AlmaLinux workflow change.
No due date•21/24 issues closedRight now it's counter-intuitive to build anything other than AlmaLinux with Build System. We want to remediate that. We reach this milestone when: - User can create a custom product :heavy_check_mark: - User can set up one or many repositories for it (#20) - Several repos in one product - User can set up certificate to sign packages with (#21) :heavy_check_mark: - User can sign built packages with proper certificate (#21) :heavy_check_mark: - Troubles in creating product sign keys - need to verify if it works (#288) - User can update certificate. (Packages signed with old certificate can NOT be validated) (#36) - User can set up their own sign node and use it to sign product packages - This is a long-term idea - In a short-term, we need to be able to limit some sign nodes to only certain products and vice versa - MUST HAVE: limit some sign nodes to only certain products (#289) - NICE TO HAVE: allow products to indicate which sign node they want to use for package signing - User can assign `product_maintainer` role for this product to (him/her) self or to another user without external help (#37) - Assign user roles in product is already possible (see [my comment](https://github.com/AlmaLinux/build-system/issues/37#issuecomment-2124556852)). I'd say that the rest of the AC can be considered low priority. - User can re-release package(s) into another one of their repositories (e.g. from "beta" to "stable") - Need to figure out how to implement this (#290) - Need an endpoint (or any machinery) in which we can create new distros on beholder side to monitor, so beholder helps on dealing with these and match packages in product from several repos (#291) - User can read straightforward, publicly available documentation on how to build custom products in ALBS (#292) - Toggle in product "requires sign", overrides in release the "need to sign" requirement, if marked as such (#293) - COPR - gpg check on vs off (#294) - Access rights for new product are not sufficient for releases - Bug when trying to add new members to product (#295) - Manager should be able to assign roles to team members (It works) :heavy_check_mark: - Prevent builds scheduler to not be overwhelmed by community builds (with configurable rate limits) (#296) - This is about ensuring that there are always available builders for AlmaLinux - Filter by product in build list (save to cookie) - "only my products"? (#297) - This is about having a user preference (like `filterByProductId`) stored in the browser and is taken into account when listing builds - UX: make it clear when you release something in product that is not built/added in product. You need to do it through "releases" menu (its about Add to product button). - The only way to release stuff should be using the release interface - Warn the person doing a release that what you're about to release does not belong to the product it was built for (if it's the case) (#299) - UX: Creating new product is not really intuitive - After we make the improvements around products, we need to update the product creation accordingly. - Needs to be discussed, scoped and described in an issue after most of the items on this milestone are done - UX: Creating release is not intuitive (need to hit "+" before hitting "submit") - Andrew suggests that, when there's only one build, clicking submit should work. When several builds are involved, you need to click +. - Vasily suggests to allow to just hit enter Limitations: * No rollout release capabilities for custom products - but we don't really think we need them in ALBS Things to check: * Ensure we sign COPR repodata - if not do it * if we're re-signing the repo in case a COPR repo gets a new key, we can't forget repodata
No due date•18/28 issues closed