diff --git a/content/learn/vp_agof.adoc b/content/learn/vp_agof.adoc new file mode 100644 index 000000000..9be958acd --- /dev/null +++ b/content/learn/vp_agof.adoc @@ -0,0 +1,311 @@ +--- +menu: + learn: + parent: Validated patterns frameworks +title: Ansible GitOps Framework +weight: 23 +aliases: /ocp-framework/agof/ +--- + +:toc: +:imagesdir: /images +:_content-type: ASSEMBLY +include::modules/comm-attributes.adoc[] + +== About the Ansible GitOps framework (AGOF)for validated patterns + +The link:/patterns/ansible-gitops-framework/[Ansible GitOps Framework] provides an extensible framework to do GitOps with https://docs.ansible.com/platform.html[Ansible Automation Platform] (AAP). It offers useful facilities for developing patterns (community and validated) that work with AAP as the GitOps engine. + +When there is no access to OpenShift Container Platform, AGOF provides a standalone solution for deploying and managing validated patterns. This framework leverages GitOps principles without relying on OpenShift. + +Administrators can use the Ansible Automation Platform, which includes automation controller, a web-based UI interface to define, operate, scale, and delegate automation across their enterprise. + +The repository at https://github.com/validatedpatterns/agof/[Ansible GitOps Framework] provides code for installing VMs on AWS, if needed. It can also be used with existing VMs or a functional AAP Controller endpoint. + +The link:https://github.com/validatedpatterns/agof/[Ansible GitOps Framework] repository contains code to set up the infrastructure for applying `controller_configuration` to an AAP instance. It includes some predefined configurations and practices to make the infrastructure as code repository more user-friendly and standardized. An AGOF pattern for example link:https://github.com/mhjacks/agof_demo_config[demo] is primarily an IaC infrastructure as code (IaC) artifact designed to be used with the `controller_configuration` collection. + +== Role of the Ansible Controller + +In the Ansible GitOps framework, the Ansible Controller refers to any machine that initiates and runs playbooks. This includes systems running Ansible Core or open source Ansible, where the machine executing the playbooks also acts as the controller. The Ansible Automation Platform (AAP) provides the Automation Controller, which is Red Hat’s productized Ansible Controller. + +The link:https://docs.ansible.com/platform.html[Automation controller] is the command and control center for Red Hat Ansible Automation Platform, replacing Ansible Tower. It includes a web UI, API, role-based access control (RBAC), a workflow visualizer, and continuous integration and continuous delivery (CI/CD) integrations to help you organize and manage automation across your enterprise. + +The Automation Controller serves as the centralized platform for managing and executing automation across infrastructure. Use the Automation Controller to create job templates that standardize the deployment and execution of Ansible playbooks, making automation more consistent and reusable. It integrates essential components such as execution environments for consistent automation execution, projects (repositories for automation content), inventories (target endpoints), and credentials (for secure access to resources). + +The webUI provides an intuitive interface to build, monitor, and manage automation workflows, while the API offers seamless integration with other tools, such as CI/CD pipelines or orchestration platforms. Overall, the Automation Controller streamlines the automation lifecycle, ensuring a scalable, secure, and maintainable automation environment. + +== Ansible framework methods + +The main methods for setting up an Ansible framework are as follows: + +[NOTE] +==== +The focus here is primarily on the AWS-based installation method. +==== + +=== AWS based install + +This method is ideal for organizations deploying AAP on AWS infrastructure. In AAP 2.4, the default installation process uses AWS and offers a fully automated setup. It requires AWS credentials and builds an AWS image with Red Hat’s ImageBuilder, setting up AAP within an AWS VPC and subnet. The installation program creates all necessary resources, including AAP Controllers and, optionally, additional components such as the Automation Hub. + +This is the easiest method if you already use AWS, as it automates the provisioning of resources, including VMs and network configurations. This requires AWS infrastructure and credentials. + +=== Custom Ansible controller (API install) + +In this method, you provide an existing Ansible Automation Platform (AAP) Controller endpoint, either on bare metal or in a private cloud, without needing AWS or pre-configured VMs. You specify the manifest, endpoint hostname, admin credentials, and pass the installation process to a predefined `controller_config_dir`. This is suitable for complex or custom topologies where you want full control over the deployment. + +This method provides maximum flexibility and is designed for advanced users who have their own AAP installations, either on-prem or in complex environments that do not fit into the default or AWS-centric model. You need an existing AAP controller, which might not be ideal for users new to AAP or those looking for more hands-off installation + +== Creating a validated pattern using the AGOF framework + +To deploy a validated pattern, follow these steps: + +. Clone the link:https://github.com/mhjacks/agof_demo_config[Ansible GitOps Framework Minimal Configuration Demo] repository by running the following command: ++ +[source,terminal] +---- +$ git clone git@github.com:mhjacks/agof_demo_config.git +---- ++ +This is a minimal pattern that demonstrates how to use the Ansible GitOps Framework. + +. Clone the link:https://github.com/validatedpatterns/agof[AGOF] repository by running the following command. ++ +[source,terminal] +---- +$ git clone git@github.com:validatedpatterns/agof.git +---- ++ +This serves as the provisioner for the pattern + +=== Deploying using the AWS-based install method + +This method: + +. Builds an AWS image using Red Hat's ImageBuilder. + +. Deploys that image onto a new AWS VPC and subnet. + +. Deploys AAP on that image using the command line installer. + +. Hands over the configuration of the AAP installation to the specified `agof_controller_config_dir`. + +.Prerequisites + +You need to provide some key information to a file named `agof_vault.yml` created in your home directory. The key pieces of information needed are: + +* The name of a hosted zone created under Route 53 in AWS. For example this could be `aws.validatedpatterns.io`. + +* Your AWS account `Account ID` + +* Your `aws key` + +* Your `secret access key` + +.Procedure + +. Copy the `agof_vault_template.yml` from the cloned `agof` directory to your home directory and rename it `agof_vault.yml`. ++ +[source,terminal] +---- +$ cp ~/agof/agof_vault_template.yml ~/agof_vault.yml +---- ++ +.Agof vault settings ++ +[cols="30%,70%", options="header"] +|=== +| Argument | Description + +| `aws_account_nbr_vault` +| Your AWS account number. This is needed for sharing composer images. + +| `aws_access_key_vault` +| Your AWS access key. + +| `aws_secret_key_vault` +| Your AWS secret key. + +| `pattern_prefix` +| A unique prefix to distinguish instances in AWS. Used as the pattern name and in public DNS entries. + +| `ec2_region` +| An AWS region that your account has access to. + +| `offline_token` +a| A Red Hat offline token used to build the RHEL image on https://console.redhat.com[console.redhat.com]. + +[NOTE] +==== +Click the `GENERATE TOKEN` link at https://access.redhat.com/management/api[Red Hat API Tokens] to create the token. +==== + +| `redhat_username` +| Red Hat Subscription username, used to log in to https://registry.redhat.io[registry.redhat.io]. + +| `redhat_password` +| Red Hat Subscription password, used to log in to https://registry.redhat.io[registry.redhat.io]. + +| `admin_password` +| An admin password for AAP Controller and Automation Hub. + +| `manifest_content` +| Content for a manifest file to entitle AAP Controller. See below for an example of how to point to a local file. + +| `org_number_vault` +| The Organization Number (Org ID) attached to your Red Hat Subscription for RHEL and AAP. + +| `activation_key_vault` +a| The name of an Activation Key to embed in the imagebuilder image. + +[NOTE] +==== +Click the `Create Activation Keys` link at https://console.redhat.com[console.redhat.com] > *Red Hat Enterprise Linux* > *Inventory* > *System Configuration* > *Activation Keys* to create an activation key. +==== + +| `skip_imagebuilder_build` +| Set these variables to provide your own AMI, or to re-use an AMI previously generated with this process. When a previously built AMI is found this check does not take place. +#imagebuilder_ami: 'The ID of an AWS AMI image, preferably one that was built with this toolkit' + +| `automation_hub_token_vault` +a| A token associated with your AAP subscription used to retrieve Automation Hub content. + +[NOTE] +==== +Click the `Load token` link at https://console.redhat.com[console.redhat.com] > *Ansible Automation Platform* > *Automation Hub* > *Connect to Hub* to generate a token. +==== + +| `automation_hub_url_certified_vault` +| Optional: The private automation hub URL for certified content. + +| `automation_hub_url_validated_vault` +| Optional: The private automation hub URL for validated content. +|=== + +. Edit the file and add the following: + +* `agof_controller_config_dir:` set it's value to `{{ '~/agof_minimal_demo' | expanduser }}`. +* `db_password:` sets an appropriate value for the postgres password for the DB instance for example `test`. +* `agof_statedir:` set its value to `"{{ '~/agof' | expanduser }}"` +* `agof_iac_repo:` set its value to `"https://github.com/mhjacks/agof_demo_config.git"` + +. Optional: Create a subscription manifest by following the guidance at link:https://docs.redhat.com/en/documentation/red_hat_ansible_automation_platform/2.4/htmlred_hat_ansible_automation_platform_operations_guide/assembly-aap-obtain-manifest-files#assembly-aap-obtain-manifest-files[Obtaining a manifest file]. + +.. Update your `agof_vault.yml` file with the path to the downloaded manifest zip file. For example add the following: ++ +[source,shell] +---- +controller_license_src_file: '~/Downloads/.zip' +manifest_content: "{{ lookup('file', controller_license_src_file) | b64encode }}" +---- ++ +.Example agof_vault.yml file + +[source,yaml] +---- +--- +aws_account_nbr_vault: '' +aws_access_key_vault: '' +aws_secret_key_vault: '' + +pattern_prefix: 'foga' +pattern_dns_zone: 'aws.validatedpatterns.io' + +ec2_name_prefix: 'foga-testing' +ec2_region: 'us-east-1' + +offline_token: '' + +redhat_username: 'rhn-support-myusername' +redhat_password: 'passwd01' + +admin_password: 'redhat123!' + +manifest_content: "Content for a manifest file to entitle AAP Controller. See below for an example of how to point to a local file" +#manifest_content: "{{ lookup('file', '~/Downloads/manifest_AVP_20230510T202608Z.zip') | b64encode }}" + +org_number_vault: "" +activation_key_vault: "kevs-agof-key" + +# Set these variables to provide your own AMI, or to re-use an AMI previously generated with this process +#skip_imagebuilder_build: 'boolean: skips imagebuilder build process' +#imagebuilder_ami: 'The ID of an AWS AMI image, preferably one that was built with this toolkit' + +automation_hub_token_vault: '' + +# These variables can be set but are optional. The previous (before AAP 2.4) conncept of sync-list was private +# to an account. +#automation_hub_url_certified_vault: 'The private automation hub URL for certified content' +#automation_hub_url_validated_vault: 'The private automation hub URL for validated content' + +controller_config_dir: "{{ '~/agof_minimal_demo' | expanduser }}" + +db_password: 'test' + +agof_statedir: "{{ '~/agof' | expanduser }}" + +agof_iac_repo: "https://github.com/mhjacks/agof_demo_config.git" +---- + +. Run the following command from the AGOF repository directory: ++ +[source,terminal] +---- +$ ./pattern.sh make install +---- +This command invokes the `controller_configuration` `dispatch` role on the controller endpoint based on the configuration found in the `controller_configuration_dir` and in your `agof_vault.yml` file. This controls all of the controller configuration. Based on the variables defined in the controller configuration, `dispatch` calls the necessary roles and modules in the right order to configure AAP to run your pattern. + +.Verification + +The default installation provides an AAP 2.4 installation deployed using the containerized installer, with services deployed this way: + +.agof_vault settings +[cols="30%,70%",options="header"] +|=== +| URL | Service + +| `https:{{ ec2_name_prefix }}.{{ domain }}:8443` +a| Controller API. + +| `https:{{ ec2_name_prefix }}.{{ domain }}:8444/` +a|Private Automation Hub + +| `https:/{{ ec2_name_prefix }}.{{ domain }}:8445/`` +a| EDA Automation Controller + +|=== + +Once the install completes, you will have a project, an inventory (consisting of the AAP controller), a credential (the private key from ec2), a job template (which runs a fact gather on the AAP controller) and a schedule that will run the job template every 5 minutes, + +. Log in to `https://aap.{{ ec2_name_prefix }}.{{ domain }}:8443` with the username `admin` and the password as configured in `admin_password` field of `agof_vault.yml`. + +. Under *Resources* > *Projects* verify the project *Ansible GitOps Framework Minimal Demo* is created with status *Successful*. + +. Under *Resources* > *Inventories* verify the inventory *AGOF Demo Inventory* is created with sync status *Success*. + +. Under *Resources* > *Templates* verify the job template *Ping Playbook* is created. + +. Under *Resources* > *Credentials* verify the ec2 ssh credential *ec2_ssh_credential* is created. + +. Under *Views* > *Schedules* verify the schedules are created. + +=== Custom Ansible controller (API install) + +In this method, you provide an existing Ansible Automation Platform (AAP) Controller endpoint, either on bare metal or in a private cloud, without needing AWS or pre-configured VMs. + +You supply the manifest contents, endpoint hostname, admin username (defaults to "admin"), and admin password, and then the installation hands off to a `controller_config_dir` you define. + +* Run the following command to install using this method: ++ +[source,terminal] +---- +$ ./pattern.sh make api_install +---- + +=== Tearing down the installation + +* To tear down the installation run the following command: ++ +[source,terminal] +---- +$ ./pattern.sh make aws_uninstall +---- \ No newline at end of file diff --git a/content/learn/vp_agof_config_controller.adoc b/content/learn/vp_agof_config_controller.adoc new file mode 100644 index 000000000..c2c45cc0b --- /dev/null +++ b/content/learn/vp_agof_config_controller.adoc @@ -0,0 +1,223 @@ +--- +menu: + learn: + parent: Validated patterns frameworks +title: Using the Controller Configuration collection +weight: 25 +aliases: /ocp-framework/agof/ +--- + +:toc: +:imagesdir: /images +:_content-type: ASSEMBLY +include::modules/comm-attributes.adoc[] + +== Overview of the Ansible GitOps Framework (AGOF) Installation Process + +The Ansible GitOps Framework (AGOF) is a powerful solution designed to automate the deployment and configuration of Ansible Automation Platform (AAP) environments using link:https://opengitops.dev/[GitOps principles]. It leverages Ansible to manage infrastructure and application provisioning in a declarative, version-controlled way. AGOF provides a structured approach to setting up cloud infrastructure, installing AAP components, and handing over control to the AAP Controller for ongoing automation and management. An overview of the steps involved in configuring a basic demo minimal demo application are listed here: + +=== 1. Pre-Init Environment (Bootstrap Ansible) + +* *Ansible Configuration*: The environment is initialized by generating an `ansible.cfg` file, which is configured with Automation Hub and public Galaxy endpoints. This process includes vault configuration to inject the Automation Hub token and install required Ansible collections from `requirements.yml`. +* *Optional Image Build*: Images are created using Red Hat Image Builder to produce AMIs, which include `cloud-init`, activation keys, and organization details. These images can be reused in future installations. + +=== 2. Infrastructure Building (AWS Only) + +* *AWS Setup*: The framework sets up AWS infrastructure, including VPC, subnets, and security groups, using predefined roles. It also manages Route53 DNS entries for VMs. +* *VM Deployment*: Virtual machines are provisioned on EC2 with persistent hostnames and updates to `/etc/hosts` for AWS nodes. DNS entries are updated when IPs change after VM reboots. + +=== 3. Handover to Ansible Controller + +* *Controller Setup*: The Ansible Automation Platform (AAP) Controller and optionally the Automation Hub are installed and configured. Entitlements are managed through a manifest, and execution environments and collections are downloaded and prepared. +* *GitOps Mode*: After configuration, AGOF transitions to GitOps mode. GGit commits made by the controller to the repositories manage all environment changes, ensuring declarative and automated infrastructure management from this point onward. + +== Controller configuration collection + +An AGOF pattern, for example, https://github.com/mhjacks/agof_demo_config is primarily an IaC (infrastructure as code) artifact designed to be used with the `controller_configuration` collections. + +The AGOF (Ansible GitOps Framework) repository https://github.com/validatedpatterns/agof contains the code and tools needed to set up a new Ansible Automation Platform (AAP) instance. This setup is automated, using Infrastructure as Code (IaC) practices. It also includes some specific preferences to make it easier for others to publicly share and manage this type of infrastructure setup. + +This approach ensures the automation controller configuration is version-controlled, dynamic, and reproducible. This method enables deployment automation with minimal manual intervention, which is useful for managing multiple controller instances or different environments in a CI/CD pipeline. + +For example, for the AGOF minimal configuration demo the file https://github.com/mhjacks/agof_demo_config/blob/main/controller_config.yml is used with Ansible's Controller Configuration Collection, allowing the automation and management of Red Hat Ansible Automation Controller (formerly known as Ansible Tower). + +[source,yaml] +---- +# vim: ft=yaml.ansible +--- +orgname_vault: 'Demo Organization' + +controller_username_vault: 'admin' +controller_password_vault: '{{ admin_password }}' + +controller_username: '{{ controller_username_vault }}' +controller_password: '{{ controller_password_vault }}' + +agof_demo_project_name: 'Ansible GitOps Framework Minimal Demo' + +controller_validate_certs: false + +controller_configuration_async_retries: 30 + +controller_settings: [] + +controller_projects: + - name: Demo Project + state: absent + + - name: '{{ agof_demo_project_name }}' + organization: "{{ orgname_vault }}" + scm_branch: main + scm_clean: "no" + scm_delete_on_update: "no" + scm_type: git + scm_update_on_launch: "yes" + scm_url: 'https://github.com/validatedpatterns-demos/agof_minimal_demo.git' + +controller_organizations: + - name: '{{ orgname_vault }}' + +controller_inventories: + - name: 'AGOF Demo Inventory' + organization: '{{ orgname_vault }}' + +controller_inventory_sources: + - name: 'AGOF Demo Inventory Source' + inventory: 'AGOF Demo Inventory' + credential: 'ec2_ssh_credential' + overwrite: true + overwrite_vars: true + update_on_launch: true + source: scm + source_project: '{{ agof_demo_project_name }}' + source_path: 'inventory' + +controller_credential_types: [] + +controller_templates: + - name: Demo Job Template + state: absent + + - name: Ping Playbook + organization: "{{ orgname_vault }}" + project: '{{ agof_demo_project_name }}' + job_type: run + playbook: 'ansible/playbooks/ping.yml' + inventory: "AGOF Demo Inventory" + credentials: + - ec2_ssh_credential + +controller_schedules: + - name: Ping Playbook + organization: "{{ orgname_vault }}" + unified_job_template: Ping Playbook + rrule: DTSTART:20191219T130500Z RRULE:FREQ=MINUTELY;INTERVAL=120 + +demo_ssh_key_file: '~/{{ ec2_name_prefix }}/{{ ec2_name_prefix }}-private.pem' + +controller_credentials: + - name: ec2_ssh_credential + description: "EC2 SSH credential" + organization: '{{ orgname_vault }}' + credential_type: Machine + inputs: + username: 'ec2-user' + ssh_key_data: "{{ lookup('file', demo_ssh_key_file) }}" + become_method: sudo + +controller_launch_jobs: + - name: Ping Playbook + organization: "{{ orgname_vault }}" +---- + +This file automates the creation, updating, or deletion of Ansible Controller objects (organizations, projects, inventories, credentials, templates, schedules). Sensitive information like passwords and keys are pulled dynamically from vaults, ensuring they are not hardcoded in the configuration. + +A Git repository manages the project’s inventory and playbooks, allowing for continuous integration and delivery (CI/CD) practices. AAP automatically schedules recurring playbook executions, eliminating the need for manual job triggers. + +== Key sections and parameters + +This section describes the parameters associated with the Ansible GitOps Framework minimal configuration demo. + +=== Vault variables + +*`orgname_vault: 'Demo Organization'`*:: +This specifies the organization name stored in a vault for security purposes. + +*`controller_username_vault: 'admin'`*:: +This is the Ansible Controller's username stored in a vault. + +*`controller_password_vault: '{{ admin_password }}'`*:: +The initial admin password that AAP is configured with to allow the controller_username to log in. This particular password is not retrieved from a vault. + +=== Dynamic variables + +*`controller_username: '{{ controller_username_vault }}'`*:: +The Ansible Controller username is retrieved from the vault variable. + +*`controller_password: '{{ controller_password_vault }}'`*:: +The password is dynamically fetched from the vault. + +=== Project configuration +Projects are git repositories that can contain inventories and collections (and collections can contain playbooks). + +*`agof_demo_project_name: 'Ansible GitOps Framework Minimal Demo'`*:: +This variable holds the name of the project being managed in the controller. + +*`controller_projects`*:: +Two projects are defined: + * One with the name *'Demo Project'*, marked for deletion (`state: absent`). + * The other is the actual project that will be created, associated with the Git repository hosted on GitHub. + +For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/projects/README.md[controller_configuration.projects]. + +=== Organizations + +Organizations represent a logical grouping for managing resources such as projects and inventories. + +*`controller_organizations`*:: +Ensures that the organization, defined in `orgname_vault`, exists within the controller. For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/organizations/README.md[controller_configuration.organizations]. + +=== Inventory and inventory sources + +*`controller_inventories`*:: +Defines an inventory called *'AGOF Demo Inventory'* under the *'Demo Organization'*. For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/inventories/README.md[controller_configuration.inventories]. + + +*`controller_inventory_sources`*:: +Configures an inventory source tied to the Git project. The inventory is pulled from source control management (SCM) and associated with credentials for SSH access (`ec2_ssh_credential`). For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/inventory_sources/README.md[controller_configuration.inventory_sources]. + +=== Job templates + +Job Templates define a specific playbook run, associating it with inventories, credentials, and other settings. + +*`controller_templates`*:: +Two job templates are managed: + * One named *'Demo Job Template'*, marked for deletion. + * The other, *'Ping Playbook'*, is tied to a specific playbook (`ping.yml`), inventory, and project, and will use the defined credentials for execution. + +For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/job_templates/README.md[controller_configuration.job_templates]. + +=== Job scheduling + +*`controller_schedules`*:: +Configures a recurring job schedule to run the *'Ping Playbook'* template every 120 minutes. The schedule uses an iCal `RRULE` format. For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/schedules/README.md[controller_configuration.schedules]. + +=== Credentials + +Credentials store authentication details for accessing external systems like clouds, networks, and SCMs. + +*`controller_credentials`*:: +A credential named *'ec2_ssh_credential'* is created with SSH access to the EC2 instances using the private key stored at the path specified in `demo_ssh_key_file`. For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/credentials/README.md[controller_configuration.credentials]. + +=== Job Launching + +*`controller_launch_jobs`*:: +Automatically launches the *'Ping Playbook'* job template within the organization defined in `orgname_vault`. For more information see, link:https://github.com/redhat-cop/controller_configuration/blob/devel/roles/job_launch/README.md[controller_configuration.job_launch]. + +For more information about the controller configuration see: + +* link:https://docs.redhat.com/en/documentation/red_hat_ansible_automation_platform/2.4/html-single/automation_controller_administration_guide/index[Red Hat Automation Controller Admin Guide] + +* link:https://github.com/redhat-cop/controller_configuration[Red Hat Communities of Practice Controller Configuration Collection] + +* link:https://galaxy.ansible.com/ui/repo/published/infra/controller_configuration/docs/[Galaxy Red Hat Communities of Practice Controller Configuration Collection documentation]