[Internal]: Incomplete Artifact List in Air-Gapped Deployment Guide

[jeanfabrice]

Description

Hi

The "Host your own artifact registry for binary downloads" guide at https://www.elastic.co/docs/reference/fleet/air-gapped#host-artifact-registry provides an example of artifacts to download and mirror internally. However, the current list is limited to the linux-x86_64 architecture and package type.

While the guide does mention downloading artifacts for various arch-package-type combinations (like deb, rpm, etc.) and references https://www.elastic.co/downloads/elastic-agent for verifying URLs, crucial information is missing. Specifically, it's not always clear which artifacts are available for all arch-package-type combinations, making it very challenging for users to assemble a comprehensive and exhaustive list of all possible artifacts to mirror.

To improve the user experience, I suggest that this page either provides a maintained and exhaustive list – or a direct link to such a list – of all possible cURL commands needed to mirror all available artifacts.

Resources

This rudimentary script shell is an attempt to establish such a list:

#!/bin/sh

VERSION="${1:-9.0.0}"

ARCHITECTURES="linux-x86_64 linux-arm64 windows-x86_64 darwin-x86_64 darwin-aarch64"

COMPONENTS="apm-server apm-server
auditbeat beats/auditbeat
elastic-agent beats/elastic-agent
filebeat beats/filebeat
heartbeat beats/heartbeat
metricbeat beats/metricbeat
osquerybeat beats/osquerybeat
packetbeat beats/packetbeat
cloudbeat cloudbeat
endpoint-security endpoint-dev
fleet-server fleet-server
pf-host-agent prodfiler
pf-elastic-collector prodfiler
pf-elastic-symbolizer prodfiler"

WINDOWS_EXCLUSIONS="apm-server osquerybeat fleet-server pf-host-agent pf-elastic-collector pf-elastic-symbolizer cloudbeat endpoint-security"
DEB_EXCLUSIONS="fleet-server endpoint-security osquerybeat cloudbeat"
RPM_EXCLUSIONS="fleet-server endpoint-security osquerybeat cloudbeat"
DARWIN_EXCLUSIONS="pf-host-agent pf-elastic-symbolizer pf-elastic-collector cloudbeat apm-server"

echo "# curl commands for Elastic $VERSION (all architectures)"
echo

for ARCH in $ARCHITECTURES; do
  echo "# Architecture: $ARCH"

  echo "$COMPONENTS" | while read COMPONENT PATH; do
    # Skip if Windows and component is excluded
    SKIP=no
    case "$ARCH" in
      windows-*)
        for EXCL in $WINDOWS_EXCLUSIONS; do
          [ "$COMPONENT" = "$EXCL" ] && SKIP=yes && break
        done
        ;;
      darwin-*)
        for EXCL in $DARWIN_EXCLUSIONS; do
          [ "$COMPONENT" = "$EXCL" ] && SKIP=yes && break
        done
        ;;
    esac
    [ "$SKIP" = "yes" ] && continue

    BASE_URL="https://artifacts.elastic.co/downloads/${PATH}"

    case "$ARCH" in
      windows-*)
        FILE="${COMPONENT}-${VERSION}-${ARCH}.msi"
        echo "curl -O ${BASE_URL}/${FILE}"
        echo "curl -O ${BASE_URL}/${FILE}.sha512"
        echo "curl -O ${BASE_URL}/${FILE}.asc"
        ;;

      darwin-*)
        FILE="${COMPONENT}-${VERSION}-${ARCH}.tar.gz"
        echo "curl -O ${BASE_URL}/${FILE}"
        echo "curl -O ${BASE_URL}/${FILE}.sha512"
        echo "curl -O ${BASE_URL}/${FILE}.asc"
        ;;

      linux-*)
        # .tar.gz package
        FILE_TAR="${COMPONENT}-${VERSION}-${ARCH}.tar.gz"
        echo "curl -O ${BASE_URL}/${FILE_TAR}"
        echo "curl -O ${BASE_URL}/${FILE_TAR}.sha512"
        echo "curl -O ${BASE_URL}/${FILE_TAR}.asc"

        # .deb package (if not excluded)
        for EXCL in $DEB_EXCLUSIONS; do
          [ "$COMPONENT" = "$EXCL" ] && SKIP_DEB=yes && break
        done
        [ "$SKIP_DEB" != "yes" ] && {
          case "$ARCH" in
            linux-x86_64) DEB_ARCH="amd64" ;;
            linux-arm64)  DEB_ARCH="arm64" ;;
          esac
          FILE_DEB="${COMPONENT}-${VERSION}-${DEB_ARCH}.deb"
          echo "curl -O ${BASE_URL}/${FILE_DEB}"
          echo "curl -O ${BASE_URL}/${FILE_DEB}.sha512"
          echo "curl -O ${BASE_URL}/${FILE_DEB}.asc"
        }
        SKIP_DEB=

        # .rpm package (if not excluded)
        for EXCL in $RPM_EXCLUSIONS; do
          [ "$COMPONENT" = "$EXCL" ] && SKIP_RPM=yes && break
        done
        [ "$SKIP_RPM" != "yes" ] && {
          case "$ARCH" in
            linux-x86_64) RPM_ARCH="x86_64" ;;
            linux-arm64)  RPM_ARCH="aarch64" ;;
          esac
          FILE_RPM="${COMPONENT}-${VERSION}-${RPM_ARCH}.rpm"
          echo "curl -O ${BASE_URL}/${FILE_RPM}"
          echo "curl -O ${BASE_URL}/${FILE_RPM}.sha512"
          echo "curl -O ${BASE_URL}/${FILE_RPM}.asc"
        }
        SKIP_RPM=
        ;;
    esac
  done

  echo
done

Unfortunately, I cannot confirm it is complete or exhaustive. It is based on trials and failures while attempting to download the artifacts, based on the publicly available informations.

Which documentation set does this change impact?

Elastic On-Prem only

Feature differences

I believe this is limited to on-prem airgapped environment

What release is this request related to?

N/A

Serverless release

n/a

Collaboration model

The documentation team

Point of contact.

Main contact: @jeanfabrice