|
| 1 | += Additional considerations |
| 2 | +:page-noindex: true |
| 3 | +:page-platform: Server v4.1, Server Admin |
| 4 | +:page-description: This page presents some items that should be considered when starting an air-gapped installation of CircleCI server v4.1. |
| 5 | +:icons: font |
| 6 | +:experimental: |
| 7 | + |
| 8 | +[#non-tls-docker-registry-installations] |
| 9 | +== Non-TLS Docker registry installations |
| 10 | + |
| 11 | +When configuring your air-gapped Docker registry, it is recommended to use TLS certificates to encrypt traffic. If using a non-TLS, or self-signed installation, the following additional steps will need to be taken. |
| 12 | + |
| 13 | +On machines that access the Docker registry using Docker, the Docker daemon config must be updated (located on Linux at `/etc/docker/daemon.json`). |
| 14 | + |
| 15 | +The insecure-registries section must be added to the file (if it exists), or the file must be created with the following if it does not. Make sure to include the full hostname and port of your registry, but do not include the protocol (`http://` or `https://`). |
| 16 | + |
| 17 | +[source, json] |
| 18 | +---- |
| 19 | +{ |
| 20 | + "insecure-registries":["docker.example.internal:5000"] |
| 21 | +} |
| 22 | +---- |
| 23 | + |
| 24 | +This file will need to be configured on the following machines: |
| 25 | + |
| 26 | +- All Nomad nodes in the air-gapped environment |
| 27 | +- Potentially all K3s nodes in the air-gapped environment, if using Docker-backed K3s |
| 28 | +
|
| 29 | +In addition, on each K3s node, the following file must be configured at `/etc/rancher/k3s/registries.yaml`. Take note to include the protocol where referenced. |
| 30 | + |
| 31 | +[source, yaml] |
| 32 | +---- |
| 33 | +mirrors: |
| 34 | + "docker.example.internal:5000": |
| 35 | + endpoint: |
| 36 | + - "http://docker.example.internal:5000" |
| 37 | +configs: |
| 38 | + "docker.example.internal:5000": |
| 39 | + tls: |
| 40 | + insecure_skip_verify: true |
| 41 | +---- |
| 42 | + |
| 43 | +--- |
| 44 | + |
| 45 | + |
| 46 | + |
| 47 | +[#service-type-load-balancers-k3s] |
| 48 | +== Service type load balancers in K3s |
| 49 | + |
| 50 | +CircleCI server makes use of Service Type: Load Balancer Kubernetes resources to listen to traffic on multiple ports. In order to function, the cluster needs to. |
| 51 | + |
| 52 | +If using a K3s installation, link:https://metallb.universe.tf/installation/[MetalLB] can be used to create a virtual load balancer on the K3s node, to allow ingress traffic to CircleCI server. |
| 53 | + |
| 54 | +Once installed, the following steps need to be followed: |
| 55 | + |
| 56 | +A ConfigMap resource needs to be created to create an address pool for MetalLB. |
| 57 | + |
| 58 | +[source, yaml] |
| 59 | +---- |
| 60 | +apiVersion: "v1" |
| 61 | +kind: ConfigMap |
| 62 | +metadata: |
| 63 | + namespace: metallb-system |
| 64 | + name: config |
| 65 | +data: |
| 66 | + config: | |
| 67 | + address-pools: |
| 68 | + - name: default |
| 69 | + protocol: layer2 |
| 70 | + addresses: |
| 71 | + - <<k3s_internal_ip_range_start>>-<<k3s_internal_ip_range_end>> |
| 72 | +---- |
| 73 | + |
| 74 | +The address pool can be named something other than "default", but the annotations in `values.yaml` will need to be updated. If there is only one k3s node, the address range should have the same IP repeated (for example, `10.0.0.5-10.0.0.5`). |
| 75 | + |
| 76 | +Once this ConfigMap resource is applied to the cluster (`kubectl apply -f metallb-configmap.yaml`), the address pool name can be updated in the `values.yaml` for the Helm installation. |
| 77 | + |
| 78 | +[source, yaml] |
| 79 | +---- |
| 80 | +# Additional nginx annotations |
| 81 | +nginx: |
| 82 | + annotations: |
| 83 | + # This example uses MetalLB as a k3s load balancer |
| 84 | + metallb.universe.tf/allow-shared-ip: default |
| 85 | +---- |
| 86 | + |
| 87 | +After installing the Helm chart, the circleci-proxy service must be patched to use the internal IP of the desired k3s node to act as the load balancer (this IP should be in the range entered in the ConfigMap above). The example below uses the IP address `10.0.0.5`. |
| 88 | + |
| 89 | +[source, bash] |
| 90 | +---- |
| 91 | +kubectl patch svc circleci-proxy -p '{"spec": {"type": "LoadBalancer", "externalIPs":["10.0.0.5"]}}' |
| 92 | +---- |
| 93 | + |
| 94 | +Once complete, DNS records can be created for your server installation (server.internal.example.com) and (*.server.internal.example.com) for 10.0.0.5. |
| 95 | + |
| 96 | +[#tls-importing] |
| 97 | +== Importing trusted TLS certificates |
| 98 | + |
| 99 | +When using a GitHub Enterprise instance with self-signed or custom certificate authority certificates, CircleCI server can be configured to trust these certificates using two methods, as described below. |
| 100 | + |
| 101 | +NOTE: Values provided for either of these methods are supported for GitHub Enterprise certificates only. No other VCS application is supported at this time. |
| 102 | + |
| 103 | +[#configuring-the-tls-imports-value] |
| 104 | +=== Configuring the TLS imports value |
| 105 | +In the `values.yaml` file, the `tls.imports` value can be configured to include a list of `hostname:port` combinations from which to retrieve and trust TLS certificates. |
| 106 | + |
| 107 | +[source, yaml] |
| 108 | +---- |
| 109 | +tls: |
| 110 | + ... |
| 111 | + import: |
| 112 | + - github.airgap.example.com:443 |
| 113 | +---- |
| 114 | + |
| 115 | +For each `hostname:port` combination, CircleCI server will, during installation, retrieve the public certificate for the particular GitHub Enterprise instance and trust it to establish connections with that instance. |
| 116 | + |
| 117 | + |
| 118 | +[#configuring-the-tls-certificates-array] |
| 119 | +=== Configuring the TLS certificates array |
| 120 | +Instead of providing a list of `hostname:port` combinations for certificates to import, the public certificate chains of the corresponding TLS certificates to trust can be provided in the `values.yaml` file, in the `tls.certificates` value, as a list of base64 encoded certificates strings. |
| 121 | + |
| 122 | +[source, yaml] |
| 123 | +---- |
| 124 | +tls: |
| 125 | + ... |
| 126 | + certificates: |
| 127 | + - <<base64-encoded-public-tls-certificate-chain>> |
| 128 | +---- |
0 commit comments