Skip to content

Commit 90e854a

Browse files
shainaraskasshainaraskas
committed
air gapped
1 parent a182f40 commit 90e854a

File tree

1 file changed

+256
-40
lines changed

1 file changed

+256
-40
lines changed

‎deploy-manage/deploy/self-managed/air-gapped-install.md‎

Lines changed: 256 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -2,85 +2,301 @@
22
mapped_urls:
33
- https://www.elastic.co/guide/en/elastic-stack/current/air-gapped-install.html
44
- https://www.elastic.co/guide/en/cloud-enterprise/current/ece-install-offline.html
5+
applies_to:
6+
deployment:
7+
self:
8+
sub:
9+
stack-version: "9.0.0"
510
---
611

712
# Air gapped install
813

9-
% What needs to be done: Refine
14+
Some components of the {{stack}} require additional configuration and local dependencies in order to deploy in environments without internet access. This guide gives an overview of this setup scenario and helps bridge together existing documentation for individual parts of the stack.
1015

11-
% GitHub issue: https://github.com/elastic/docs-projects/issues/309
16+
Refer to the section for each Elastic component for air-gapped installation configuration and dependencies in a self-managed Linux environment.
1217

13-
% Use migrated content from existing pages that map to this page:
18+
## {{es}} [air-gapped-elasticsearch]
1419

15-
% - [ ] ./raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md
16-
% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md
20+
Air-gapped install of {{es}} may require additional steps in order to access some of the features. General install and configuration guides are available in the [{{es}} install documentation](/deploy-manage/deploy/self-managed/installing-elasticsearch.md).
1721

18-
% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):
22+
Specifically:
1923

20-
$$$air-gapped-self-managed-linux$$$
24+
* To be able to use the GeoIP processor, refer to [the GeoIP processor documentation](asciidocalypse://docs/elasticsearch/docs/reference/ingestion-tools/enrich-processor/geoip-processor.md#manually-update-geoip-databases) for instructions on downloading and deploying the required databases.
25+
* Refer to [{{ml-cap}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-machine-learning) for instructions on deploying the Elastic Learned Sparse EncodeR (ELSER) natural language processing (NLP) model and other trained {{ml}} models.
2126

22-
$$$air-gapped-elasticsearch$$$
2327

24-
$$$air-gapped-kibana$$$
28+
## {{kib}} [air-gapped-kibana]
2529

26-
$$$air-gapped-beats$$$
30+
Air-gapped install of {{kib}} may require a number of additional services in the local network in order to access some of the features. General install and configuration guides are available in the [{{kib}} install documentation](/deploy-manage/deploy/self-managed/install-kibana.md).
2731

28-
$$$air-gapped-logstash$$$
32+
Specifically:
2933

30-
$$$air-gapped-elastic-agent$$$
34+
* To be able to use {{kib}} mapping visualizations, you need to set up and configure the [Elastic Maps Service](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-maps-service).
35+
* To be able to use {{kib}} sample data, install or update hundreds of prebuilt alert rules, and explore available data integrations, you need to set up and configure the [{{package-registry}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-package-registry).
36+
* To provide detection rule updates for {{endpoint-sec}} agents, you need to set up and configure the [Elastic Endpoint Artifact Repository](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-endpoint-artifact-repository).
37+
* To access the APM integration, you need to set up and configure [Elastic APM](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-apm).
38+
* To install and use the Elastic documentation for {{kib}} AI assistants, you need to set up and configure the [Elastic product documentation for {{kib}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-kibana-product-documentation).
3139

32-
$$$air-gapped-fleet$$$
3340

34-
$$$air-gapped-elastic-apm$$$
41+
## {{beats}} [air-gapped-beats]
3542

36-
$$$air-gapped-elastic-maps-service$$$
43+
Elastic {{beats}} are light-weight data shippers. They do not require any unique setup in the air-gapped scenario. To learn more, refer to the [{{beats}} documentation](asciidocalypse://docs/beats/docs/reference/index.md).
3744

38-
$$$air-gapped-enterprise-search$$$
3945

40-
$$$air-gapped-elastic-package-registry$$$
46+
## {{ls}} [air-gapped-logstash]
4147

42-
$$$air-gapped-elastic-artifact-registry$$$
48+
{{ls}} is a versatile data shipping and processing application. It does not require any unique setup in the air-gapped scenario. To learn more, refer to the [{{ls}} documentation](asciidocalypse://docs/logstash/docs/reference/index.md).
4349

44-
$$$air-gapped-elastic-endpoint-artifact-repository$$$
4550

46-
$$$air-gapped-machine-learning$$$
51+
## {{agent}} [air-gapped-elastic-agent]
4752

48-
$$$air-gapped-kubernetes-and-openshift$$$
53+
Air-gapped install of {{agent}} depends on the [{{package-registry}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-package-registry) and the [{{artifact-registry}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry) for most use-cases. The agent itself is fairly lightweight and installs dependencies only as required by its configuration. In terms of connections to these dependencies, {{agents}} need to be able to connect to the {{artifact-registry}} directly, but {{package-registry}} connections are handled through [{{kib}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-kibana).
4954

50-
$$$air-gapped-k8s-os-elastic-kubernetes-operator$$$
55+
Additionally, if the {{agent}} {{elastic-defend}} integration is used, then access to the [Elastic Endpoint Artifact Repository](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-endpoint-artifact-repository) is necessary in order to deploy updates for some of the detection and prevention capabilities.
5156

52-
$$$air-gapped-k8s-os-elastic-package-registry$$$
57+
To learn more about install and configuration, refer to the [{{agent}} install documentation](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/install-elastic-agents.md). Make sure to check the requirements specific to running {{agents}} in an [air-gapped environment](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/air-gapped.md).
5358

54-
$$$air-gapped-k8s-os-elastic-artifact-registry$$$
59+
To get a better understanding of how to work with {{agent}} configuration settings and policies, refer to [Appendix D - Agent Integration Guide](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-agent-integration-guide).
5560

56-
$$$air-gapped-k8s-os-elastic-endpoint-artifact-repository$$$
5761

58-
$$$air-gapped-k8s-os-ironbank-secure-images$$$
62+
## {{fleet-server}} [air-gapped-fleet]
5963

60-
$$$air-gapped-ece$$$
64+
{{fleet-server}} is a required middleware component for any scalable deployment of the {{agent}}. The air-gapped dependencies of {{fleet-server}} are the same as those of the [{{agent}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-agent).
6165

62-
$$$air-gapped-elastic-package-registry-example$$$
66+
To learn more about installing {{fleet-server}}, refer to the [{{fleet-server}} set up documentation](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/fleet-server.md).
6367

64-
$$$air-gapped-elastic-artifact-registry-example$$$
6568

66-
$$$air-gapped-epr-kubernetes-example$$$
69+
## Elastic APM [air-gapped-elastic-apm]
6770

68-
$$$air-gapped-agent-integration-guide$$$
71+
Air-gapped setup of the APM server is possible in two ways:
6972

70-
$$$air-gapped-agent-integration-terminology$$$
73+
* By setting up one of the {{agent}} deployments with an APM integration, as described in [Switch a self-installation to the APM integration](/solutions/observability/apps/switch-self-installation-to-apm-integration.md).
74+
* Or, by installing a standalone Elastic APM Server, as described in the [APM configuration documentation](/solutions/observability/apps/configure-apm-server.md).
7175

72-
$$$air-gapped-agent-integration-configure$$$
7376

74-
$$$air-gapped-agent-integration-configure-kibana$$$
77+
## {{ems}} [air-gapped-elastic-maps-service]
7578

76-
$$$air-gapped-agent-integration-configure-yml$$$
79+
Refer to [Connect to {{ems}}](../../../explore-analyze/visualize/maps/maps-connect-to-ems.md) in the {{kib}} documentation to learn how to configure your firewall to connect to {{ems}}, host it locally, or disable it completely.
7780

78-
$$$air-gapped-agent-integration-configure-fleet-api$$$
7981

80-
$$$air-gapped-kibana-product-documentation$$$
82+
## {{package-registry}} [air-gapped-elastic-package-registry]
8183

84+
Air-gapped install of the EPR is possible using any OCI-compatible runtime like Podman (a typical choice for RHEL-like Linux systems) or Docker. Links to the official container image and usage guide is available on the [Air-gapped environments](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/air-gapped.md) page in the {{fleet}} and {{agent}} Guide.
8285

83-
**This page is a work in progress.** The documentation team is working to combine content pulled from the following pages:
86+
::::{note}
87+
Besides setting up the EPR service, you also need to [configure {{kib}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-kibana) to use this service. If using TLS with the EPR service, it is also necessary to set up {{kib}} to trust the certificate presented by the EPR.
88+
::::
8489

85-
* [/raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md](/raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md)
86-
* [/raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md](/raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md)
90+
### Additional {{package-registry}} examples
91+
92+
:::{dropdown} Script to generate a SystemD service file on a RHEL 8 system
93+
94+
The following script generates a SystemD service file on a RHEL 8 system in order to run EPR with Podman in a production environment.
95+
96+
```sh
97+
#!/usr/bin/env bash
98+
99+
EPR_BIND_ADDRESS="0.0.0.0"
100+
EPR_BIND_PORT="8443"
101+
EPR_TLS_CERT="/etc/elastic/epr/epr.pem"
102+
EPR_TLS_KEY="/etc/elastic/epr/epr-key.pem"
103+
EPR_IMAGE="docker.elastic.co/package-registry/distribution:{{stack-version}}"
104+
105+
podman create \
106+
--name "elastic-epr" \
107+
-p "$EPR_BIND_ADDRESS:$EPR_BIND_PORT:$EPR_BIND_PORT" \
108+
-v "$EPR_TLS_CERT:/etc/ssl/epr.crt:ro" \
109+
-v "$EPR_TLS_KEY:/etc/ssl/epr.key:ro" \
110+
-e "EPR_ADDRESS=0.0.0.0:$EPR_BIND_PORT" \
111+
-e "EPR_TLS_CERT=/etc/ssl/epr.crt" \
112+
-e "EPR_TLS_KEY=/etc/ssl/epr.key" \
113+
"$EPR_IMAGE"
114+
115+
## creates service file in the root directory
116+
# podman generate systemd --new --files --name elastic-epr --restart-policy always
117+
```
118+
:::
119+
120+
:::{dropdown} SystemD service file launched as a Podman service
121+
122+
The following is an example of an actual SystemD service file for an EPR, launched as a Podman service.
123+
124+
```ini
125+
# container-elastic-epr.service
126+
# autogenerated by Podman 4.1.1
127+
# Wed Oct 19 13:12:33 UTC 2022
128+
129+
[Unit]
130+
Description=Podman container-elastic-epr.service
131+
Documentation=man:podman-generate-systemd(1)
132+
Wants=network-online.target
133+
After=network-online.target
134+
RequiresMountsFor=%t/containers
135+
136+
[Service]
137+
Environment=PODMAN_SYSTEMD_UNIT=%n
138+
Restart=always
139+
TimeoutStopSec=70
140+
ExecStartPre=/bin/rm -f %t/%n.ctr-id
141+
ExecStart=/usr/bin/podman run \
142+
--cidfile=%t/%n.ctr-id \
143+
--cgroups=no-conmon \
144+
--rm \
145+
--sdnotify=conmon \
146+
-d \
147+
--replace \
148+
--name elastic-epr \
149+
-p 0.0.0.0:8443:8443 \
150+
-v /etc/elastic/epr/epr.pem:/etc/ssl/epr.crt:ro \
151+
-v /etc/elastic/epr/epr-key.pem:/etc/ssl/epr.key:ro \
152+
-e EPR_ADDRESS=0.0.0.0:8443 \
153+
-e EPR_TLS_CERT=/etc/ssl/epr.crt \
154+
-e EPR_TLS_KEY=/etc/ssl/epr.key docker.elastic.co/package-registry/distribution:{{stack-version}}
155+
ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
156+
ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=%t/%n.ctr-id
157+
Type=notify
158+
NotifyAccess=all
159+
160+
[Install]
161+
WantedBy=default.target
162+
```
163+
:::
164+
165+
## {{artifact-registry}} [air-gapped-elastic-artifact-registry]
166+
167+
Air-gapped install of the {{artifact-registry}} is necessary in order to enable {{agent}} deployments to perform self-upgrades and install certain components which are needed for some of the data integrations (that is, in addition to what is also retrieved from the EPR). To learn more, refer to [Host your own artifact registry for binary downloads](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/air-gapped.md#host-artifact-registry) in the {{fleet}} and {{elastic-agent}} Guide.
168+
169+
::::{note}
170+
When setting up own web server, such as NGINX, to function as the {{artifact-registry}}, it is recommended not to use TLS as there are, currently, no direct ways to establish certificate trust between {{agents}} and this service.
171+
::::
172+
173+
### Additional {{artifact-registry}} examples
174+
175+
:::{dropdown} Artifact download script
176+
177+
The following example script downloads artifacts from the internet to be later served as a private Elastic Package Registry.
178+
179+
```sh
180+
#!/usr/bin/env bash
181+
set -o nounset -o errexit -o pipefail
182+
183+
STACK_VERSION={{stack-version}}
184+
ARTIFACT_DOWNLOADS_BASE_URL=https://artifacts.elastic.co/downloads
185+
186+
DOWNLOAD_BASE_DIR=${DOWNLOAD_BASE_DIR:?"Make sure to set DOWNLOAD_BASE_DIR when running this script"}
187+
188+
COMMON_PACKAGE_PREFIXES="apm-server/apm-server beats/auditbeat/auditbeat beats/elastic-agent/elastic-agent beats/filebeat/filebeat beats/heartbeat/heartbeat beats/metricbeat/metricbeat beats/osquerybeat/osquerybeat beats/packetbeat/packetbeat cloudbeat/cloudbeat endpoint-dev/endpoint-security fleet-server/fleet-server"
189+
190+
WIN_ONLY_PACKAGE_PREFIXES="beats/winlogbeat/winlogbeat"
191+
192+
RPM_PACKAGES="beats/elastic-agent/elastic-agent"
193+
DEB_PACKAGES="beats/elastic-agent/elastic-agent"
194+
195+
function download_packages() {
196+
local url_suffix="$1"
197+
local package_prefixes="$2"
198+
199+
local _url_suffixes="$url_suffix ${url_suffix}.sha512 ${url_suffix}.asc"
200+
local _pkg_dir=""
201+
local _dl_url=""
202+
203+
for _download_prefix in $package_prefixes; do
204+
for _pkg_url_suffix in $_url_suffixes; do
205+
_pkg_dir=$(dirname ${DOWNLOAD_BASE_DIR}/${_download_prefix})
206+
_dl_url="${ARTIFACT_DOWNLOADS_BASE_URL}/${_download_prefix}-${_pkg_url_suffix}"
207+
(mkdir -p $_pkg_dir && cd $_pkg_dir && curl -O "$_dl_url")
208+
done
209+
done
210+
}
211+
212+
# and we download
213+
for _os in linux windows; do
214+
case "$_os" in
215+
linux)
216+
PKG_URL_SUFFIX="${STACK_VERSION}-${_os}-x86_64.tar.gz"
217+
;;
218+
windows)
219+
PKG_URL_SUFFIX="${STACK_VERSION}-${_os}-x86_64.zip"
220+
;;
221+
*)
222+
echo "[ERROR] Something happened"
223+
exit 1
224+
;;
225+
esac
226+
227+
download_packages "$PKG_URL_SUFFIX" "$COMMON_PACKAGE_PREFIXES"
228+
229+
if [[ "$_os" = "windows" ]]; then
230+
download_packages "$PKG_URL_SUFFIX" "$WIN_ONLY_PACKAGE_PREFIXES"
231+
fi
232+
233+
if [[ "$_os" = "linux" ]]; then
234+
download_packages "${STACK_VERSION}-x86_64.rpm" "$RPM_PACKAGES"
235+
download_packages "${STACK_VERSION}-amd64.deb" "$DEB_PACKAGES"
236+
fi
237+
done
238+
239+
240+
## selinux tweaks
241+
# semanage fcontext -a -t "httpd_sys_content_t" '/opt/elastic-packages(/.*)?'
242+
# restorecon -Rv /opt/elastic-packages
243+
```
244+
:::
245+
246+
:::{dropdown} NGINX config for private {{artifact-registry}} web server
247+
The following is an example NGINX configuration for running a web server for the {{artifact-registry}}.
248+
249+
```sh
250+
user nginx;
251+
worker_processes 2;
252+
253+
error_log /var/log/nginx/error.log notice;
254+
pid /var/run/nginx.pid;
255+
256+
events {
257+
worker_connections 1024;
258+
}
259+
260+
http {
261+
include /etc/nginx/mime.types;
262+
default_type application/octet-stream;
263+
264+
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
265+
'$status $body_bytes_sent "$http_referer" '
266+
'"$http_user_agent" "$http_x_forwarded_for"';
267+
268+
access_log /var/log/nginx/access.log main;
269+
sendfile on;
270+
keepalive_timeout 65;
271+
272+
server {
273+
listen 9080 default_server;
274+
server_name _;
275+
root /opt/elastic-packages;
276+
277+
location / {
278+
279+
}
280+
}
281+
282+
}
283+
```
284+
:::
285+
286+
287+
## Elastic Endpoint Artifact Repository [air-gapped-elastic-endpoint-artifact-repository]
288+
289+
Air-gapped setup of this component is, essentially, identical to the setup of the [{{artifact-registry}}](/deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry) except that different artifacts are served. To learn more, refer to [Configure offline endpoints and air-gapped environments](../../../solutions/security/configure-elastic-defend/configure-offline-endpoints-air-gapped-environments.md) in the Elastic Security guide.
290+
291+
292+
## {{ml-cap}} [air-gapped-machine-learning]
293+
294+
Some {{ml}} features, like natural language processing (NLP), require you to deploy trained models. To learn about deploying {{ml}} models in an air-gapped environment, refer to:
295+
296+
* [Deploy ELSER in an air-gapped environment](../../../explore-analyze/machine-learning/nlp/ml-nlp-elser.md#air-gapped-install).
297+
* [Install trained models in an air-gapped environment with Eland](asciidocalypse://docs/eland/docs/reference/machine-learning.md#ml-nlp-pytorch-air-gapped).
298+
299+
300+
## {{kib}} Product documentation for AI Assistants [air-gapped-kibana-product-documentation]
301+
302+
Detailed install and configuration instructions are available in the [{{kib}} AI Assistants settings documentation](asciidocalypse://docs/kibana/docs/reference/configuration-reference/ai-assistant-settings.md).

0 commit comments

Comments
 (0)