|
| 1 | +--- |
| 2 | +applies_to: |
| 3 | + deployment: |
| 4 | + eck: ga |
| 5 | + self: ga |
| 6 | +products: |
| 7 | + - id: elasticsearch |
| 8 | + - id: cloud-kubernetes |
| 9 | +--- |
| 10 | + |
| 11 | +# File-based access recovery |
| 12 | + |
| 13 | +The [built-in `file` realm](/deploy-manage/users-roles/cluster-or-deployment-auth/file-based.md) is commonly used as a fallback or recovery realm. |
| 14 | + |
| 15 | +The main {{stack}} {{security-features}} rely on the `security` [feature state](/deploy-manage/tools/snapshot-and-restore.md) which is mostly composed of the `.security*` [system indices](elasticsearch://reference/elasticsearch/rest-apis/api-conventions.md#system-indices). The `file` realm acts as a failsafe to expand this feature's functionality from the cluster level down to each individual node. |
| 16 | + |
| 17 | +The `file` realm is especially helpful for recovery scenarios where normal security mechanisms aren't accessible: |
| 18 | + |
| 19 | +* Node services are running but cluster is unresponsive |
| 20 | +* {{stack}} {{security-features}} are unavailable to the current node |
| 21 | +* {{stack}} {{security-features}} are [lost and need to be restored](/troubleshoot/elasticsearch/red-yellow-cluster-status.md#fix-cluster-status-restore) |
| 22 | +* Administrative users' passwords are lost and [need to be reset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-change-password) |
| 23 | + |
| 24 | +The {{stack}} {{security-features}} only apply the `file` realm to the modified local node and do not apply changes across all nodes within the cluster. Administrators of self-managed deployments are responsible for ensuring one of the following: |
| 25 | + |
| 26 | +* The same users and roles are defined across every node in the cluster. |
| 27 | + |
| 28 | + Frequently, administrators choose to apply the change on one {{es}} node and then distribute or copy the files to all other nodes in the cluster. Files can be distributed either manually or using a configuration management system such as Puppet or Chef. |
| 29 | +* The related API requests are directed to the local node rather than to any cluster-level load balancer or proxy URL. |
| 30 | + |
| 31 | +:::{tip} |
| 32 | +Refer to [enabling a file realm user for recovery](https://www.youtube.com/watch?v=sueO7sz1buw) for a video walkthrough of this process. |
| 33 | +::: |
| 34 | + |
| 35 | +## Step 1 (Optional): Configure the realm [file-realm-recovery-configuration] |
| 36 | + |
| 37 | +You don’t need to explicitly configure a `file` realm. The `file` and `native` realms are added to the realm chain by default. Unless configured otherwise, the `file` realm is added first, followed by the `native` realm. You can define only one `file` realm on each node. |
| 38 | + |
| 39 | +To learn how to override the default `file` realm configuration, refer to [Configure a file realm](https://www.elastic.co/docs/deploy-manage/users-roles/cluster-or-deployment-auth/file-based#file-realm-configuration). |
| 40 | + |
| 41 | +{{es}} reads security-related files upon the local node's initial startup and as periodically refreshed based on the `resource.reload.interval.high` setting. You do not need to restart nodes for changes to take effect. These files are located under the [`ES_PATH_CONF` directory](/deploy-manage/deploy/self-managed/configure-elasticsearch.md#config-files-location) and contain the following information: |
| 42 | + |
| 43 | +* `roles.yml` for [defining roles](/deploy-manage/users-roles/cluster-or-deployment-auth/defining-roles.md) |
| 44 | +* `role_mapping.yml` for [mapping external users and groups to roles](/deploy-manage/users-roles/cluster-or-deployment-auth/mapping-users-groups-to-roles.md) |
| 45 | +* `users` for [user password-based authentication](/deploy-manage/users-roles/cluster-or-deployment-auth/user-authentication.md) |
| 46 | +* `user_roles` for [user role-based authorization](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md) |
| 47 | + |
| 48 | +## Step 2: Choose roles for recovery [file-realm-recovery-roles] |
| 49 | + |
| 50 | +Before granting a `file` realm user any roles, you need to ensure that those desired roles exist. You can use the following types of roles: |
| 51 | + |
| 52 | +* [Built-in roles](elasticsearch://reference/elasticsearch/roles.md) |
| 53 | +* [Custom roles](/deploy-manage/users-roles/cluster-or-deployment-auth/defining-roles.md) defined under the {{stack}} {{security-features}} |
| 54 | +* Roles defined in [`roles.yml`](/deploy-manage/users-roles/cluster-or-deployment-auth/defining-roles.md#roles-management-file) |
| 55 | + |
| 56 | +{{es}} recommends following the industry's [principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) when granting user permissions. {{es}} follows this guidance itself by [restricting system indices](/deploy-manage/users-roles/cluster-or-deployment-auth/role-structure.md#roles-indices-priv) by default, even from [`superuser` role](elasticsearch://reference/elasticsearch/roles.md#available-roles) administrators including the [`elastic` built-in user](/deploy-manage/users-roles/cluster-or-deployment-auth/built-in-users.md). |
| 57 | + |
| 58 | +The main {{stack}} {{security-features}} rely on the `security` [feature state](/deploy-manage/tools/snapshot-and-restore.md) which is mostly composed of the `.security*` [system indices](elasticsearch://reference/elasticsearch/rest-apis/api-conventions.md#system-indices). When recovering {{stack}} {{security-features}}, you will likely need to temporarily define a custom role with the [`allow_restricted_indices` setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-put-role) enabled. |
| 59 | + |
| 60 | +For example, to grant the administrative privileges of `superuser` role alongside `allow_restricted_indices: true` you can create a new role called `superduperuser` with the following definition: |
| 61 | + |
| 62 | +```yaml |
| 63 | +superduperuser: |
| 64 | + cluster: [ 'all' ] |
| 65 | + indices: |
| 66 | + - names: [ '*' ] |
| 67 | + privileges: [ 'all' ] |
| 68 | + allow_restricted_indices: true |
| 69 | +``` |
| 70 | +
|
| 71 | +After you decide on your role definitions, you can add your users and any custom roles to your local node's configuration. |
| 72 | +
|
| 73 | +:::{{warning}} |
| 74 | +Restricted indices are a special category of indices that are used to store cluster configuration data and should not normally be directly accessed. **Toggling this flag is normally _strongly_ discouraged because it could effectively grant unrestricted operations on critical data, making the entire system unstable or leaking sensitive information.** |
| 75 | +
|
| 76 | +If `allow_restricted_indices` needs temporarily enabled for a user in order to recover the {{stack}} {{security-features}}, {{es}} recommends ensuring to remove this role with sensitive access from the user upon task completion. |
| 77 | +::: |
| 78 | + |
| 79 | +## Step 3: Add the recovery user and role to the node [file-realm-recovery-files] |
| 80 | + |
| 81 | +For most administrators, {{es}} recommends that you use the [`elasticsearch-users` tool](elasticsearch://reference/elasticsearch/command-line-tools/users-command.md), which compiles the `users` and `users_roles` files on your behalf. |
| 82 | + |
| 83 | +In this example, we create an advanced administrative user with the `superduperuser` role we designed [in the previous step](#file-realm-recovery-roles). |
| 84 | + |
| 85 | +::::{tab-set} |
| 86 | + |
| 87 | +:::{tab-item} Self-managed |
| 88 | + |
| 89 | +``` |
| 90 | +bin/elasticsearch-users useradd admin -p changeme -r superduperuser |
| 91 | +``` |
| 92 | +::: |
| 93 | +
|
| 94 | +:::{tab-item} ECK secrets |
| 95 | +The following is an example of invoking the `elasticsearch-users` tool in a Docker container: |
| 96 | +
|
| 97 | +```sh subs=true |
| 98 | +# create a folder with the 2 files |
| 99 | +mkdir filerealm |
| 100 | +touch filerealm/users filerealm/users_roles |
| 101 | +
|
| 102 | +# create user 'admin' with role 'superduperuser' |
| 103 | +docker run \ |
| 104 | + -v $(pwd)/filerealm:/usr/share/elasticsearch/config \ |
| 105 | + docker.elastic.co/elasticsearch/elasticsearch:{{version.stack}} \ |
| 106 | + bin/elasticsearch-users useradd admin -p changeme -r superduperuser |
| 107 | +
|
| 108 | +# create a Kubernetes secret with the file realm content |
| 109 | +kubectl create secret generic my-file-realm-secret --from-file filerealm |
| 110 | +``` |
| 111 | + |
| 112 | +You can pass `users` and `user_roles` files to {{eck}} using a file realm secret: |
| 113 | + |
| 114 | +```yaml subs=true |
| 115 | +apiVersion: elasticsearch.k8s.elastic.co/v1 |
| 116 | +kind: Elasticsearch |
| 117 | +metadata: |
| 118 | + name: elasticsearch-sample |
| 119 | +spec: |
| 120 | + version: {{version.stack}} |
| 121 | + auth: |
| 122 | + fileRealm: |
| 123 | + - secretName: my-filerealm-secret-1 |
| 124 | + - secretName: my-filerealm-secret-2 |
| 125 | + nodeSets: |
| 126 | + - name: default |
| 127 | + count: 1 |
| 128 | +``` |
| 129 | +
|
| 130 | +A file realm secret is composed of two entries: a `users` entry and a `users_roles` entry. You can provide either one entry or both entries in each secret. |
| 131 | + |
| 132 | +If you specify multiple users with the same name in more than one secret, the last one takes precedence. If you specify multiple roles with the same name in more than one secret, a single entry per role is derived from the concatenation of its corresponding users from all secrets. |
| 133 | + |
| 134 | +The following secret specifies three users and their respective built-in and custom roles: |
| 135 | + |
| 136 | +```yaml |
| 137 | +kind: Secret |
| 138 | +apiVersion: v1 |
| 139 | +metadata: |
| 140 | + name: my-filerealm-secret |
| 141 | +stringData: |
| 142 | + roles.yml: |- |
| 143 | + superduperuser: |
| 144 | + cluster: |
| 145 | + - all |
| 146 | + indices: |
| 147 | + - names: [ '*' ] |
| 148 | + privileges: [ 'all' ] |
| 149 | + allow_restricted_indices: true |
| 150 | + users: |- |
| 151 | + rdeniro:$2a$10$BBJ/ILiyJ1eBTYoRKxkqbuDEdYECplvxnqQ47uiowE7yGqvCEgj9W |
| 152 | + alpacino:$2a$10$cNwHnElYiMYZ/T3K4PvzGeJ1KbpXZp2PfoQD.gfaVdImnHOwIuBKS |
| 153 | + jacknich:{PBKDF2}50000$z1CLJt0MEFjkIK5iEfgvfnA6xq7lF25uasspsTKSo5Q=$XxCVLbaKDimOdyWgLCLJiyoiWpA/XDMe/xtVgn1r5Sg= |
| 154 | + users_roles: |- |
| 155 | + admin:rdeniro |
| 156 | + superduperuser:alpacino,jacknich |
| 157 | + user:jacknich |
| 158 | +``` |
| 159 | +::: |
| 160 | + |
| 161 | + |
| 162 | +:::{tab-item} ECK basic auth |
| 163 | + |
| 164 | +You can also add `file` realm users using [{{k8s}} basic authentication secrets](https://kubernetes.io/docs/concepts/configuration/secret/#basic-authentication-secret). For more information, refer to [](/deploy-manage/users-roles/cluster-or-deployment-auth/file-based.md#k8s-basic). |
| 165 | + |
| 166 | +1. Create a secret `my-roles-secret` that adds the `superduperuser` role definition: |
| 167 | + |
| 168 | + ```yaml |
| 169 | + apiVersion: v1 |
| 170 | + kind: Secret |
| 171 | + metadata: |
| 172 | + name: my-roles-secret |
| 173 | + stringData: |
| 174 | + roles.yml: |- |
| 175 | + superduperuser: |
| 176 | + cluster: |
| 177 | + - all |
| 178 | + indices: |
| 179 | + - names: [ '*' ] |
| 180 | + privileges: [ 'all' ] |
| 181 | + allow_restricted_indices: true |
| 182 | + ``` |
| 183 | + |
| 184 | +2. Set up a basic authentication secret `secret-basic-auth` which contains its `username`, `password`, and a comma-separated list of `roles`: |
| 185 | + |
| 186 | + ```yaml |
| 187 | + apiVersion: v1 |
| 188 | + kind: Secret |
| 189 | + metadata: |
| 190 | + name: secret-basic-auth |
| 191 | + type: kubernetes.io/basic-auth |
| 192 | + stringData: |
| 193 | + username: admin # required field for kubernetes.io/basic-auth |
| 194 | + password: changeme # required field for kubernetes.io/basic-auth |
| 195 | + roles: superduperuser,superuser # optional, not part of kubernetes.io/basic-auth |
| 196 | + ``` |
| 197 | + |
| 198 | +3. Make the secrets available to {{eck}} by adding them to your {{es}} manifest: |
| 199 | + |
| 200 | + ```yaml subs=true |
| 201 | + apiVersion: elasticsearch.k8s.elastic.co/v1 |
| 202 | + kind: Elasticsearch |
| 203 | + metadata: |
| 204 | + name: elasticsearch-sample |
| 205 | + spec: |
| 206 | + version: {{version.stack}} |
| 207 | + auth: |
| 208 | + fileRealm: |
| 209 | + - secretName: secret-basic-auth |
| 210 | + roles: |
| 211 | + - secretName: my-roles-secret |
| 212 | + nodeSets: |
| 213 | + - name: default |
| 214 | + count: 1 |
| 215 | + ``` |
| 216 | + |
| 217 | +::: |
| 218 | + |
| 219 | +:::: |
| 220 | + |
| 221 | +## Step 4: Recover {{security-features}} [file-realm-recovery-curl] |
| 222 | + |
| 223 | +At this point, the local {{es}} node will accept [Elasticsearch API requests](https://www.elastic.co/docs/reference/elasticsearch/rest-apis) with the created `file` based username and password. |
| 224 | + |
| 225 | +For example, if you created a user with the username `admin`, password `changeme`, and role `superduperuser`, then you can now send a cURL request to the [Get cluster info API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-info) from the node's local shell: |
| 226 | +```bash |
| 227 | +curl -X GET -sk -u "admin:changeme" "https://localhost:9200/" |
| 228 | +``` |
| 229 | + |
| 230 | +:::{{tip}} |
| 231 | +The related API requests need to be directed to the local nodes where `file` has been configured, rather than to any cluster-level load balancer or proxy URL. |
| 232 | +::: |
| 233 | + |
| 234 | +You can confirm that the desired `superduperuser` role is applied to your `admin` username using the [Authenticate a user API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-authenticate): |
| 235 | +```bash |
| 236 | +curl -X GET -sk -u "admin:changeme" "https://localhost:9200/_security/_authenticate?pretty=true" |
| 237 | +``` |
| 238 | + |
| 239 | +Now that you have regained recovery access to the cluster, you can investigate and recover the {{stack}} {{security-features}} as needed. |
0 commit comments