Skip to content

Commit 58b404d

Browse files
zvonandzvonand
authored
Merge pull request #1078 from Altinity/oauth-antalya-25.8
25.8 Antalya: Token Authentication and Authorization
2 parents c816023 + f909c33 commit 58b404d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+3165
-44
lines changed

‎ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,6 +291,8 @@ DoubleDelta
291291
Doxygen
292292
Dresseler
293293
Durre
294+
ECDSA
295+
EdDSA
294296
ECMA
295297
ETag
296298
EachRow
@@ -539,6 +541,8 @@ JoinAlgorithm
539541
JoinStrictness
540542
JumpConsistentHash
541543
Jupyter
544+
jwks
545+
JWKS
542546
KDevelop
543547
KafkaAssignedPartitions
544548
KafkaBackgroundReads
@@ -3379,6 +3383,7 @@ uuid
33793383
uuids
33803384
uuidv
33813385
vCPU
3386+
validators
33823387
varPop
33833388
varPopStable
33843389
varSamp
@@ -3396,6 +3401,8 @@ vectorscan
33963401
vendoring
33973402
verificationDepth
33983403
verificationMode
3404+
verifier
3405+
verifiers
33993406
versionedcollapsingmergetree
34003407
vhost
34013408
virtualized

‎contrib/jwt-cpp-cmake/CMakeLists.txt‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
1-
set(ENABLE_JWT_CPP_DEFAULT OFF)
2-
if(ENABLE_LIBRARIES AND CLICKHOUSE_CLOUD)
3-
set(ENABLE_JWT_CPP_DEFAULT ON)
4-
endif()
1+
set(ENABLE_JWT_CPP_DEFAULT ON)
52

63
option(ENABLE_JWT_CPP "Enable jwt-cpp library" ${ENABLE_JWT_CPP_DEFAULT})
74

@@ -20,4 +17,4 @@ set (JWT_CPP_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/jwt-cpp/include")
2017

2118
add_library (_jwt-cpp INTERFACE)
2219
target_include_directories(_jwt-cpp SYSTEM BEFORE INTERFACE ${JWT_CPP_INCLUDE_DIR})
23-
add_library(ch_contrib::jwt-cpp ALIAS _jwt-cpp)
20+
add_library(ch_contrib::jwt-cpp ALIAS _jwt-cpp)

‎docs/en/operations/external-authenticators/index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,4 +18,5 @@ The following external authenticators and directories are supported:
1818
- [LDAP](/operations/external-authenticators/ldap#ldap-external-authenticator) [Authenticator](./ldap.md#ldap-external-authenticator) and [Directory](./ldap.md#ldap-external-user-directory)
1919
- Kerberos [Authenticator](/operations/external-authenticators/kerberos#kerberos-as-an-external-authenticator-for-existing-users)
2020
- [SSL X.509 authentication](/operations/external-authenticators/ssl-x509)
21-
- HTTP [Authenticator](./http.md)
21+
- HTTP [Authenticator](./http.md)
22+
- Token-based [Authenticator](./tokens.md)

‎docs/en/operations/external-authenticators/tokens.md‎

Lines changed: 278 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
---
2+
slug: /en/operations/external-authenticators/oauth
3+
title: "Token-based authentication"
4+
---
5+
import SelfManaged from '@site/docs/en/_snippets/_self_managed_only_no_roadmap.md';
6+
7+
<SelfManaged />
8+
9+
ClickHouse users can be authenticated using tokens. This works in two ways:
10+
11+
- An existing user (defined in `users.xml` or in local access control paths) can be authenticated with a token if this user can be `IDENTIFIED WITH jwt`.
12+
- Use the information from the token or from an external Identity Provider (IdP) as a source of user definitions and allow locally undefined users to be authenticated with a valid token.
13+
14+
Although not all tokens are JWTs, under the hood both ways are treated as the same authentication method to maintain better compatibility.
15+
16+
# Token Processors
17+
18+
## Configuration
19+
20+
Token-based authentication is enabled by default. To disable it, set `enable_token_auth` to `0` in `config.xml`:
21+
22+
```xml
23+
<enable_token_auth>0</enable_token_auth>
24+
```
25+
26+
When disabled, token processors are not parsed, TokenAccessStorage is not available, and authentication via tokens (`--jwt` option or `Authorization: Bearer` header) is rejected.
27+
28+
To use token-based authentication, add `token_processors` section to `config.xml` and define at least one token processor in it.
29+
Its contents are different for different token processor types.
30+
31+
**Common parameters**
32+
- `type` -- type of token processor. Supported values: "jwt_static_key", "jwt_static_jwks", "jwt_dynamic_jwks", "azure", "openid". Mandatory. Case-insensitive.
33+
- `token_cache_lifetime` -- maximum lifetime of cached token (in seconds). Optional, default: 3600.
34+
- `username_claim` -- name of claim (field) that will be treated as ClickHouse username. Optional, default: "sub".
35+
- `groups_claim` -- name of claim (field) that contains list of groups user belongs to. This claim will be looked up in the token itself (in case token is a valid JWT, e.g. in Keycloak) or in response from `/userinfo`. Optional, default: "groups".
36+
37+
For each type, there are additional specific parameters (some of them are mandatory).
38+
If some parameters that are not required for current processor type are specified, they are ignored.
39+
40+
## JWT (JSON Web Token)
41+
42+
JWT itself is a source of information about user.
43+
It is decoded locally and its integrity is verified using either a local static key or JWKS (JSON Web Key Set), local or remote.
44+
45+
### JWT with static key:
46+
```xml
47+
<clickhouse>
48+
<token_processors>
49+
<my_static_key_validator>
50+
<type>jwt_static_key</type>
51+
<algo>HS256</algo>
52+
<static_key>my_static_secret</static_key>
53+
</my_static_key_validator>
54+
</token_processors>
55+
</clickhouse>
56+
```
57+
**Parameters:**
58+
- `algo` - Algorithm for signature validation. Mandatory. Supported values:
59+
60+
| HMAC | RSA | ECDSA | PSS | EdDSA |
61+
|-------| ----- | ------ | ----- | ------- |
62+
| HS256 | RS256 | ES256 | PS256 | Ed25519 |
63+
| HS384 | RS384 | ES384 | PS384 | Ed448 |
64+
| HS512 | RS512 | ES512 | PS512 | |
65+
| | | ES256K | | |
66+
Also supports None (not recommended and must *NEVER* be used in production).
67+
- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional.
68+
- `static_key` - key for symmetric algorithms. Mandatory for `HS*` family algorithms.
69+
- `static_key_in_base64` - indicates if the `static_key` key is base64-encoded. Optional, default: `False`.
70+
- `public_key` - public key for asymmetric algorithms. Mandatory except for `HS*` family algorithms and `None`.
71+
- `private_key` - private key for asymmetric algorithms. Optional.
72+
- `public_key_password` - public key password. Optional.
73+
- `private_key_password` - private key password. Optional.
74+
- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional.
75+
- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional.
76+
- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`.
77+
78+
### JWT with static JWKS
79+
```xml
80+
<clickhouse>
81+
<token_processors>
82+
<my_static_jwks_validator>
83+
<type>jwt_static_jwks</type>
84+
<static_jwks>{"keys": [{"kty": "RSA", "alg": "RS256", "kid": "mykid", "n": "_public_key_mod_", "e": "AQAB"}]}</static_jwks>
85+
</my_static_jwks_validator>
86+
</token_processors>
87+
</clickhouse>
88+
```
89+
90+
**Parameters:**
91+
92+
- `static_jwks` - content of JWKS in JSON
93+
- `static_jwks_file` - path to a file with JWKS
94+
- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional.
95+
- `verifier_leeway` - Clock skew tolerance (seconds). Useful for handling small differences in system clocks between ClickHouse and the token issuer. Optional.
96+
- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional.
97+
- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional.
98+
- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`.
99+
100+
:::note
101+
Only one of `static_jwks` or `static_jwks_file` keys must be present in one verifier
102+
:::
103+
104+
:::note
105+
Only RS* family algorithms are supported!
106+
:::
107+
108+
### JWT with remote JWKS
109+
```xml
110+
<clickhouse>
111+
<token_processors>
112+
<basic_auth_server>
113+
<type>jwt_dynamic_jwks</type>
114+
<jwks_uri>http://localhost:8000/.well-known/jwks.json</jwks_uri>
115+
<jwks_cache_lifetime>3600</jwks_cache_lifetime>
116+
</basic_auth_server>
117+
</token_processors>
118+
</clickhouse>
119+
```
120+
121+
**Parameters:**
122+
123+
- `uri` - JWKS endpoint. Mandatory.
124+
- `jwks_cache_lifetime` - Period for resend request for refreshing JWKS. Optional, default: 3600.
125+
- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional.
126+
- `verifier_leeway` - Clock skew tolerance (seconds). Useful for handling small differences in system clocks between ClickHouse and the token issuer. Optional.
127+
- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional.
128+
- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional.
129+
- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`.
130+
131+
132+
## Processors with external providers
133+
134+
Some tokens cannot be decoded and validated locally. External service is needed in this case. "Azure" and "OpenID" (a generic type) are supported now.
135+
136+
### Azure
137+
```xml
138+
<clickhouse>
139+
<token_processors>
140+
<azure_processor>
141+
<type>azure</type>
142+
</azure_processor>
143+
</token_processors>
144+
</clickhouse>
145+
```
146+
147+
No additional parameters are required.
148+
149+
### OpenID
150+
```xml
151+
<clickhouse>
152+
<token_processors>
153+
<oid_processor_1>
154+
<type>openid</type>
155+
<configuration_endpoint>url/.well-known/openid-configuration</configuration_endpoint>
156+
<verifier_leeway>60</verifier_leeway>
157+
<jwks_cache_lifetime>3600</jwks_cache_lifetime>
158+
</oid_processor_1>
159+
<oid_processor_2>
160+
<type>openid</type>
161+
<userinfo_endpoint>url/userinfo</userinfo_endpoint>
162+
<token_introspection_endpoint>url/tokeninfo</token_introspection_endpoint>
163+
<jwks_uri>url/.well-known/jwks.json</jwks_uri>
164+
<verifier_leeway>60</verifier_leeway>
165+
<jwks_cache_lifetime>3600</jwks_cache_lifetime>
166+
</oid_processor_2>
167+
</token_processors>
168+
</clickhouse>
169+
```
170+
171+
:::note
172+
Either `configuration_endpoint` or both `userinfo_endpoint` and `token_introspection_endpoint` (and, optionally, `jwks_uri`) shall be set. If none of them are set or all three are set, this is an invalid configuration that will not be parsed.
173+
:::
174+
175+
**Parameters:**
176+
177+
- `configuration_endpoint` - URI of OpenID configuration (often ends with `.well-known/openid-configuration`);
178+
- `userinfo_endpoint` - URI of endpoint that returns user information in exchange for a valid token;
179+
- `token_introspection_endpoint` - URI of token introspection endpoint (returns information about a valid token);
180+
- `jwks_uri` - URI of OpenID configuration (often ends with `.well-known/jwks.json`)
181+
- `jwks_cache_lifetime` - Period for resend request for refreshing JWKS. Optional, default: 3600.
182+
- `verifier_leeway` - Clock skew tolerance (seconds). Useful for handling small differences in system clocks between ClickHouse and the token issuer. Optional, default: 60
183+
- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional.
184+
- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional.
185+
- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`.
186+
187+
Sometimes a token is a valid JWT. In that case token will be decoded and validated locally if configuration endpoint returns JWKS URI (or `jwks_uri` is specified alongside `userinfo_endpoint` and `token_introspection_endpoint`).
188+
189+
### Tokens cache
190+
To reduce number of requests to IdP, tokens are cached internally for a maximum period of `token_cache_lifetime` seconds.
191+
If token expires sooner than `token_cache_lifetime`, then cache entry for this token will only be valid while token is valid.
192+
If token lifetime is longer than `token_cache_lifetime`, cache entry for this token will be valid for `token_cache_lifetime`.
193+
194+
## Enabling token authentication for a user in `users.xml` {#enabling-jwt-auth-in-users-xml}
195+
196+
In order to enable token-based authentication for the user, specify `jwt` section instead of `password` or other similar sections in the user definition.
197+
198+
Parameters:
199+
- `claims` - An optional string containing a json object that should be contained in the token payload.
200+
201+
Example (goes into `users.xml`):
202+
```xml
203+
<clickhouse>
204+
<my_user>
205+
<jwt>
206+
<claims>{"resource_access":{"account": {"roles": ["view-profile"]}}}</claims>
207+
</jwt>
208+
</my_user>
209+
</clickhouse>
210+
```
211+
212+
Here, the JWT payload must contain `["view-profile"]` on path `resource_access.account.roles`, otherwise authentication will not succeed even with a valid JWT.
213+
214+
:::note
215+
If `claims` is defined, this user will not be able to authenticate using opaque tokens, so, only JWT-based authentication will be available.
216+
:::
217+
218+
```
219+
{
220+
...
221+
"resource_access": {
222+
"account": {
223+
"roles": ["view-profile"]
224+
}
225+
},
226+
...
227+
}
228+
```
229+
230+
:::note
231+
A user cannot have JWT authentication together with any other authentication method. The presence of any other sections like `password` alongside `jwt` will force ClickHouse to shut down.
232+
:::
233+
234+
## Enabling token authentication using SQL {#enabling-jwt-auth-using-sql}
235+
236+
Users with "JWT" authentication type cannot be created using SQL now.
237+
238+
## Identity Provider as an External User Directory {#idp-external-user-directory}
239+
240+
If there is no suitable user pre-defined in ClickHouse, authentication is still possible: Identity Provider can be used as source of user information.
241+
To allow this, add `token` section to the `users_directories` section of the `config.xml` file.
242+
243+
At each login attempt, ClickHouse tries to find the user definition locally and authenticate it as usual.
244+
If a token is provided but the user is not defined, ClickHouse will treat the user as externally defined and will try to validate the token and get user information from the specified processor.
245+
If validated successfully, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section.
246+
All this implies that the SQL-driven [Access Control and Account Management](/docs/en/guides/sre/user-management/index.md#access-control) is enabled and roles are created using the [CREATE ROLE](/docs/en/sql-reference/statements/create/role.md#create-role-statement) statement.
247+
248+
**Example**
249+
250+
```xml
251+
<clickhouse>
252+
<user_directories>
253+
<token>
254+
<processor>token_processor_name</processor>
255+
<common_roles>
256+
<token_test_role_1 />
257+
</common_roles>
258+
<default_profile>my_profile</default_profile>
259+
<roles_filter>
260+
\bclickhouse-[a-zA-Z0-9]+\b
261+
</roles_filter>
262+
<roles_transform>s/-/_/g</roles_transform>
263+
</token>
264+
</user_directories>
265+
</clickhouse>
266+
```
267+
268+
:::note
269+
For now, no more than one `token` section can be defined inside `user_directories`. This _may_ change in future.
270+
:::
271+
272+
**Parameters**
273+
274+
- `processor` — Name of one of processors defined in `token_processors` config section described above. This parameter is mandatory and cannot be empty.
275+
- `common_roles` — Section with a list of locally defined roles that will be assigned to each user retrieved from the IdP. Optional.
276+
- `default_profile` — Name of a locally defined settings profile that will be assigned to each user retrieved from the IdP. If the profile does not exist, a warning will be logged and the user will be created without a profile. Optional.
277+
- `roles_filter` — Regex string for groups filtering. Only groups matching this regex will be mapped to roles. Optional.
278+
- `roles_transform` — Sed-style transform pattern to apply to group names before mapping to roles. Format: `s/pattern/replacement/flags`. The `g` flag applies the replacement globally (all occurrences). Example: `s/-/_/g` converts `clickhouse-grp-dba` to `clickhouse_grp_dba`. Optional.

0 commit comments

Comments
 (0)