Skip to content

Commit 2c47b40

Browse files
zvonandzvonand
committed
Squashed all JWT auth
Add aspell Enable jwt-cpp in fasttest Add test + some minor improvements reduce unneeded possible clash points fix parsing create user identified with jwt refactor + fix not lowercase update test fix typo in docs fix logical_error some refactor fix alg in jwks fix jwks fix user auth method not being checked update docs better exception on no sub claim throw exception if algo not specified in jwk Support access token authorization of existing users Also possible to filter users by e-mail using regex fix token accessstorage Add Azure token processor, move JWKS logic to separate file remove docs that will be obsolete in future remove redundant resolve tokenCredentials on creation add basic docs for oauth add caching, cleanup bs fix credentials cast + some better code fix include fix invalid token handling add basic openid auth
1 parent 1baa927 commit 2c47b40

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+2746
-21
lines changed

‎ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -286,6 +286,8 @@ DoubleDelta
286286
Doxygen
287287
Dresseler
288288
Durre
289+
ECDSA
290+
EdDSA
289291
ECMA
290292
ETag
291293
EachRow
@@ -519,6 +521,8 @@ JoinAlgorithm
519521
JoinStrictness
520522
JumpConsistentHash
521523
Jupyter
524+
jwks
525+
JWKS
522526
KDevelop
523527
KafkaAssignedPartitions
524528
KafkaBackgroundReads
@@ -3223,6 +3227,7 @@ uuid
32233227
uuids
32243228
uuidv
32253229
vCPU
3230+
validators
32263231
varPop
32273232
varPopStable
32283233
varSamp
@@ -3240,6 +3245,8 @@ vectorscan
32403245
vendoring
32413246
verificationDepth
32423247
verificationMode
3248+
verifier
3249+
verifiers
32433250
versionedcollapsingmergetree
32443251
vhost
32453252
virtualized

‎contrib/jwt-cpp-cmake/CMakeLists.txt‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
1-
set(ENABLE_JWT_CPP_DEFAULT OFF)
2-
if(ENABLE_LIBRARIES AND CLICKHOUSE_CLOUD)
3-
set(ENABLE_JWT_CPP_DEFAULT ON)
4-
endif()
1+
set(ENABLE_JWT_CPP_DEFAULT ON)
52

63
option(ENABLE_JWT_CPP "Enable jwt-cpp library" ${ENABLE_JWT_CPP_DEFAULT})
74

@@ -20,4 +17,4 @@ set (JWT_CPP_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/jwt-cpp/include")
2017

2118
add_library (_jwt-cpp INTERFACE)
2219
target_include_directories(_jwt-cpp SYSTEM BEFORE INTERFACE ${JWT_CPP_INCLUDE_DIR})
23-
add_library(ch_contrib::jwt-cpp ALIAS _jwt-cpp)
20+
add_library(ch_contrib::jwt-cpp ALIAS _jwt-cpp)

‎docs/en/operations/external-authenticators/index.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,4 +18,6 @@ The following external authenticators and directories are supported:
1818
- [LDAP](/operations/external-authenticators/ldap#ldap-external-authenticator) [Authenticator](./ldap.md#ldap-external-authenticator) and [Directory](./ldap.md#ldap-external-user-directory)
1919
- Kerberos [Authenticator](/operations/external-authenticators/kerberos#kerberos-as-an-external-authenticator-for-existing-users)
2020
- [SSL X.509 authentication](/operations/external-authenticators/ssl-x509)
21-
- HTTP [Authenticator](./http.md)
21+
- HTTP [Authenticator](./http.md)
22+
- JWT [Authenticator](./jwt.md)
23+
- Access Token [Authenticator](./tokens.md)

‎docs/en/operations/external-authenticators/jwt.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
---
2+
slug: /en/operations/external-authenticators/jwt
3+
---
4+
# JWT
5+
import SelfManaged from '@site/docs/en/_snippets/_self_managed_only_no_roadmap.md';
6+
7+
<SelfManaged />
8+
9+
Existing and properly configured ClickHouse users can be authenticated via JWT.
10+
11+
Currently, JWT can only be used as an external authenticator for existing users, which are defined in `users.xml` or in local access control paths.
12+
The username will be extracted from the JWT after validating the token expiration and against the signature. Signature can be validated by:
13+
- static public key
14+
- static JWKS
15+
- received from the JWKS servers
16+
17+
It is mandatory for a JWT tot indicate the name of the ClickHouse user under `"sub"` claim, otherwise it will not be accepted.
18+
19+
A JWT may additionally be verified by checking the JWT payload.
20+
In this case, the occurrence of specified claims from the user settings in the JWT payload is checked.
21+
See [Enabling JWT authentication in `users.xml`](#enabling-jwt-auth-in-users-xml)
22+
23+
To use JWT authentication, JWT validators must be configured in ClickHouse config.
24+
25+
26+
## Enabling JWT validators in ClickHouse {#enabling-jwt-validators-in-clickhouse}
27+
28+
To enable JWT validators, add `token_validators` section in `config.xml`. This section may contain several JWT verifiers, minimum is 1.
29+
30+
### Verifying JWT signature using static key {$verifying-jwt-signature-using-static-key}
31+
32+
**Example**
33+
```xml
34+
<clickhouse>
35+
<!- ... -->
36+
<jwt_validators>
37+
<my_static_key_validator>
38+
<algo>HS256</algo>
39+
<static_key>my_static_secret</static_key>
40+
</my_static_key_validator>
41+
</jwt_validators>
42+
</clickhouse>
43+
```
44+
45+
#### Parameters:
46+
47+
- `algo` - Algorithm for validate signature. Supported:
48+
49+
| HMAC | RSA | ECDSA | PSS | EdDSA |
50+
|-------| ----- | ------ | ----- | ------- |
51+
| HS256 | RS256 | ES256 | PS256 | Ed25519 |
52+
| HS384 | RS384 | ES384 | PS384 | Ed448 |
53+
| HS512 | RS512 | ES512 | PS512 | |
54+
| | | ES256K | | |
55+
Also support None.
56+
- `static_key` - key for symmetric algorithms. Mandatory for `HS*` family algorithms.
57+
- `static_key_in_base64` - indicates if the `static_key` key is base64-encoded. Optional, default: `False`.
58+
- `public_key` - public key for asymmetric algorithms. Mandatory except for `HS*` family algorithms and `None`.
59+
- `private_key` - private key for asymmetric algorithms. Optional.
60+
- `public_key_password` - public key password. Optional.
61+
- `private_key_password` - private key password. Optional.
62+
63+
### Verifying JWT signature using static JWKS {$verifying-jwt-signature-using-static-jwks}
64+
65+
:::note
66+
Only RS* family algorithms are supported!
67+
:::
68+
69+
**Example**
70+
```xml
71+
<clickhouse>
72+
<!- ... -->
73+
<jwt_validators>
74+
<my_static_jwks_validator>
75+
<static_jwks>{"keys": [{"kty": "RSA", "alg": "RS256", "kid": "mykid", "n": "_public_key_mod_", "e": "AQAB"}]}</static_jwks>
76+
</my_static_jwks_validator>
77+
</jwt_validators>
78+
</clickhouse>
79+
```
80+
81+
#### Parameters:
82+
- `static_jwks` - content of JWKS in json
83+
- `static_jwks_file` - path to file with JWKS
84+
85+
:::note
86+
Only one of `static_jwks` or `static_jwks_file` keys must be present in one verifier
87+
:::
88+
89+
### Verifying JWT signature using JWKS servers {$verifying-jwt-signature-using-static-jwks}
90+
91+
**Example**
92+
```xml
93+
<clickhouse>
94+
<!- ... -->
95+
<jwt_validators>
96+
<basic_auth_server>
97+
<uri>http://localhost:8000/.well-known/jwks.json</uri>
98+
<connection_timeout_ms>1000</connection_timeout_ms>
99+
<receive_timeout_ms>1000</receive_timeout_ms>
100+
<send_timeout_ms>1000</send_timeout_ms>
101+
<max_tries>3</max_tries>
102+
<retry_initial_backoff_ms>50</retry_initial_backoff_ms>
103+
<retry_max_backoff_ms>1000</retry_max_backoff_ms>
104+
<refresh_ms>300000</refresh_ms>
105+
</basic_auth_server>
106+
</jwt_validators>
107+
</clickhouse>
108+
```
109+
110+
#### Parameters:
111+
112+
- `uri` - JWKS endpoint. Mandatory.
113+
- `refresh_ms` - Period for resend request for refreshing JWKS. Optional, default: 300000.
114+
115+
Timeouts in milliseconds on the socket used for communicating with the server (optional):
116+
- `connection_timeout_ms` - Default: 1000.
117+
- `receive_timeout_ms` - Default: 1000.
118+
- `send_timeout_ms` - Default: 1000.
119+
120+
Retry parameters (optional):
121+
- `max_tries` - The maximum number of attempts to make an authentication request. Default: 3.
122+
- `retry_initial_backoff_ms` - The backoff initial interval on retry. Default: 50.
123+
- `retry_max_backoff_ms` - The maximum backoff interval. Default: 1000.
124+
125+
### Verifying access tokens {$verifying-access-tokens}
126+
127+
Access tokens that are not JWT (and thus no data can be extracted from the token directly) need to be resolved by external providers.
128+
129+
**Example**
130+
```xml
131+
<clickhouse>
132+
<!- ... -->
133+
<access_token_processors>
134+
<my_access_token_processor>
135+
<provider>google</provider>
136+
</my_access_token_processor>
137+
</access_token_processors>
138+
</clickhouse>
139+
```
140+
141+
#### Parameters:
142+
143+
- `provider` - name of provider that will be used for token processing. Mandatory parameter. Possible options: `google`.
144+
145+
146+
### Enabling JWT authentication in `users.xml` {#enabling-jwt-auth-in-users-xml}
147+
148+
In order to enable JWT authentication for the user, specify `jwt` section instead of `password` or other similar sections in the user definition.
149+
150+
Parameters:
151+
- `claims` - An optional string containing a json object that should be contained in the token payload.
152+
153+
Example (goes into `users.xml`):
154+
```xml
155+
<clickhouse>
156+
<!- ... -->
157+
<my_user>
158+
<!- ... -->
159+
<jwt>
160+
<claims>{"resource_access":{"account": {"roles": ["view-profile"]}}}</claims>
161+
</jwt>
162+
</my_user>
163+
</clickhouse>
164+
```
165+
166+
Here, the JWT payload must contain `["view-profile"]` on path `resource_access.account.roles`, otherwise authentication will not succeed even with a valid JWT.
167+
168+
```
169+
{
170+
...
171+
"resource_access": {
172+
"account": {
173+
"roles": ["view-profile"]
174+
}
175+
},
176+
...
177+
}
178+
```
179+
180+
:::note
181+
JWT authentication cannot be used together with any other authentication method. The presence of any other sections like `password` alongside `jwt` will force ClickHouse to shut down.
182+
:::
183+
184+
### Enabling JWT authentication using SQL {#enabling-jwt-auth-using-sql}
185+
186+
When [SQL-driven Access Control and Account Management](/docs/en/guides/sre/user-management/index.md#access-control) is enabled in ClickHouse, users identified by JWT authentication can also be created using SQL statements.
187+
188+
```sql
189+
CREATE USER my_user IDENTIFIED WITH jwt CLAIMS '{"resource_access":{"account": {"roles": ["view-profile"]}}}'
190+
```
191+
192+
Or without additional JWT payload checks:
193+
194+
```sql
195+
CREATE USER my_user IDENTIFIED WITH jwt
196+
```
197+
198+
## JWT authentication examples {#jwt-authentication-examples}
199+
200+
#### Console client
201+
202+
```
203+
clickhouse-client -jwt <token>
204+
```
205+
206+
#### HTTP requests
207+
208+
```
209+
curl 'http://localhost:8080/?' \
210+
-H 'Authorization: Bearer <TOKEN>' \
211+
-H 'Content type: text/plain;charset=UTF-8' \
212+
--data-raw 'SELECT current_user()'
213+
```
214+
:::note
215+
ClickHouse will look for a JWT token in (by priority):
216+
1. `X-ClickHouse-JWT-Token` header.
217+
2. `Authorization` header.
218+
3. `token` request parameter. In this case, the "Bearer" prefix should not exist.
219+
:::

‎docs/en/operations/external-authenticators/tokens.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
---
2+
slug: /en/operations/external-authenticators/oauth
3+
title: "OAuth 2.0"
4+
---
5+
import SelfManaged from '@site/docs/en/_snippets/_self_managed_only_no_roadmap.md';
6+
7+
<SelfManaged />
8+
9+
OAuth 2.0 access tokens can be used to authenticate ClickHouse users. This works in two ways:
10+
11+
- Existing users (defined in `users.xml` or in local access control paths) can be authenticated with access token if this user can be `IDENTIFIED WITH jwt`.
12+
- Use Identity Provider (IdP) as an external user directory and allow locally undefined users to be authenticated with a token if it is valid and recognized by the provider.
13+
14+
Though this authentication method is different from JWT authentication, it works under the same authentication method to maintain better compatibility.
15+
16+
For both of these approaches a definition of `access_token_processors` is mandatory.
17+
18+
## Access Token Processors
19+
20+
To define an access token processor, add `access_token_processors` section to `config.xml`. Example:
21+
```xml
22+
<clickhouse>
23+
<access_token_processors>
24+
<gogoogle>
25+
<provider>Google</provider>
26+
<email_filter>^[A-Za-z0-9._%+-]+@example\.com$</email_filter>
27+
<cache_lifetime>600</cache_lifetime>
28+
</gogoogle>
29+
<azuure>
30+
<provider>azure</provider>
31+
<client_id>CLIENT_ID</client_id>
32+
<tenant_id>TENANT_ID</tenant_id>
33+
</azuure>
34+
</access_token_processors>
35+
</clickhouse>
36+
```
37+
38+
:::note
39+
Different providers have different sets of parameters.
40+
:::
41+
42+
**Parameters**
43+
44+
- `provider` -- name of identity provider. Mandatory, case-insensitive. Supported options: "Google", "Azure".
45+
- `cache_lifetime` -- maximum lifetime of cached token (in seconds). Optional, default: 3600.
46+
- `email_filter` -- Regex for validation of user emails. Optional parameter, only for Google IdP.
47+
- `client_id` -- Azure AD (Entra ID) client ID. Optional parameter, only for Azure IdP.
48+
- `tenant_id` -- Azure AD (Entra ID) tenant ID. Optional parameter, only for Azure IdP.
49+
50+
### Tokens cache
51+
To reduce number of requests to IdP, tokens are cached internally for no longer then `cache_lifetime` seconds.
52+
If token expires sooner than `cache_lifetime`, then cache entry for this token will only be valid while token is valid.
53+
If token lifetime is longer than `cache_lifetime`, cache entry for this token will be valid for `cache_lifetime`.
54+
55+
## IdP as External Authenticator {#idp-external-authenticator}
56+
57+
Locally defined users can be authenticated with an access token. To allow this, `jwt` must be specified as user's authentication method. Example:
58+
59+
```xml
60+
<clickhouse>
61+
<!- ... -->
62+
<users>
63+
<!- ... -->
64+
<my_user>
65+
<!- ... -->
66+
</jwt>
67+
</my_user>
68+
</users>
69+
</clickhouse>
70+
```
71+
72+
At each login attempt, ClickHouse will attempt to validate token and get user info against every defined access token provider.
73+
74+
When SQL-driven [Access Control and Account Management](/docs/en/guides/sre/user-management/index.md#access-control) is enabled, users that are authenticated with tokens can also be created using the [CREATE USER](/docs/en/sql-reference/statements/create/user.md#create-user-statement) statement.
75+
76+
Query:
77+
78+
```sql
79+
CREATE USER my_user IDENTIFIED WITH jwt;
80+
```
81+
82+
## Identity Provider as an External User Directory {#idp-external-user-directory}
83+
84+
If there is no suitable user pre-defined in ClickHouse, authentication is still possible: Identity Provider can be used as source of user information.
85+
To allow this, add `token` section to the `users_directories` section of the `config.xml` file.
86+
87+
At each login attempt, ClickHouse tries to find the user definition locally and authenticate it as usual.
88+
If the user is not defined, ClickHouse will treat user as externally defined, and will try to validate the token and get user information from the specified processor.
89+
If validated successfully, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section.
90+
All this implies that the SQL-driven [Access Control and Account Management](/docs/en/guides/sre/user-management/index.md#access-control) is enabled and roles are created using the [CREATE ROLE](/docs/en/sql-reference/statements/create/role.md#create-role-statement) statement.
91+
92+
**Example**
93+
94+
```xml
95+
<clickhouse>
96+
<token>
97+
<processor>gogoogle</processor>
98+
<roles>
99+
<token_test_role_1 />
100+
</roles>
101+
</token>
102+
</clickhouse>
103+
```
104+
105+
**Parameters**
106+
107+
- `server` — Name of one of processors defined in `access_token_processors` config section described above. This parameter is mandatory and cannot be empty.
108+
- `roles` — Section with a list of locally defined roles that will be assigned to each user retrieved from the IdP.

0 commit comments

Comments
 (0)