Skip to content

Commit a4ca3a9

Browse files
vertex451Artem Shcherbatiukfourthisle
authored
feat(gql-gateway): Initial updated implementation of gql-gatway (#13)
* example * add schema conversion poc * grapqhl is compiling, resolver remaining * runtime client with empty result * apiVersion is not null * schema * implement using controller-runtime client with types generated from K8s cluster Open API schema JSON * add explanatory comments * two resources * fix name duplication issue * several resources, spec is nested, labels and namespace selection works * removed redundant code * improved research.go * improved gateway.go * improved resolver.go * fixed logger, improved start time * getItem query * create item works * made labels as map, not a string * made update the patch way * it compiles * refactored code * removed ws * refactor * refactor * get rid of spec expanding * better dir struct * workspace manager * tests * integration tests * added readme * cleaned Taskfile * final cleanup * taken care of comments * fixed subscriptions * added how to generate spec file * improved a readme * added support of CRDs * fixed tests, deprecated old crd service * more tests * added subscription to a specific field * added support of kcp workspaces * fix linter * improved subscription, added --use-current-context flag * fetch workspace from url, not from token * cleanup * cleanup * changed pipeline to golang-app * increased lint deadline * moved tresholds to Taskfile * moved tresholds to Taskfile * fix test coverage command * increase timeout * decreased a timeout * returned rename test * returned rename test * fix linter, again * added config * took care of comments * excluded config.go from coverage * added subscription tests * added threshold to pipeline * addressed comments, part 1 * fix tests * increased a timeout * added USER 1001:1001 to dockerfile * get rid of restMapper * returned original gateway * cleanup * changed dockerfile base to suppot kubectl cp --------- Co-authored-by: Artem Shcherbatiuk <[email protected]> Co-authored-by: Ali Khlifi <[email protected]>
1 parent 39685c3 commit a4ca3a9

29 files changed

+3455
-77
lines changed

‎.github/workflows/pipeline.yml

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,13 @@ jobs:
66
concurrency:
77
group: ${{ github.ref }}
88
cancel-in-progress: true
9-
uses: openmfp/gha/.github/workflows/pipeline-golang-module.yml@main
9+
uses: openmfp/gha/.github/workflows/pipeline-golang-app.yml@main
1010
secrets: inherit
1111
with:
12+
useTask: true
1213
useLocalCoverageConfig: true
13-
coverageThreasholdFile: 0
14-
coverageThreasholdPackage: 0
15-
coverageThreasholdTotal: 0
14+
imageTagName: ghcr.io/openmfp/gql-gateway
15+
coverageThreasholdFile: 50
16+
coverageThreasholdPackage: 51
17+
coverageThreasholdTotal: 56
18+

‎.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,16 @@ bin
1818

1919
# Output of the go coverage tool, specifically when used with LiteIDE
2020
*.out
21+
coverage.html
2122

2223
# Dependency directories (remove the comment below to include it)
2324
# vendor/
2425

2526
# Go workspace file
2627
go.work
28+
29+
# binary files
30+
main
31+
32+
# kcp
33+
.kcp

‎.testcoverage.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
11
exclude:
22
paths:
3-
- ^cmd/*
3+
- main.go
4+
- cmd
5+
- gateway
6+
- tests
7+
- internal/config/config.go

‎Dockerfile

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
FROM golang:1.23-alpine AS builder
2+
WORKDIR /app
3+
COPY . .
4+
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags '-w -s' main.go
5+
6+
FROM alpine
7+
8+
WORKDIR /app
9+
10+
11+
COPY --from=builder /app/main .
12+
13+
USER 1001:1001
14+
15+
ENTRYPOINT ["./main"]
16+
CMD ["gql-gateway"]

‎README.md

Lines changed: 187 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,83 +1,218 @@
11
> [!WARNING]
22
> This repository is under construction and not yet ready for public consumption. Please check back later for updates.
33
4+
# GQL Gateway
45

5-
# crd-gql-gateway
6+
The goal of this library is to provide a reusable and generic way of exposing k8s resources from within a cluster using GraphQL.
7+
This enables UIs that need to consume these objects to do so in a developer-friendly way, leveraging a rich ecosystem.
68

7-
The goal of this library is to provide a reusable and generic way of exposing Custom Resource Definitions from within a cluster using GraphQL. This enables UIs that need to consume these objects to do so in a developer-friendly way, leveraging a rich ecosystem.
9+
## Overview
10+
GQL Gateway expects a directory as input to watch for files containing OpenAPI specifications with resources.
811

9-
For each registered CRD, the gateway provides the following:
12+
Each file in that directory will correspond to a KCP workspace (or API server).
1013

11-
- A list query that allows the client to request a list of specific CRDs based on label selectors and/or namespace.
12-
- A query for an individual resource.
13-
- Create/Update/Delete mutations.
14-
- A list subscription type that opens a watch and serves the client live updates from CRDs within the cluster.
14+
For each file it will create a separate URL like `/<workspace-name>/graphql` which will be used to query the resources of that workspace.
1515

16-
Additionally, the gateway ensures that client requests are authorized to perform the desired actions using `SubjectAccessReview`, which ensures proper authorization.
16+
It will be watching for changes in the directory and update the schema accordingly.
1717

1818
## Usage
1919

20-
The goal is to provide a reusable library that can serve Custom Resources from any cluster without being specifically tied to a cluster/setup. The library is also able to dynamically infer which custom resource to expose based on the registered types in the [`runtime.Scheme`](https://pkg.go.dev/k8s.io/apimachinery/pkg/runtime#Scheme), which need to be registered anyway in order to get a functioning `controller-runtime` client.
20+
### OpenAPI Spec
2121

22-
To get started, you can consume the library in the following way:
22+
You can run the gateway using the existing generic OpenAPI spec file which is located in the `./definitions` directory.
2323

24-
#### 1. Create a `controller-runtime.Client` however you like
25-
26-
Please make sure to also include the `k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1` and the `k8s.io/api/authorization/v1` types, so the library can create `SubjectAccessReviews` and load `CustomResourceDefinitions`.
27-
28-
```go
29-
schema := runtime.NewScheme()
30-
apiextensionsv1.AddToScheme(schema)
31-
authzv1.AddToScheme(schema)
24+
(Optional) Or you can generate a new one from your own cluster by running the following command:
25+
```shell
26+
kubectl get --raw /openapi/v2 > filename
27+
```
28+
### Start the Service
29+
```shell
30+
task start
31+
```
32+
OR
33+
```shell
34+
go run main.go start --watched-dir=./definitions
35+
# where ./definitions is the directory containing the OpenAPI spec files
3236
```
3337

34-
After you set up the generally needed schema, feel free to add the types of any CRD that is available in your target cluster to the scheme. For every type that you register, there will be a set of queries, mutations, and subscriptions generated to expose your type via the gateway.
38+
After service start you can access the GraphQL playground.
39+
All addresses correspond the content of the watched directory and can be found in the terminal output.
3540

36-
```go
37-
package main
41+
For example, we have two KCP workspaces: `root` and `root:alpha`, for each of them we have a separate spec file in the `./definitions` directory.
3842

39-
import (
40-
// ...
41-
accountv1alpha1 "github.com/openmfp/account-operator/api/v1alpha1"
42-
// ...
43-
)
43+
Then we will have two URLs:
44+
- `http://localhost:3000/root/graphql`
45+
- `http://localhost:3000/root:alpha/graphql`
4446

45-
func main() {
46-
// ...
47-
accountv1alpha1.AddToScheme(schema)
47+
Open the URL in the browser and you will see the GraphQL playground.
4848

49-
cfg := controllerruntime.GetConfigOrDie()
49+
### Authorization
5050

51-
cl, err := client.NewWithWatch(cfg, client.Options{
52-
Scheme: schema,
53-
})
54-
if err != nil {
55-
panic(err)
51+
To send the request, you can attach the `Authorization` header with the token from kubeconfig `users.user.token`:
52+
```shell
53+
{
54+
"Authorization": "5f89bc76-c5b8-4d6f-b575-9ca7a6240bca"
55+
}
56+
```
57+
58+
**If you skip that header, service will try to use a runtime client with current context.(`kubectl config current-context`)**
59+
60+
P.S. Skipping the header works with both API server and KCP workspace.
61+
62+
#### Sending queries
63+
64+
##### Create a Pod:
65+
66+
```shell
67+
mutation {
68+
core {
69+
createPod(
70+
namespace: "default",
71+
object: {
72+
metadata: {
73+
name: "my-new-pod",
74+
labels: {
75+
app: "my-app"
76+
}
77+
}
78+
spec: {
79+
containers: [
80+
{
81+
name: "nginx-container"
82+
image: "nginx:latest"
83+
ports: [
84+
{
85+
containerPort: 80
86+
}
87+
]
88+
}
89+
]
90+
restartPolicy: "Always"
91+
}
92+
}
93+
) {
94+
metadata {
95+
name
96+
namespace
97+
labels
98+
}
99+
spec {
100+
containers {
101+
name
102+
image
103+
ports {
104+
containerPort
105+
}
106+
}
107+
restartPolicy
108+
}
109+
status {
110+
phase
111+
}
56112
}
113+
}
57114
}
58115
```
59116

60-
#### 2. Pass the client to the gateway library and see your resource being exposed :rocket:
117+
##### Get the created Pod:
118+
```shell
119+
query {
120+
core {
121+
Pod(name:"my-new-pod", namespace:"default") {
122+
metadata {
123+
name
124+
}
125+
spec{
126+
containers {
127+
image
128+
ports {
129+
containerPort
130+
}
131+
}
132+
}
133+
}
134+
}
135+
}
136+
```
61137

62-
```go
63-
gqlSchema, err := gateway.New(cmd.Context(), gateway.Config{
64-
Client: cl,
65-
})
66-
if err != nil {
67-
return err
138+
##### Delete the created Pod:
139+
```shell
140+
mutation {
141+
core {
142+
deletePod(
143+
namespace: "default",
144+
name: "my-new-pod"
145+
)
146+
}
68147
}
148+
```
149+
### Components Overview
150+
151+
#### Workspace manager
152+
153+
Holds the logic for watching a directory, triggering schema generation, and binding it to an HTTP handler.
154+
155+
*P.S. We are going to have an Event Listener that will watch the KCP workspace and write the OpenAPI spec into that directory.*
156+
157+
#### Gateway
158+
159+
Is responsible for the conversion from OpenAPI spec into the GraphQL schema.
160+
161+
#### Resolver
162+
163+
Holds the logic of interaction with the cluster.
69164

70-
http.Handle("/graphql", gateway.Handler(gateway.HandlerConfig{
71-
Config: &handler.Config{
72-
Schema: &gqlSchema,
73-
Pretty: true,
74-
Playground: true,
75-
},
76-
UserClaim: "mail",
77-
}))
165+
### Testing
166+
167+
```shell
168+
task test
169+
```
170+
171+
If you want to run single test, you need to export a KUBEBUILDER_ASSETS environment variable:
172+
```shell
173+
KUBEBUILDER_ASSETS=$(pwd)/bin/k8s/$DIR_WITH_ASSETS
174+
# where $DIR_WITH_ASSETS is the directory that contains binaries for your OS.
175+
```
176+
P.S. You can also integrate it within your IDE run configuration.
177+
178+
Then you can run the test:
179+
```
180+
181+
182+
You can also check the coverage:
183+
```shell
184+
task coverage
185+
```
186+
P.S. If you want to exclude some files from the coverage report, you can add them to the `.testcoverage.yml` file.
187+
188+
189+
190+
### Linting
191+
192+
```shell
193+
task lint
78194
```
79195

80-
You can expose the `gateway.Handler()` via the normal `net/http` package.
196+
### Subscriptions
197+
198+
To subscribe to events, you should use the SSE (Server-Sent Events) protocol.
81199

82-
It takes care of serving the right protocol based on the `Content-Type` header, as it exposes the `subscriptions` via the [`SSE`](https://html.spec.whatwg.org/multipage/server-sent-events.html) standard.
200+
Since GraphQL playground doesn't support it, you should use curl.
83201

202+
For instance, to subscribe to a change of a specific fields of the deployments, you can run the following command:
203+
```shell
204+
curl -H "Accept: text/event-stream" -H "Content-Type: application/json" http://localhost:3000/fullSchema/subscriptions \
205+
-d '{"query": "subscription { apps_deployments(namespace: \"default\") { metadata { name } spec { replicas } } }"}'
206+
```
207+
Fields that will be listened are defined in the graphql query within the `{}` brackets.
208+
209+
If you want to listen to all fields, you can set `subscribeToAll` to `true`:
210+
```shell
211+
curl -H "Accept: text/event-stream" -H "Content-Type: application/json" http://localhost:3000/fullSchema/subscriptions \
212+
-d '{"query": "subscription { apps_deployments(namespace: \"default\", subscribeToAll: true) { metadata { name } spec { replicas } } }"}'
213+
```
214+
If you want to listen to a specific deployment:
215+
```shell
216+
curl -H "Accept: text/event-stream" -H "Content-Type: application/json" http://localhost:3000/fullSchema/subscriptions \
217+
-d '{"query": "subscription { apps_deployment(namespace: \"default\", name: \"my-new-deployment\") { metadata { name } spec { replicas } } }"}'
218+
```

0 commit comments

Comments
 (0)