|
1 | | -# Otomi Api |
| 1 | +# Akamai App Platform API |
2 | 2 |
|
3 | | -This application provides a HTTP REST API (definition in [OpenApiV3](https://swagger.io/specification/) standard) to manipulate values for teams and their services. |
4 | | -Git is used as the persistent storage for the values that will be consumed by [otomi-stack](https://github.com/redkubes/otomi-stack) for reconciling the state of the cluster landscape. (For an example look at the [otomi-values](https://github.com/redkubes/otomi-values-demo) repo with demo values.) |
5 | | -Every api deployment will result in a commit to the values repo with the author's email in the title. |
| 3 | +**The brain of Akamai App Platform** - A REST API service that manages Kubernetes teams, workloads, and services using Git as persistent storage. |
6 | 4 |
|
7 | | -## Design documents |
| 5 | +### Prerequisites |
8 | 6 |
|
9 | | -- [REST API and GitOps](docs/gitops.md) |
| 7 | +- **Node.js**: v22.x (see `.nvmrc`) |
| 8 | +- **npm**: v10.x+ |
| 9 | +- **Git**: For version control and values repository |
| 10 | +- **Akamai App Platform Core** [Akamai App Platform Core](https://github.com/linode/apl-core) (see #Development Setup) |
10 | 11 |
|
11 | | -## 1. Development |
12 | | - |
13 | | -### 1.1 Prerequisites |
14 | | - |
15 | | -- npm@~10.0 installed |
16 | | -- a valid values repo: follow these [instructions in otomi-core](https://github.com/redkubes/otomi-core/blob/main/docs/setup.md#a-valid-values-repo) |
17 | | - |
18 | | -### 1.2 Setting up environment |
19 | | - |
20 | | -The following two steps only need to be performed once: |
21 | | - |
22 | | -1. Copy `.env.sample` to `.env` and edit accordingly. |
23 | | -2. Download `otomi-api/.secrets` file from [Google Drive secrets](https://drive.google.com/drive/folders/1N802vs0IplKehkZq8SxMi67RipyO1pHN) and put content in `.env`. |
24 | | - |
25 | | -The last step is running `npm install`. |
26 | | - |
27 | | -### 1.3 Running dependencies |
28 | | - |
29 | | -The api depends on a running `otomi-core` tools server. It can be started from the `otomi-core` repo with: |
| 12 | +### 1. Setup Environment |
30 | 13 |
|
31 | 14 | ```bash |
32 | | -export ENV_DIR={location of your values repo} |
33 | | -otomi server |
34 | | -``` |
35 | | - |
36 | | -Another way to start it in docker-compose (from within this repo): |
37 | | - |
38 | | -``` |
39 | | -bin/dc.sh up-deps & |
| 15 | +# Clone the repository |
| 16 | +git clone https://github.com/linode/apl-api.git |
| 17 | +cd apl-api |
| 18 | +# Install dependencies |
| 19 | +npm install |
| 20 | +# Setup environment variables |
| 21 | +cp .env.sample .env |
| 22 | +# Edit .env with your configuration |
| 23 | +npm run dev |
40 | 24 | ``` |
41 | 25 |
|
42 | | -(This setup and fragile and might be broken. If that is the case just clone `otomi-core` and follow the first suggestion.) |
| 26 | +The API will be available at `http://localhost:8080` |
43 | 27 |
|
44 | | -### 1.4 Run the dev server |
| 28 | +## 📋 Essential Commands |
45 | 29 |
|
46 | | -From the root of this project: |
| 30 | +### Development |
47 | 31 |
|
48 | 32 | ```bash |
49 | | -export ENV_DIR={location of your values repo} |
50 | | -export GIT_LOCAL_PATH=$ENV_DIR |
51 | | -npm run dev |
| 33 | +npm run dev # Start development server with hot reload |
| 34 | +npm run build # Full production build |
| 35 | +npm run start # Start production server |
| 36 | +npm test # Run all tests |
52 | 37 | ``` |
53 | 38 |
|
54 | | -### 1.5 Mock different users |
55 | | - |
56 | | -In order to test websocket communication between two browsers you can prime the api to register two different users. |
57 | | -To set the api to register user 0 or 1: |
| 39 | +### Code Quality |
58 | 40 |
|
| 41 | +```bash |
| 42 | +npm run lint # Check code style and types |
| 43 | +npm run lint:fix # Auto-fix linting issues |
| 44 | +npm run types # Type check only |
59 | 45 | ``` |
60 | | -http://localhost:3000/api/mock/0 |
61 | | -http://localhost:3000/api/mock/1 |
62 | | -``` |
63 | | - |
64 | | -See `src/mocks.ts` for details. |
65 | 46 |
|
66 | | -## 2. Api design |
| 47 | +### OpenAPI & Models |
67 | 48 |
|
68 | | -### 2.1 Specification |
69 | | - |
70 | | -The API is defined in [src/openapi/api.yaml](src/openapi/api.yaml) file in OpenApi v3 format. |
| 49 | +```bash |
| 50 | +npm run build:models # Generate TypeScript models from OpenAPI |
| 51 | +npm run build:spec # Build OpenAPI specification |
| 52 | +``` |
71 | 53 |
|
72 | | -This file is used to generate server API endpoints and client API library as well. The api endpoints are bound to a |
73 | | -given function that developer can implement. For example: |
| 54 | +## 🏗️ Architecture |
74 | 55 |
|
75 | | -``` |
76 | | -paths: |
77 | | - '/teams/{teamId}': |
78 | | - operationId: getTeam |
79 | | - get: |
80 | | - parameters: |
81 | | - - name: teamId |
82 | | - in: path |
83 | | - required: true |
84 | | - schema: |
85 | | - type: string |
86 | | -``` |
| 56 | +### Core Concept |
87 | 57 |
|
88 | | -For the api server it is expected that the `src/api/teams/{teamId}.ts` file exists and implements api endpoints for get |
89 | | -method. |
| 58 | +- **Git-as-Database**: All configuration stored as YAML in Git repository |
| 59 | +- **OpenAPI-First**: All endpoints auto-generated from `src/openapi/*.yaml` specs |
| 60 | +- **Multi-Tenant**: Team isolation with RBAC/ABAC authorization |
| 61 | +- **Real-time**: WebSocket updates for live status monitoring |
90 | 62 |
|
91 | | -For the api client there is an `operationId` property defined. It can be used to client with expected method names (see |
92 | | -relevant usage in otomi-web repo) |
| 63 | +### Key Components |
93 | 64 |
|
94 | | -### 2.2 Authentication |
| 65 | +| Component | Purpose | |
| 66 | +| -------------------- | ----------------------------------- | |
| 67 | +| `src/app.ts` | Express server setup and middleware | |
| 68 | +| `src/otomi-stack.ts` | Core business logic engine | |
| 69 | +| `src/authz.ts` | Authorization system (CASL-based) | |
| 70 | +| `src/api/` | Auto-generated route handlers | |
| 71 | +| `src/openapi/` | OpenAPI specifications | |
| 72 | +| `src/middleware/` | JWT, session, authz middleware | |
95 | 73 |
|
96 | | -The authentication ensures that a user is identified, so the request contains required headers. |
| 74 | +## 🔐 Authentication & Authorization |
97 | 75 |
|
98 | | -The authentication security schemas are defined under `components.securitySchemes` in `src/api.yaml` file. In the same |
99 | | -file a global authentication schema is defined under the `security` property and is applied to all API HTTP methods |
100 | | -unless it is explicitly defined at a given HTTP method. |
| 76 | +### Authentication |
101 | 77 |
|
102 | | -For example: |
| 78 | +- **JWT tokens** with user identity and teams |
| 79 | +- **Headers required**: `Authorization`, `Auth-Group` |
| 80 | +- **Mock users** available for testing |
103 | 81 |
|
104 | | -``` |
105 | | -paths: |
106 | | - /secrets: |
107 | | - get: |
108 | | - responses: |
109 | | - '200': |
110 | | -``` |
| 82 | +### Authorization (RBAC + ABAC) |
111 | 83 |
|
112 | | -From above: |
| 84 | +- **platformAdmin**: Full system access |
| 85 | +- **teamAdmin**: Manage own team resources |
| 86 | +- **teamMember**: Limited team resource access |
113 | 87 |
|
114 | | -- the GET /secrets request handler authenticate it by using security schema defined under global `security` property. |
| 88 | +### Testing Auth |
115 | 89 |
|
116 | | -How to set headers for OWASP: |
| 90 | +Mock different users: |
117 | 91 |
|
118 | 92 | ``` |
119 | | -X-Frame-Options: sameorigin |
120 | | -X-Content-Type-Options: nosniff |
121 | | -Referrer-Policy: no-referrer |
122 | | -Cross-Origin-Embedder-Policy: require-corp |
123 | | -Cross-Origin-Opener-Policy: same-origin |
124 | | -X-Content-Type-Options: same-origin |
| 93 | +GET http://localhost:8080/api/mock/0 # Mock user 0 |
| 94 | +GET http://localhost:8080/api/mock/1 # Mock user 1 |
125 | 95 | ``` |
126 | 96 |
|
127 | | -### 2.3 Authorization |
128 | | - |
129 | | -An authorization is defined in `src/api.yaml` file as an extension to OpenApiV3 spec. |
130 | | - |
131 | | -There are two keywords used to specify authorization: |
| 97 | +## 🛠️ Development Guide |
132 | 98 |
|
133 | | -- **x-aclSchema** - indicates a schema that contains `x-acl` properties applicable for a given API request, |
134 | | -- **x-acl** - defines roles and allowed CRUD operations for each role. |
| 99 | +### Adding New Endpoints |
135 | 100 |
|
136 | | -For example: |
| 101 | +1. **Define in OpenAPI** (`src/openapi/*.yaml`): |
137 | 102 |
|
138 | | -``` |
| 103 | +```yaml |
139 | 104 | paths: |
140 | | - /services: |
| 105 | + '/teams/{teamId}/services': |
141 | 106 | get: |
142 | | - x-aclSchema: Services |
143 | | -
|
144 | | -components: |
145 | | - schemas: |
146 | | - Services: |
147 | | - x-acl: |
148 | | - platformAdmin: [read] |
149 | | - teamAdmin: [read] |
150 | | - teamMember: [read] |
151 | | - type: array |
| 107 | + operationId: getTeamServices |
| 108 | + parameters: |
| 109 | + - name: teamId |
| 110 | + in: path |
| 111 | + required: true |
152 | 112 | ``` |
153 | 113 |
|
154 | | -From above: |
155 | | - |
156 | | -- on GET /services request the permissions from `Services` schema are applied |
| 114 | +2. **Implement Handler** (`src/api/v1/teams/{teamId}/services.ts`): |
157 | 115 |
|
158 | | -The authorization is only applied if authentication is enabled, so required header are available. |
159 | | - |
160 | | -#### 2.3.1 Resource Based Access Control (RBAC) |
161 | | - |
162 | | -The RBAC is used to define allowed CRUD operations on resource level. It also guards resource ownership by comparing |
163 | | -`teamId` from HTTP request parameter against content of `Auth-Group` HTTP header. |
| 116 | +```typescript |
| 117 | +export async function getTeamServices(req: OpenApiRequestExt): Promise<AplResponseObject> { |
| 118 | + // Implementation here |
| 119 | +} |
| 120 | +``` |
164 | 121 |
|
165 | | -The following example briefly introduce possible configurations: |
| 122 | +3. **Add Authorization** (in OpenAPI spec): |
166 | 123 |
|
167 | | -``` |
| 124 | +```yaml |
| 125 | +x-aclSchema: Service |
168 | 126 | components: |
169 | 127 | schemas: |
170 | 128 | Service: |
171 | 129 | x-acl: |
172 | | - platformAdmin: [delete-any, read-any, create-any, update-any] |
173 | | - teamAdmin: [delete, read, create, update] |
174 | | - teamMember: [delete, read, create, update] |
175 | | - type: object |
176 | | - properties: |
| 130 | + platformAdmin: [read-any, create-any, update-any, delete-any] |
| 131 | + teamAdmin: [read, create, update, delete] |
| 132 | + teamMember: [read] |
177 | 133 | ``` |
178 | 134 |
|
179 | | -From above: |
180 | | - |
181 | | -- a user with admin role can perform all CRUD operations regardless resource ownership (the `-any` postfix), |
182 | | -- a user with team role can perform all CRUD operations only on its own resource. |
183 | | - |
184 | | -**Note:** |
185 | | - |
186 | | -- use `-any` if a given role grands permission to perform operations regardless resource ownership |
187 | | -- the `-any` is supported only for RBAC permissions |
188 | | - |
189 | | -#### 2.3.2 Attribute Based Access Control (ABAC) |
190 | | - |
191 | | -By default all resource attributes can be modified by any user that is allowed to access the resource (RBAC) |
192 | | -ABAC aims to restrict control of changing specific attributes that belong to a given resource. |
193 | | - |
194 | | -All possible ABAC configurations are defined in the `TeamSelfService` schema. This schema can be used to define a team's `selfService` configuration. Only one with `admin` role can modify that property. |
195 | | - |
196 | | -The `TeamSelfService` schema is composed by: |
| 135 | +### Working with Git Storage |
197 | 136 |
|
198 | | -- property that corresponds to a schema name from `api.yaml` file. |
199 | | -- `enum` property that indicates JSON paths for attributes that shall be controlled. |
| 137 | +All data operations go through `OtomiStack` class: |
200 | 138 |
|
201 | | -**Note:** |
202 | | - |
203 | | -- `delete` permission cannot be set for ABAC |
204 | | - |
205 | | -For example: |
206 | | - |
207 | | -``` |
208 | | - Service: |
209 | | - x-acl: |
210 | | - platformAdmin: [delete-any, read-any, create-any, update-any] |
211 | | - teamAdmin: [delete, read, create, update] |
212 | | - teamMember: [delete, read, create, update] |
213 | | - type: object |
214 | | - properties: |
215 | | - name: |
216 | | - type: string |
217 | | - ingress: |
218 | | - type: object |
219 | | - x-acl: |
220 | | - platformAdmin: [read, create] |
221 | | - teamAdmin: [read] |
222 | | - teamMember: [read] |
| 139 | +```typescript |
| 140 | +const stack = new OtomiStack() |
| 141 | +await stack.getTeamConfigService('team-id') |
| 142 | +await stack.createService('team-id', serviceData) |
223 | 143 | ``` |
224 | 144 |
|
225 | | -From above: |
226 | | - |
227 | | -A user with admin role can: |
228 | | - |
229 | | -- perform all CRUD operations regardless resource ownership (RBAC) |
230 | | -- all attributes can be edited except ingress that can be only set on resource creation event (ABAC) |
231 | | - |
232 | | -A user with team role can: |
233 | | - |
234 | | -- perform all CRUD operations only withing its own team (RBAC) |
235 | | -- all attributes can be edited except ingress that isn be only read (ABAC) |
236 | | - |
237 | | -#### 2.3.3 Limitations |
| 145 | +### Testing |
238 | 146 |
|
239 | | -##### 2.3.3.1 OpenAPI-generator limitations |
240 | | - |
241 | | -<!-- Add more issues if you spot them and know the limitations/work-arounds --> |
242 | | - |
243 | | -Known issues: |
244 | | - |
245 | | -- https://github.com/redkubes/otomi-api/issues/155 |
246 | | - |
247 | | -###### Problem |
248 | | - |
249 | | -It doesn't matter if you've entered a valid OpenAPI specification, it isn't useful as long as it isn't generated as a client library. |
250 | | - |
251 | | -###### Cause |
252 | | - |
253 | | -There are too many variations of this problem to be listed here and still make sense, but they follow the following cycle in general: |
254 | | - |
255 | | -1. `src/openapi/\*.yaml` cannot be dereferenced/bundled by parsing JSON `$refs`. |
256 | | -2. Dereferenced/bundled OpenAPI spec cannot be generated |
257 | | -3. Client libary in `vendors/client/otomi-api/axios/...` cannot be compiled with `tsc` |
258 | | -4. Code cannot be committed in version control (Git) |
259 | | -5. Consume API methods and/or models |
260 | | - |
261 | | -###### Solutions |
262 | | - |
263 | | -In this paragraph the causes are addressed by the corresponding number under "Cause": |
264 | | - |
265 | | -1. In the `npm run ...` scripts, `vendors/openapi/otomi-api.json` may be deleted to see if the spec can be successfully dereferenced/bundled and used as input for `openapi-generator`. |
266 | | -2. The `openapi-generator` can throw useful/meaningful errors. But there are errors under known issues (see above) that need a work-around. |
267 | | -3. These errors happen the most arbitrarily. See if you can go back in your small increments in `src/openapi/...` until you can successfully build the client library again. |
268 | | -4. These errors are often due to our own code. E.g.: a generated model is used, and by changing the OpenAPI spec you change the schema. Models used to rely on the schema and now they are missing. |
269 | | -5. If you change the name of a schema, add a title, etc., the respective reference might change. Then the consumption in the API library might break. |
270 | | - |
271 | | -###### Note |
272 | | - |
273 | | -- Also check if you can successfully generate the client library again after committing, just as a pre-caution. |
274 | | -- To determine a successful generation of the client library, please check out the generated models in `vendors/client/otomi-api/axios/models` if they make sense or not. |
275 | | -- As general advice, make sure to increment the specification VERY slowly and always see if a spec can be generated or not. |
276 | | - |
277 | | -## 3. Viewing/consuming openapi spec |
278 | | - |
279 | | -In order to inspect the api file it is recommended to either: |
280 | | - |
281 | | -- install `swagger viewer` plugin in you vscode |
282 | | -- or copy file content and paste in <https://editor.swagger.io> |
283 | | - |
284 | | -Client code can get the API doc by querying the following endpoint: |
285 | | - |
286 | | -``` |
287 | | -GET http://127.0.0.1:8080/v1/apiDocs |
| 147 | +```bash |
| 148 | +npm test # All tests |
| 149 | +npm run test:pattern -- MyTest # Specific test |
288 | 150 | ``` |
289 | 151 |
|
290 | | -Moreover the `openapi.yaml` file can be used with `Postman` (File -> Import). |
291 | | - |
292 | | -## 4. Models generated from spec |
293 | | - |
294 | | -When any of the `src/openapi/*.yaml` files change, new models will be generated into `src/generated-schema.ts`. These models are exported as types in `src/otomi-models.ts` and used throughout the code. |
| 152 | +## Contributing |
295 | 153 |
|
296 | | -**IMPORTANT:** |
| 154 | +See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. |
297 | 155 |
|
298 | | -Because openapi bundler optimizes by re-using schema blocks but does not take into account depth of the schema, it creates a schema that is not usable by the `npm run build:client` task. This only happens when a subschema references root schemas, and only for certain props. We had to apply some yaml anchor hacking to make it work. |
| 156 | +## License |
299 | 157 |
|
300 | | -When you encounter errors during client generation, instead of referencing the faulty props by `$ref` use a yaml anchor. Example file: `src/openapi/team.yaml`. |
| 158 | +Licensed under Apache License, Version 2.0. See [LICENSE.md](LICENSE.md). |
0 commit comments