You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The LLM service will be available at [http://localhost:8000](http://localhost:8000).
248
255
249
256
## Development Workflow
@@ -274,16 +281,20 @@ The LLM service will be available at [http://localhost:8000](http://localhost:80
274
281
- Tests are in the `genai/tests` directory.
275
282
276
283
#### GenAI Usage
284
+
277
285
- The GenAI service is responsible for all interactions with the language model used to generate and modify recipes in RecipAI. It receives user inputs as free-text prompts and responds with structured outputs such as complete recipes, meal plans, or modified instructions. It is implemented using FastAPI, LangChain, and integrates with local and cloud large language models.
278
286
279
287
##### Retrieval-Augmented Generation
288
+
280
289
- The GenAI service uses Qdrant as a vector store to retrieve relevant documents before querying the LLM. It adds the retrieved context to the prompt to improve the relevance of answers.
281
290
282
291
##### Integration
292
+
283
293
- The client UI sends user requests to the server, which forwards them to the GenAI service along with the user’s query and chat history to support multi-turn conversations. GenAI service then makes a similarity search in the vector database with the given query, and generates a respective answer. GenAI service is able to provide a proper answer altough no similar context is found in the vector database. (Endpoint: POST - `genai/generate`)
284
294
- If the user wants to upload a recipe file, client UI sends the file content directly to the GenAI service, where the content of the file is chunked, embedded, and stored in the vector database. (Endpoint: POST - `genai/upload`)
285
295
286
296
##### Vector Database - Qdrant
297
+
287
298
We use Qdrant as the vector database to enable semantic search and retrieval-augmented generation (RAG) in RecipAI. Embeddings are generated using OpenAI’s small embedding model `text-embedding-3-small`.
@@ -397,34 +411,41 @@ The project includes Helm charts for Kubernetes deployment in the `recipai-chart
397
411
## CI/CD Pipeline
398
412
399
413
The project includes a GitHub Actions workflow `ci-cd.yml` for:
400
-
-**Testing Server**: For each push, server tests on server microservices are run.
414
+
415
+
-**Testing Server**: For each push, server tests on server microservices are run.
401
416
-**Building Docker Images**: Automatically builds and pushes Docker images to GitHub Container Registry.
402
417
-**Deploying Docker Images**: Automatically deploys the application to a production environment by using deployment manifests in helm for K8s cluster.
403
418
404
419
The project includes a GitHub Actions workflow `helm-manual.yml` for:
420
+
405
421
-**Deploying Docker Images**: Manually deploys the application to a production environment by using deployment manifests in helm for K8s cluster.
406
422
407
423
The project includes a GitHub Actions workflow `ansible-manual.yml` for:
424
+
408
425
-**Running Ansible Playbook**: Manually runs any Ansible playbook defined in the `ansible/playbooks` directory against an EC2 instance securely using SSH and Ansible Vault.
409
426
410
427
The project includes a GitHub Actions workflow `genai-tests.yml` for:
428
+
411
429
-**Running GenAI Tests**: Automatically runs the tests defined in the `genai/tests` directory on every code push in genai module.
412
430
413
431
## API Documentation
414
432
415
433
### User Service
434
+
416
435
API documentation is available in the [`server/user/openapi.yml`](server/user/openapi.yml) file.
417
436
418
437
### Chat Service
438
+
419
439
API documentation is available in the [`server/chat/openapi.yml`](server/chat/openapi.yml) file.
420
440
421
441
### GenAI Service
422
-
API documentation is available in the [`genai/openapi.yaml`](genai/openapi.yaml) file.
423
442
443
+
API documentation is available in the [`genai/openapi.yaml`](genai/openapi.yaml) file.
424
444
425
445
## Requirements
426
446
427
447
### Functional Requirements
448
+
428
449
- Recipe Generation: The system must allow users to input a description or ingredients and receive a complete recipe in response.
429
450
- Meal Planning: The system must allow users to request a meal plan for a specified number of days and meals per day.
430
451
- Chat-Based Interaction: The user must be able to interact with the application via a conversational interface and therefore the system must preserve context in multi-turn conversations.
@@ -433,6 +454,7 @@ API documentation is available in the [`genai/openapi.yaml`](genai/openapi.yaml)
433
454
- Chat History: The user must be able to view and revisit previous recipe requests and conversations.
434
455
435
456
### Nonfunctional Requirements
457
+
436
458
- Performance: The system must generate a recipe in response to a user query within 15 seconds in 95% of cases.
437
459
- Scalability: The architecture must allow horizontal scaling of microservices.
438
460
- Availability: The system must maintain 99.5% uptime during working hours.
@@ -446,58 +468,66 @@ API documentation is available in the [`genai/openapi.yaml`](genai/openapi.yaml)
446
468
- Usability: The chat interface must be responsive and intuitive.
447
469
- Observability & Monitoring: The system must expose Prometheus metrics for all critical services. Dashboards must be created in Grafana to visualize response latency, error rates, and user request volume. Besides that, at least one alert must be defined.
448
470
449
-
450
471
## Architecture Overview - TODO
451
472
452
-
453
-
454
473
## Monitoring and Observability
474
+
455
475
RecipAI is equipped with a monitoring stack powered by Prometheus and Grafana, deployed in a Kubernetes environment. This setup enables real-time observability across all microservices, including the GenAI service, user and chat services, and the API gateway.
456
476
457
477
### Prometheus Configuration
478
+
458
479
Prometheus is configured via a Kubernetes ConfigMap named prometheus-config at [`recipai-chart/templates/prometheus/prometheus-configmap.yml`](recipai-chart/templates/prometheus/prometheus-configmap.yml). The configuration defines scrape jobs for each service, enabling Prometheus to collect metrics every 15 seconds.
459
480
460
481
All services expose Prometheus-compatible metrics:
482
+
461
483
- GenAI service uses a standard `/metrics` endpoint.
462
484
- Server (Spring Boot) services (e.g., chat, user, api-gw) expose metrics via `/actuator/prometheus`.
463
485
464
486
For dockerized setup, respective prometheus config is also defined at [`monitoring/prometheus/prometheus.yml`](monitoring/prometheus/prometheus.yml)
465
487
466
488
### Grafana Dashboards
489
+
467
490
Grafana is used to visualize metrics collected by Prometheus. It is deployed via Helm and configured with:
491
+
468
492
- Dashboards for GenAI and chat service latency, error rates, and request counts.
469
493
- Dashboard for general system metrics of the cluster
- For dockerized setup: [`monitoring/grafana/dashboards/system-dashboard.json`](monitoring/grafana/dashboards/system-dashboard.json)
475
-
- For helm setup: [`recipai-chart/dashboards/system-dashboard.json`](recipai-chart/dashboards/system-dashboard.json)
499
+
- For dockerized setup: [`monitoring/grafana/dashboards/system-dashboard.json`](monitoring/grafana/dashboards/system-dashboard.json)
500
+
- For helm setup: [`recipai-chart/dashboards/system-dashboard.json`](recipai-chart/dashboards/system-dashboard.json)
476
501
- genai-dashboard.json: latency, error rates, and request counts
477
-
- For dockerized setup: [`monitoring/grafana/dashboards/genai-dashboard.json`](monitoring/grafana/dashboards/genai-dashboard.json)
478
-
- For helm setup: [`recipai-chart/dashboards/genai-dashboard.json`](recipai-chart/dashboards/genai-dashboard.json)
502
+
- For dockerized setup: [`monitoring/grafana/dashboards/genai-dashboard.json`](monitoring/grafana/dashboards/genai-dashboard.json)
503
+
- For helm setup: [`recipai-chart/dashboards/genai-dashboard.json`](recipai-chart/dashboards/genai-dashboard.json)
479
504
- chat-dashboard.json: latency, error rates, and request counts
480
-
- For dockerized setup: [`monitoring/grafana/dashboards/chat-dashboard.json`](monitoring/grafana/dashboards/chat-dashboard.json)
481
-
- For helm setup: [`recipai-chart/dashboards/chat-dashboard.json`](recipai-chart/dashboards/chat-dashboard.json)
505
+
- For dockerized setup: [`monitoring/grafana/dashboards/chat-dashboard.json`](monitoring/grafana/dashboards/chat-dashboard.json)
506
+
- For helm setup: [`recipai-chart/dashboards/chat-dashboard.json`](recipai-chart/dashboards/chat-dashboard.json)
482
507
483
508
#### Accessing Grafana
509
+
484
510
- To access Grafana locally from the cluster, we can do port-forwarding:
485
-
```bash
486
-
kubectl port-forward svc/grafana 3000:3000
487
-
```
488
-
Then, it should be available at [`http://localhost:3000`](http://localhost:3000)
511
+
512
+
```bash
513
+
kubectl port-forward svc/grafana 3000:3000
514
+
```
515
+
516
+
Then, it should be available at [`http://localhost:3000`](http://localhost:3000)
489
517
490
518
- Or you can access it directly via our ingress in the cluster: [`https://grafana-tcd.student.k8s.aet.cit.tum.de`](https://grafana-tcd.student.k8s.aet.cit.tum.de)
491
519
492
520
- Grafana Credentials:
493
-
- Username: `admin`
494
-
- Password: `admin`
521
+
- Username: `admin`
522
+
- Password: `admin`
495
523
496
524
### Alerting Setup
525
+
497
526
Alerts are configured using Grafana’s Unified Alerting system. It defines receivers (e.g., Discord webhook) for high response generation duration in the GenAI service (for >20 seconds).
527
+
498
528
- Contact points are defined:
499
-
- For dockerized setup: [`monitoring/grafana/provisioning/alerting/contact-points.yaml`](monitoring/grafana/provisioning/alerting/contact-points.yaml)
500
-
- For helm setup: [`recipai-chart/templates/grafana/grafana-alerting.yml`](recipai-chart/templates/grafana/grafana-alerting.yml)
529
+
- For dockerized setup: [`monitoring/grafana/provisioning/alerting/contact-points.yaml`](monitoring/grafana/provisioning/alerting/contact-points.yaml)
530
+
- For helm setup: [`recipai-chart/templates/grafana/grafana-alerting.yml`](recipai-chart/templates/grafana/grafana-alerting.yml)
0 commit comments