Merge pull request #76 from CS3219-AY2425S1/titus/update-readme

docs: update documentation

Author: tituschewxj

README.md

@@ -9,3 +9,38 @@
 - You can choose to develop individual microservices within separate folders within this repository **OR** use individual repositories (all public) for each microservice.
 - In the latter scenario, you should enable sub-modules on this GitHub classroom repository to manage the development/deployment **AND** add your mentor to the individual repositories as a collaborator.
 - The teaching team should be given access to the repositories as we may require viewing the history of the repository in case of any disputes or disagreements.
+
+---
+
+## Architecture Diagram
+
+![Overall Architecture Diagram](./docs/architecture_diagram.png)
+
+The overall architecture of PeerPrep follows a microservices architecture. The client acts as an orchestrator for the interaction between the different services.
+
+## Screenshots
+
+![Home Page](./docs/home_page.png)
+
+![Collaboration Page](./docs/collab_page_1.png)
+
+![Collaboration Page](./docs/collab_page_2.png)
+
+![Question Page](./docs/question_page.png)
+
+![Question Page](./docs/indiv_question_page.png)
+
+![History Page](./docs/submission_history_page.png)
+
+## More details
+
+- [Frontend](./apps/frontend/README.md)
+- [User Service](./apps/user-service/README.md)
+- [Question Service](./apps/question-service/README.md)
+- [Matching Service](./apps/matching-service/README.md)
+- [Signalling Service](./apps/signalling-service/README.md)
+- [History Service](./apps/history-service/README.md)
+- [Execution Service](./apps/execution-service/README.md)
+- [CI/CD Guide](./docs/cicid.md)
+- [Docker Compose Guide](./apps/README.md)
+- [Set Up Guide](./docs/setup.md)

apps/README.md

@@ -2,6 +2,8 @@
 
 This project uses Docker Compose to manage multiple services such as a frontend, backend, and a database. The configuration is defined in the `docker-compose.yml` file, and environment variables can be stored in environment files for different environments (e.g., development, production).
 
+More details on how to set up Docker Compose can be found [here](../docs/setup.md)
+
 ## Prerequisites
 
 Before you begin, ensure you have the following installed on your machine:
@@ -30,11 +32,25 @@ In the `./apps` directory:
 ├── user-service
 │   ├── Dockerfile           # Dockerfile for user-service
 │   └── ... (other user-service files)
-
+├── execution-service
+│   ├── Dockerfile           # Dockerfile for execution-service
+│   └── ... (other execution-service files)
+├── signalling-service
+│   ├── Dockerfile           # Dockerfile for signalling-service
+│   └── ... (other signalling-service files)
+├── history-service
+│   ├── Dockerfile           # Dockerfile for history-service
+│   └── ... (other history-service files)
 ```
 
 ## Docker Compose Setup
 
+Ensure that you are currently using **Docker Compose v2** in your local Docker Desktop.
+- Launch your local Docker Desktop application
+- Click on settings button at the top right hand corner (beside the name)
+- Under the General tab, scroll down until you see a checkbox that says Use Docker Compose V2, ensure that the box is checked then apply and restart (refer to the image below)
+![Docker Compose V2](https://github.com/user-attachments/assets/3b8d47c2-c488-4fc1-804d-418ffebbdd9c)
+
 By using multiple Dockerfiles in Docker Compose, we can manage complex multi-container applications where each service has its own environment and build process.
 
 1. Build and Start the Application
@@ -54,11 +70,15 @@ This will:
 
 Once running, you can access:
 
-- The **frontend** at http://localhost:3000
-- The **user service** at http://localhost:3001
-- The **question service** at http://localhost:8080 (REST) and http://localhost:50051 (gRPC)
-- The **matching service** at http://localhost:8081
-- The **redis service** at http://localhost:6379
+- The [**frontend**](./frontend/README.md) at http://localhost:3000
+- The [**user-service**](./user-service/README.md) at http://localhost:3001
+- The [**question-service**](./question-service/README.md) at http://localhost:8080 (REST) and http://localhost:50051 (gRPC)
+- The [**matching-service**](./matching-service/README.md) at http://localhost:8081
+- The [**history-service**](./history-service/README.md) at http://localhost:8082
+- The [**execution-service**](./execution-service/README.md) at http://localhost:8083
+- The [**signalling-service**](./signalling-service/README.md) at http://localhost:4444
+- The **redis** at http://localhost:6379
+- The **rabbitmq** at http://localhost:5672
 
 3. Stopping Services
 
@@ -76,6 +96,11 @@ This command will stop and remove the containers, networks, and volumes created
 
 - **Port Conflicts**: If you encounter port conflicts, ensure the host ports specified in docker-compose.yml (e.g., 3000:3000) are not in use by other applications.
 - **Environment Variables Not Loaded**: Ensure the `.env` files are in the correct directories as found in the `docker-compose.yml` file.
+- **Command execution failed**: When you try running test cases or submitting the code in the collaborative environment, if you encounter the following error message:
+    ```bash
+  Command execution failed: Unable to find image 'apps-python-sandbox:latest' locally docker: Error response from daemon: pull access denied for apps-python-sandbox, repository does not exist or may require 'docker login': denied: requested access to the resource is denied. See 'docker run --help'. : exit status 125
+    ```
+  Ensure that you have **Docker Compose V2** enabled for your Docker Desktop application. Please refer to the Docker Compose setup guide above to enable it locally.
 
 ### Known Issues
 

apps/execution-service/README.md

@@ -1,5 +1,59 @@
 # Execution Service
 
+The Execution Service provides backend functionality for running and validating code executions or submissions within a coding platform. It enables users to execute code against test cases and receive feedback on the correctness of their solutions.
+
+The Execution Service incorporates a code execution mechanism designed to run user-submitted solutions within an isolated, sandboxed environment. This approach enhances security by preventing arbitrary code from interacting with the host system directly and allows for performance monitoring
+
+### Technology Stack
+
+- Golang (Go): Statically typed, compiled language with low latency. Fast and efficient processing is ideal for high-read, high-write environments like in Execution Service, when many users run tests or submit tests.
+- Rest Server: chi router was utilized which supports CORS, logging and timeout via middlewares. It is stateless, which reduces coupling and enhances scalability and reliability, simplicity and flexibility. For example, clients may make requests to different server instances when scaled.
+- Firebase Firestore: NoSQL Document database that is designed for automatic horizontal scaling and schema-less design that allows for flexibility as number of tests increases or more users run tests.
+- Docker: used to containerize the Execution Service to simplify deployment. Additionally used to provide a sandboxed execution environment for user-submitted code, ensuring security by limiting code access to the host system and managing dependencies independently.
+
+### Execution Process
+
+For execution of user code (running of test cases without submission), only visible (public) and custom test cases are executed.
+
+![Diagram of code execution process](../../docs/exeuction_process.png)
+
+### Submission Process
+
+For submission of user code, both visible (public) and hidden testcases are executed, before calling the history-service API to submit the submission data, code and test results.
+
+![Diagram of code submission process](../../docs/submission_process.png)
+
+### Design Decisions
+
+1. **Docker Containerisation**
+   a. Upon receiving a code execution request, the service dynamically creates a Docker container with a controlled environment tailored to Python
+   b. The Docker container is set up with only the minimal permissions and resources needed to execute the code, restricting the execution environment to reduce risk
+   c. This containerized environment is automatically destroyed after execution, ensuring no residual data or state remains between executions
+
+2. **Security and Isolation**
+   a. Containers provide isolation from the host system, limiting any interaction between user code and the underlying infrastructure
+   b. Only essential files and libraries required for code execution are included, reducing potential attack surfaces within each container. The sandboxed, container-based execution system provides a secure and efficient way to run user code submissions.
+
+The sandboxed, container-based execution system provides a secure and efficient way to run user code submissions.
+
+### Communication between Execution and History Service
+
+The communication between the Execution service and the History service is implemented through a RabbitMQ Message Queue. RabbitMQ is ideal for message queues in microservices due to its reliability, flexible routing, and scalability. It ensures messages aren’t lost through durable queues and supports complex routing to handle diverse messaging needs.
+
+Asynchronous communication was chosen as a user’s submission history did not need to be updated immediately. Instead of waiting for a response, the Execution Service can put the message in a queue and continue processing other requests.
+
+![RabbitMQ Message Queue](./../../docs/rabbit_mq_queue.png)
+
+A message queue allows services to communicate without depending on each other's availability. The Execution Service can send a message to the queue, and the History Service can process it when it’s ready. This decoupling promotes loose coupling and reduces dependencies between services, which helps maintain a robust and adaptable system.
+
+---
+
+## Setup
+
+### Prerequisites
+
+Ensure you have Go installed on your machine.
+
 ### Installation
 
 1. Install dependencies:
@@ -61,10 +115,10 @@ The server will be available at http://localhost:8083.
 
 ## API Endpoints
 
-- `POST /tests/populate`
-- `GET /tests/{questionDocRefId}/`
-- `POST /tests/{questionDocRefId}/execute`
-- `POST /tests/{questionDocRefId}/submit`
+- `POST: /tests/populate`: Deletes and repopulates all tests in Firebase
+- `GET: /{questionDocRefId}`: Reads the public testcases for the question, identified by the question reference ID
+- `POST: /{questionDocRefId}/execute`: Executes the public testcases for the question, identified by the question reference ID
+- `POST: /{questionDocRefId}/submit`: Executes the public and hidden testcases for the question, identified by the question reference ID, and submits the code submission to History Service
 
 ## Managing Firebase
 

apps/frontend/README.md

@@ -1,11 +1,27 @@
-This is the frontend for the question service.
+# Frontend
 
-## Tech Stack
+![Home page](../../docs/home_page.png)
 
-- Next.js
-- TypeScript
-- Ant Design
-- SCSS
+### Tech Stack
+
+- React: React is one of the most popular UI libraries that allows the creation of reusable UI functional components. Its community ecosystem also offers React hooks that simplify the implementation of some of our frontend components, such as websockets.
+- Next.js: A React framework for building single-page applications. It comes with several useful features such as automatic page routing based on filesystem.
+- Ant Design: An enterprise-level design system that comes with several extensible UI components and solutions out-of-the-box, which allows us to quickly create nice-looking components that can be adjusted according to our requirements.
+- Typescript: A language extension of Javascript that allows us to perform static type-checking, to ensure that issues with incorrectly used types are caught and resolved as early as possible, improving code maintainability.
+
+### Authorization-based Route Protection with Next.js Middleware
+
+Middleware is a Next.js feature that allows the webpage server to intercept page requests and perform checks before serving the webpage. We used this feature to protect page access from unauthenticated users. This was done by checking the request’s JWT token (passed as a cookie) against the user service and redirecting users without authorized access to a public route (namely, the login page).
+
+### User Flow and Communication between Microservices
+
+Clients interact with the microservices through dedicated endpoints, with each microservice managing its own database for independent reading and writing.
+
+Having individual databases per microservice improves data security, scalability, fault isolation, flexibility in database choice, and development efficiency. This approach allows each microservice to operate independently, optimizing stability, performance, and adaptability in the system.
+
+![Diagram for user flow and communication between microservices](../../docs/userflow.png)
+
+---
 
 ## Getting Started
 

apps/history-service/README.md

@@ -1,4 +1,27 @@
-# Question Service
+# History Service
+
+The History Service is designed to store and retrieve a user’s code submission history. Users can view their past submission records of a collaboration session, with details such as the submitted date, the question attempted on, and the matched username. The information on the submission is stored within the history service’s database, and the data is accessed through querying the history-service from the frontend. It uses Google Firestore as a cloud-based NoSQL database for efficient and scalable data storage and retrieval. It is developed with a RESTful API structure, allowing flexibility for client applications to interact with the service.
+
+### Technology Stack
+
+- Golang (Go): Statically typed, compiled language with low latency. Fast and efficient processing is ideal for high-read, high-write environments like in history service.
+- Firebase Firestore: NoSQL Document database that is designed for automatic horizontal scaling and schema-less design that allows for flexibility as application grows and new features are added.
+- REST Server: chi router was utilized which supports CORS, logging and timeout via middlewares. It is stateless, which reduces coupling and enhances scalability and reliability, simplicity and flexibility. For example, clients may make requests to different server instances when scaled.
+- Docker: used to containerize the History Service to simplify deployment.
+
+### Design Decisions
+
+The submission history is organized with the most recent submission displayed first, making it easy for users to review their past submissions. Pagination is implemented to help users find specific records efficiently and reduce the amount of data transferred when loading the page.
+
+![Screenshot of the submission history page with pagination](../../docs/submission_history_page.png)
+
+On the question page, users can view their past submissions for that question, allowing them to see their submitted code alongside the question details for better context.
+
+![Screenshot of a question’s submission history](../../docs/indiv_question_page.png)
+
+Each submission record is created through the execution service via an asynchronous call, ensuring smooth and efficient processing (more details provided in the next section).
+
+---
 
 ## Overview
 
@@ -90,10 +113,10 @@ The server will be available at http://localhost:8082.
 
 ## API Endpoints
 
-- `POST /histories`
-- `GET /histories/{docRefId}`
-- `PUT /histories/{docRefId}`
-- `DELETE /histories/{docRefId}`
+- `POST: /histories`: Create a history record of the code submission
+- `GET: /histories/{historyDocRefId}`: Reads the history record of the code submission, identified by the history reference ID
+- `GET: /histories/user/{username}`: Returns a paginated list of history records by a user, identified by the username
+- `GET: /histories/user/{username}/question/{questionDocRefId}`: Returns a paginated list of history records by a user, for a question, identified by the username and question reference ID
 
 ```bash
 go run main.go