|
| 1 | +# 1. **Message-Based Delivery Microservices Application** |
| 2 | + |
| 3 | +<p align="center"> |
| 4 | + <a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="200" alt="Nest Logo" /></a> |
| 5 | +</p> |
| 6 | + |
| 7 | +## Project Overview |
| 8 | + |
| 9 | +This project is a delivery service microservices architecture implemented using the NestJS framework. It provides various functionalities including user management, product management, order processing, payment processing, and notification services. This application follows microservices patterns for scalability, maintainability, and fault isolation. |
| 10 | + |
| 11 | +## Architecture |
| 12 | + |
| 13 | +This application consists of the following microservices: |
| 14 | + |
| 15 | +1. **Gateway Service**: Acts as an API gateway that handles all client requests and routes them to appropriate microservices. It processes JWT-based authentication and provides API endpoints. |
| 16 | +2. **User Service**: Responsible for user management and authentication. Provides user registration, login, and profile management features. Uses PostgreSQL database. |
| 17 | +3. **Product Service**: Handles product information management. Provides product listing and detailed information retrieval features. Uses PostgreSQL database. |
| 18 | +4. **Order Service**: Handles order processing. Provides order creation, order status management, and delivery information management features. Uses MongoDB database. |
| 19 | +5. **Payment-Command Service**: Handles payment processing. Provides payment processing and payment status management features. Uses PostgreSQL database along with MongoDB. |
| 20 | +6. **Payment-Query Service**: Handles payment information retrieval. Provides payment history inquiry features. Configured to use MongoDB in development environment and PostgreSQL in production environment. |
| 21 | +7. **Notification Service**: Handles notification processing. Processes notifications for events such as order status changes and payment completion. Uses MongoDB database. |
| 22 | + |
| 23 | +### Inter-Service Communication |
| 24 | + |
| 25 | +- **Synchronous Communication**: Direct communication between services is done using gRPC protocol. Each service communicates through interfaces defined in proto files. |
| 26 | +- **Asynchronous Communication**: Event-based communication is done through Kafka. This maintains loose coupling between services. |
| 27 | + |
| 28 | +### Data Storage |
| 29 | + |
| 30 | +- **PostgreSQL**: Used by User, Product, and Payment-Command services for storing relational data. |
| 31 | +- **MongoDB**: Used by Order, Notification, and Payment-Query services for storing document-based data. Payment-Command service also stores some data here. |
| 32 | +- **Redis**: Used for caching and session management. |
| 33 | + |
| 34 | +## Technology Stack |
| 35 | + |
| 36 | +- **Backend Framework**: NestJS (Node.js based) |
| 37 | +- **Language**: TypeScript |
| 38 | +- **Package Manager**: pnpm v9.1.1 |
| 39 | +- **Databases**: |
| 40 | + - PostgreSQL v16: Relational data storage |
| 41 | + - MongoDB v8: Document-based data storage |
| 42 | +- **Inter-Service Communication**: |
| 43 | + - gRPC: Synchronous inter-service communication |
| 44 | + - Kafka v3.8.0: Asynchronous event-based communication |
| 45 | +- **Caching**: Redis |
| 46 | +- **Containerization and Orchestration**: |
| 47 | + - Docker, Docker Compose: Local development and testing |
| 48 | + - Kubernetes: Production deployment |
| 49 | + - Helm: Kubernetes resource management |
| 50 | +- **Authentication and Security**: JWT (JSON Web Tokens) |
| 51 | +- **Testing**: Jest |
| 52 | + |
| 53 | +## Prerequisites |
| 54 | + |
| 55 | +- Node.js (v16 or higher) |
| 56 | +- pnpm (v9.1.1 or higher) |
| 57 | +- Docker and Docker Compose |
| 58 | +- Git |
| 59 | + |
| 60 | +## Installation |
| 61 | + |
| 62 | +1. Clone the repository: |
| 63 | + |
| 64 | +```bash |
| 65 | +$ git clone <repository-url> |
| 66 | +$ cd delivery |
| 67 | +``` |
| 68 | + |
| 69 | +2. Install dependencies: |
| 70 | + |
| 71 | +```bash |
| 72 | +$ pnpm install |
| 73 | +``` |
| 74 | + |
| 75 | +3. Set up environment variables for each service: |
| 76 | + - `envs/gateway/.env`: API gateway configuration (JWT secret, port, etc.) |
| 77 | + - `envs/user/.env`: User service configuration (database connection info, etc.) |
| 78 | + - `envs/product/.env`: Product service configuration (database connection info, etc.) |
| 79 | + - `envs/order/.env`: Order service configuration (database connection info, etc.) |
| 80 | + - `envs/payment-command/.env`: Payment command service configuration (database connection info, etc.) |
| 81 | + - `envs/payment-query/.env`: Payment query service configuration (database connection info, etc.) |
| 82 | + - `envs/notification/.env`: Notification service configuration (database connection info, etc.) |
| 83 | + |
| 84 | +Each environment variable file should include the following information: |
| 85 | +- Database connection information (host, port, username, password, database name) |
| 86 | +- gRPC server configuration (host, port) |
| 87 | +- Other service-specific configurations |
| 88 | + |
| 89 | +## Running the Application |
| 90 | + |
| 91 | +### Local Development Environment |
| 92 | + |
| 93 | +```bash |
| 94 | +# Development mode |
| 95 | +$ pnpm run start |
| 96 | + |
| 97 | +# Development mode (watch mode) |
| 98 | +$ pnpm run start:dev |
| 99 | + |
| 100 | +# Production mode |
| 101 | +$ pnpm run start:prod |
| 102 | +``` |
| 103 | + |
| 104 | +### Using Docker Compose |
| 105 | + |
| 106 | +The project includes three Docker Compose configuration files: |
| 107 | + |
| 108 | +1. **docker-compose.yml**: Basic configuration file for development environment that builds directly from source code. |
| 109 | +2. **docker-compose.image-test.yml**: Configuration file for testing pre-built Docker images. |
| 110 | +3. **docker-compose.prod.yml**: Simplified configuration file for production environment that uses external databases. |
| 111 | + |
| 112 | +#### Development Environment |
| 113 | + |
| 114 | +Run all services and databases at once: |
| 115 | + |
| 116 | +```bash |
| 117 | +$ docker-compose up |
| 118 | +``` |
| 119 | + |
| 120 | +Run specific services only: |
| 121 | + |
| 122 | +```bash |
| 123 | +$ docker-compose up gateway user |
| 124 | +``` |
| 125 | + |
| 126 | +Run in background: |
| 127 | + |
| 128 | +```bash |
| 129 | +$ docker-compose up -d |
| 130 | +``` |
| 131 | + |
| 132 | +#### Image Testing |
| 133 | + |
| 134 | +Test pre-built Docker images: |
| 135 | + |
| 136 | +```bash |
| 137 | +$ docker-compose -f docker-compose.image-test.yml up |
| 138 | +``` |
| 139 | + |
| 140 | +#### Production Environment |
| 141 | + |
| 142 | +Run in production environment: |
| 143 | + |
| 144 | +```bash |
| 145 | +$ docker-compose -f docker-compose.prod.yml up -d |
| 146 | +``` |
| 147 | + |
| 148 | +### Service Access |
| 149 | + |
| 150 | +- Gateway API: http://localhost:3000 |
| 151 | +- Each service communicates through internal gRPC ports. |
| 152 | +- Database ports: |
| 153 | + - PostgreSQL (User): 6001 |
| 154 | + - PostgreSQL (Product): 6002 |
| 155 | + - MongoDB (Order): 6003 |
| 156 | + - PostgreSQL (Payment-Command): 6005 |
| 157 | + - MongoDB (Payment-Command): 6010 |
| 158 | + - MongoDB (Payment-Query): 6011 |
| 159 | + - MongoDB (Notification): 6006 |
| 160 | +- Kafka: |
| 161 | + - Internal communication port: 9092 |
| 162 | + - External access port: 29092 |
| 163 | + |
| 164 | +## Testing |
| 165 | + |
| 166 | +```bash |
| 167 | +# Unit tests |
| 168 | +$ pnpm run test |
| 169 | + |
| 170 | +# e2e tests |
| 171 | +$ pnpm run test:e2e |
| 172 | + |
| 173 | +# Test coverage |
| 174 | +$ pnpm run test:cov |
| 175 | +``` |
| 176 | + |
| 177 | +## API Documentation |
| 178 | + |
| 179 | +API documentation is provided as a Postman collection. You can import the `docs/NestJS Microservice.postman_environment.json` file into Postman for use. |
| 180 | + |
| 181 | +Main API endpoints: |
| 182 | +- Authentication: `/auth/register`, `/auth/login` |
| 183 | +- Products: `/product` |
| 184 | +- Orders: `/order` |
| 185 | +- Payment Command: `/payment-command` |
| 186 | +- Payment Query: `/payment-query` |
| 187 | +- Notifications: `/notification` |
| 188 | + |
| 189 | +## Project Structure |
| 190 | + |
| 191 | +``` |
| 192 | +delivery-microservices-messaging/ |
| 193 | +├── .github/ # GitHub related configurations |
| 194 | +├── .idea/ # IDE settings (JetBrains) |
| 195 | +├── apps/ # Microservice applications |
| 196 | +│ ├── gateway/ # API Gateway (Port: 3000) |
| 197 | +│ │ ├── src/ # Source code |
| 198 | +│ │ │ ├── auth/ # Authentication related code |
| 199 | +│ │ │ ├── health/ # Health check related code |
| 200 | +│ │ │ ├── order/ # Order related code |
| 201 | +│ │ │ └── product/ # Product related code |
| 202 | +│ │ ├── .env # Development environment variables |
| 203 | +│ │ ├── prod.env # Production environment variables |
| 204 | +│ │ └── Dockerfile # Docker build configuration |
| 205 | +│ ├── user/ # User service |
| 206 | +│ │ ├── src/ # Source code |
| 207 | +│ │ │ ├── auth/ # Authentication related code |
| 208 | +│ │ │ └── user/ # User related code |
| 209 | +│ │ ├── .env # Development environment variables |
| 210 | +│ │ ├── prod.env # Production environment variables |
| 211 | +│ │ └── Dockerfile # Docker build configuration |
| 212 | +│ ├── product/ # Product service |
| 213 | +│ │ ├── src/ # Source code |
| 214 | +│ │ │ └── product/ # Product related code |
| 215 | +│ │ ├── .env # Development environment variables |
| 216 | +│ │ ├── prod.env # Production environment variables |
| 217 | +│ │ └── Dockerfile # Docker build configuration |
| 218 | +│ ├── order/ # Order service |
| 219 | +│ │ ├── src/ # Source code |
| 220 | +│ │ │ └── order/ # Order related code |
| 221 | +│ │ ├── .env # Development environment variables |
| 222 | +│ │ ├── prod.env # Production environment variables |
| 223 | +│ │ └── Dockerfile # Docker build configuration |
| 224 | +│ ├── payment-command/ # Payment command service |
| 225 | +│ │ ├── src/ # Source code |
| 226 | +│ │ │ └── payment/ # Payment related code |
| 227 | +│ │ ├── .env # Development environment variables |
| 228 | +│ │ ├── prod.env # Production environment variables |
| 229 | +│ │ └── Dockerfile # Docker build configuration |
| 230 | +│ ├── payment-query/ # Payment query service |
| 231 | +│ │ ├── src/ # Source code |
| 232 | +│ │ │ └── payment/ # Payment related code |
| 233 | +│ │ ├── .env # Development environment variables |
| 234 | +│ │ ├── prod.env # Production environment variables |
| 235 | +│ │ └── Dockerfile # Docker build configuration |
| 236 | +│ └── notification/ # Notification service |
| 237 | +│ ├── src/ # Source code |
| 238 | +│ │ └── notification/ # Notification related code |
| 239 | +│ ├── .env # Development environment variables |
| 240 | +│ ├── prod.env # Production environment variables |
| 241 | +│ └── Dockerfile # Docker build configuration |
| 242 | +├── envs/ # Environment variable configuration files |
| 243 | +│ ├── gateway/ # Gateway environment variables |
| 244 | +│ ├── user/ # User service environment variables |
| 245 | +│ ├── product/ # Product service environment variables |
| 246 | +│ ├── order/ # Order service environment variables |
| 247 | +│ ├── payment-command/ # Payment command service environment variables |
| 248 | +│ ├── payment-query/ # Payment query service environment variables |
| 249 | +│ └── notification/ # Notification service environment variables |
| 250 | +├── docs/ # Documentation |
| 251 | +│ ├── auth-login-scripts.md # Authentication and login scripts documentation |
| 252 | +│ ├── dockerhub_image_push.txt # Docker Hub image push guide |
| 253 | +│ ├── NestJS Microservice.postman_environment.json # Postman environment configuration |
| 254 | +│ └── post_order.md # Order creation related documentation |
| 255 | +├── k8s/ # Kubernetes and Helm related configurations |
| 256 | +│ ├── delivery/ # Delivery service Kubernetes configuration |
| 257 | +│ ├── helm/ # Helm charts |
| 258 | +│ ├── infra/ # Infrastructure related configuration |
| 259 | +│ └── kubernetes/ # Basic Kubernetes configuration |
| 260 | +├── libs/ # Shared libraries |
| 261 | +│ └── common/ # Common modules |
| 262 | +│ ├── src/ # Source code |
| 263 | +│ │ ├── const/ # Constants |
| 264 | +│ │ ├── dto/ # Data Transfer Objects |
| 265 | +│ │ ├── grpc/ # gRPC related code |
| 266 | +│ │ └── interceptor/ # Interceptors |
| 267 | +│ └── tsconfig.lib.json # TypeScript configuration |
| 268 | +├── node_modules/ # Node modules (package dependencies) |
| 269 | +├── postgres/ # PostgreSQL data directory |
| 270 | +│ ├── user/ # User service data |
| 271 | +│ ├── product/ # Product service data |
| 272 | +│ └── payment_command/ # Payment command service data |
| 273 | +├── mongo/ # MongoDB data directory |
| 274 | +│ ├── order/ # Order service data |
| 275 | +│ ├── payment_command/ # Payment command service data |
| 276 | +│ ├── payment_query/ # Payment query service data |
| 277 | +│ └── notification/ # Notification service data |
| 278 | +├── proto/ # gRPC protocol definitions |
| 279 | +│ ├── notification.proto # Notification service protocol |
| 280 | +│ ├── order.proto # Order service protocol |
| 281 | +│ ├── payment.proto # Payment service protocol |
| 282 | +│ ├── product.proto # Product service protocol |
| 283 | +│ └── user.proto # User service protocol |
| 284 | +├── .dockerignore # Docker build exclusion file |
| 285 | +├── .eslintrc.js # ESLint configuration |
| 286 | +├── .gitignore # Git exclusion file |
| 287 | +├── .prettierrc # Prettier configuration |
| 288 | +├── build-and-push-ps.ps1 # PowerShell script (Docker image build and push) |
| 289 | +├── build-and-push.sh # Bash script (Docker image build and push) |
| 290 | +├── docker-compose.image-test.yml # Docker Compose image test configuration |
| 291 | +├── docker-compose.prod.yml # Docker Compose production configuration |
| 292 | +├── docker-compose.yml # Docker Compose development configuration |
| 293 | +├── nest-cli.json # NestJS CLI configuration |
| 294 | +├── package.json # Project metadata and dependencies |
| 295 | +├── pnpm-lock.yaml # pnpm lock file |
| 296 | +├── README.md # Project documentation |
| 297 | +├── run-docker-compose.ps1 # PowerShell script (Docker Compose execution) |
| 298 | +├── tsconfig.build.json # TypeScript build configuration |
| 299 | +├── tsconfig.json # TypeScript basic configuration |
| 300 | +└── webpack.config.js # Webpack configuration |
| 301 | +``` |
| 302 | + |
| 303 | +## Messaging System (Kafka) |
| 304 | + |
| 305 | +This project uses Kafka for asynchronous communication between services. Kafka is a distributed event streaming platform that provides high throughput, durability, and scalability. |
| 306 | + |
| 307 | +### Kafka Configuration |
| 308 | + |
| 309 | +In development and test environments, Kafka is automatically configured through Docker Compose: |
| 310 | + |
| 311 | +- Kafka version: 3.8.0 (Bitnami image) |
| 312 | +- Internal communication port: 9092 |
| 313 | +- External access port: 29092 |
| 314 | +- KRaft mode enabled (runs without ZooKeeper) |
| 315 | + |
| 316 | +### Key Event Flows |
| 317 | + |
| 318 | +1. Order Creation → Payment Processing → Notification Sending |
| 319 | +2. Payment Completion → Order Status Update → Notification Sending |
| 320 | +3. Order Status Change → Notification Sending |
| 321 | + |
| 322 | +Each service produces and consumes relevant events to implement a loosely coupled microservices architecture. |
| 323 | + |
| 324 | +## Kubernetes Deployment |
| 325 | + |
| 326 | +The project includes configuration files for Kubernetes deployment. In the `k8s/kubernetes` directory, you can find examples of the following Kubernetes resources: |
| 327 | + |
| 328 | +- Pods |
| 329 | +- ReplicaSets |
| 330 | +- Deployments |
| 331 | +- Namespaces |
| 332 | +- ConfigMaps and Secrets |
| 333 | +- Liveness Probes and Readiness Probes |
| 334 | +- Services (NodePort, ClusterIP) |
| 335 | +- Persistent Volumes and Persistent Volume Claims |
| 336 | + |
| 337 | +You can also find deployment examples using Helm charts in the `k8s/helm` directory. Actual deployment configurations can be found in the `k8s/delivery` directory, and infrastructure-related configurations are in the `k8s/infra` directory. |
| 338 | + |
| 339 | +## Contributing |
| 340 | + |
| 341 | +1. Fork this repository. |
| 342 | +2. Create a new branch: `git checkout -b feature/your-feature-name` |
| 343 | +3. Commit your changes: `git commit -m 'Add some feature'` |
| 344 | +4. Push to your forked repository: `git push origin feature/your-feature-name` |
| 345 | +5. Create a Pull Request. |
| 346 | + |
| 347 | +## License |
| 348 | + |
| 349 | +This project follows the [MIT License](LICENSE). |
0 commit comments