Skip to content

Commit ecf4cb7

Browse files
kaiz-iokaiz-io
committed
Add documentation: README and SUMMARY
1 parent 55fdf52 commit ecf4cb7

File tree

2 files changed

+318
-0
lines changed

2 files changed

+318
-0
lines changed

‎typescript/postgres-lambda/README.md‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
# PostgreSQL and Lambda Integration Example
2+
3+
A complete AWS CDK example demonstrating bidirectional integration between Aurora PostgreSQL Serverless v2 and AWS Lambda functions.
4+
5+
## What This Example Demonstrates
6+
7+
- **Lambda → PostgreSQL**: Lambda function that connects to and queries PostgreSQL
8+
- **PostgreSQL → Lambda**: PostgreSQL database that invokes Lambda functions using the `aws_lambda` extension
9+
- **Secure Architecture**: Private subnets, IAM roles, and Secrets Manager integration
10+
- **Production-Ready**: Includes error handling, connection pooling, and security best practices
11+
- **Automated Setup**: Custom CDK resource automatically configures PostgreSQL extensions and functions
12+
13+
## Architecture
14+
15+
```mermaid
16+
graph TD
17+
subgraph VPC
18+
subgraph "Private Subnet"
19+
DB[Aurora PostgreSQL\nServerless v2]
20+
L1[Lambda Function\nLambdaToPostgres]
21+
L2[Lambda Function\nPostgresFunction]
22+
L3[Lambda Function\nPostgresSetup]
23+
end
24+
end
25+
26+
L1 -->|"(1) Connect and Query"| DB
27+
DB -->|"(2) Invoke via aws_lambda extension"| L2
28+
L2 -->|"(3) Return Result"| DB
29+
L3 -->|"(4) Setup Extensions & Functions"| DB
30+
31+
SM[AWS Secrets Manager] -->|Provide Credentials| L1
32+
SM -->|Provide Credentials| L3
33+
34+
style DB fill:#E3F2FD,stroke:#1976D2,stroke-width:2px,color:#000
35+
style L1 fill:#FFF3E0,stroke:#F57C00,stroke-width:2px,color:#000
36+
style L2 fill:#FFF3E0,stroke:#F57C00,stroke-width:2px,color:#000
37+
style L3 fill:#E8F5E8,stroke:#4CAF50,stroke-width:2px,color:#000
38+
style SM fill:#F3E5F5,stroke:#7B1FA2,stroke-width:2px,color:#000
39+
```
40+
41+
**Components:**
42+
- Aurora PostgreSQL Serverless v2 cluster (private subnet)
43+
- Lambda function for database operations (`LambdaToPostgres`)
44+
- Lambda function invokable from PostgreSQL (`PostgresFunction`)
45+
- Lambda function for automated setup (`PostgresSetupFunction`)
46+
- IAM roles with least-privilege permissions
47+
- Security groups for network access control
48+
- AWS Secrets Manager for credential storage
49+
- Custom CDK resource for automated PostgreSQL configuration
50+
51+
## Quick Start
52+
53+
### Prerequisites
54+
55+
- AWS CDK v2 installed (`npm install -g aws-cdk`)
56+
- Node.js 18.x or later
57+
- AWS CLI configured with appropriate credentials
58+
59+
### Deploy
60+
61+
```bash
62+
# Install dependencies
63+
npm install
64+
65+
# Deploy the stack (setup is now automated!)
66+
npx cdk deploy
67+
```
68+
69+
The deployment will automatically:
70+
- Create the Aurora PostgreSQL cluster
71+
- Deploy all Lambda functions
72+
- Configure PostgreSQL extensions and functions
73+
- Set up all necessary permissions
74+
75+
No manual setup required! 🎉
76+
77+
## Testing
78+
79+
### Test Lambda → PostgreSQL
80+
81+
Using the provided test script:
82+
```bash
83+
./test-lambda.sh --function-name <LAMBDA_TO_POSTGRES_FUNCTION_NAME> --message "Hello World"
84+
```
85+
86+
Or using AWS CLI directly:
87+
```bash
88+
aws lambda invoke \
89+
--function-name <LAMBDA_TO_POSTGRES_FUNCTION_NAME> \
90+
--payload '{"message": "Hello from CLI!"}' \
91+
response.json && cat response.json
92+
```
93+
94+
### Test PostgreSQL → Lambda
95+
96+
Connect to PostgreSQL and test the functions:
97+
```bash
98+
psql -h <DB_CLUSTER_ENDPOINT> -U postgres -d demodb
99+
```
100+
101+
```sql
102+
-- Test the PostgreSQL to Lambda integration
103+
SELECT process_data('{"id": 123, "value": "test"}'::JSONB);
104+
SELECT transform_data('{"id": 456, "value": "hello world"}'::JSONB);
105+
SELECT validate_data('{"id": 789, "value": "valid data"}'::JSONB);
106+
```
107+
108+
## How It Works
109+
110+
### Automated Setup Process
111+
112+
1. **CDK Deployment**: Stack creates all resources including a setup Lambda function
113+
2. **Custom Resource**: Triggers the setup Lambda after database is ready
114+
3. **Extension Installation**: Setup function creates the `aws_lambda` extension
115+
4. **Function Creation**: Creates SQL functions that wrap Lambda invocations
116+
5. **Ready to Use**: Database is immediately ready for bidirectional Lambda integration
117+
118+
### Lambda to PostgreSQL Flow
119+
120+
1. **Credential Retrieval**: Function retrieves DB credentials from Secrets Manager
121+
2. **Connection**: Establishes secure SSL connection to PostgreSQL
122+
3. **Table Management**: Creates demo table if it doesn't exist
123+
4. **Data Operations**: Inserts message and queries recent records
124+
5. **Response**: Returns formatted results with error handling
125+
126+
### PostgreSQL to Lambda Flow
127+
128+
1. **Extension Setup**: Uses `aws_lambda` extension for Lambda invocation (automated)
129+
2. **Function Creation**: SQL functions wrap Lambda calls with proper ARN construction (automated)
130+
3. **Event Processing**: Lambda receives structured JSON events from PostgreSQL
131+
4. **Result Return**: Lambda response becomes available in SQL query results
132+
133+
## Project Structure
134+
135+
```
136+
├── bin/ # CDK app entry point
137+
├── lib/ # CDK stack definition
138+
├── lambda/ # Lambda function source code
139+
│ ├── lambda-to-postgres/ # Lambda that calls PostgreSQL
140+
│ ├── postgres-to-lambda/ # Lambda called by PostgreSQL
141+
│ └── postgres-setup/ # Lambda for automated setup
142+
├── test/ # Unit tests
143+
├── setup-postgres-lambda.sql # Reference SQL (now automated)
144+
├── test-lambda.sh # Lambda testing script
145+
└── README.md # This file
146+
```
147+
148+
## Configuration
149+
150+
### Environment Variables
151+
152+
The Lambda functions use these environment variables (set automatically by CDK):
153+
154+
- `DB_SECRET_ARN`: ARN of the database credentials secret
155+
- `DB_NAME`: Database name (default: `demodb`)
156+
- `POSTGRES_FUNCTION_NAME`: Name of the Lambda function called by PostgreSQL
157+
- `AWS_REGION`: AWS region for Lambda ARN construction
158+
159+
### Customization
160+
161+
- **Database Configuration**: Modify `lib/postgres-lambda-stack.ts`
162+
- **Lambda Logic**: Update files in `lambda/` directories
163+
- **Setup SQL**: Customize `lambda/postgres-setup/index.js`
164+
165+
## Security Features
166+
167+
✅ **Network Security**
168+
- Database in private subnets
169+
- Security groups with minimal required access
170+
- No direct internet access to database
171+
172+
✅ **Access Control**
173+
- IAM roles with least-privilege permissions
174+
- Secrets Manager for credential storage
175+
- SSL/TLS encryption for database connections
176+
177+
✅ **Monitoring**
178+
- CloudWatch logs for all Lambda functions
179+
- Database performance insights available
180+
- VPC Flow Logs (can be enabled)
181+
182+
## Production Considerations
183+
184+
Before using in production:
185+
186+
- [ ] Enable SSL certificate validation (`rejectUnauthorized: true`)
187+
- [ ] Implement connection pooling (consider RDS Proxy)
188+
- [ ] Set up proper monitoring and alerting
189+
- [ ] Configure backup and disaster recovery
190+
- [ ] Review and tighten IAM policies
191+
- [ ] Enable database encryption at rest
192+
- [ ] Set up VPC endpoints for AWS services
193+
- [ ] Implement proper error handling and retry logic
194+
195+
## Troubleshooting
196+
197+
### Common Issues
198+
199+
**Connection Timeouts**
200+
- Check security group rules
201+
- Verify Lambda is in correct VPC/subnets
202+
- Confirm database is running
203+
204+
**Permission Errors**
205+
- Verify IAM roles have required permissions
206+
- Check Secrets Manager access
207+
- Confirm Lambda execution role
208+
209+
**Setup Function Issues**
210+
- Check CloudWatch logs for the PostgresSetupFunction
211+
- Verify custom resource completed successfully
212+
- Ensure database is accessible from setup Lambda
213+
214+
### Useful Commands
215+
216+
```bash
217+
# Build and watch for changes
218+
npm run watch
219+
220+
# Run tests
221+
npm run test
222+
223+
# View CloudFormation template
224+
npx cdk synth
225+
226+
# Compare deployed vs current state
227+
npx cdk diff
228+
229+
# View stack outputs
230+
aws cloudformation describe-stacks --stack-name PostgresLambdaStack --query 'Stacks[0].Outputs'
231+
232+
# Check setup function logs
233+
aws logs describe-log-groups --log-group-name-prefix /aws/lambda/PostgresLambdaStack-PostgresSetupFunction
234+
```
235+
236+
## Cleanup
237+
238+
```bash
239+
npx cdk destroy
240+
```
241+
242+
**Note**: This will delete all resources including the database and any data stored in it.
243+
244+
## Cost Optimization
245+
246+
- Aurora Serverless v2 scales to zero when not in use
247+
- Lambda functions only charge for execution time
248+
- Setup function runs only once during deployment
249+
- Consider Reserved Capacity for consistent workloads
250+
- Monitor usage with AWS Cost Explorer
251+
252+
## Related Examples
253+
254+
- [Lambda with RDS Proxy](../lambda-rds-proxy/)
255+
- [Aurora Serverless v1](../aurora-serverless-v1/)
256+
- [PostgreSQL with CDK](../postgresql-cdk/)
257+
258+
## Contributing
259+
260+
1. Fork the repository
261+
2. Create a feature branch
262+
3. Make your changes
263+
4. Add tests if applicable
264+
5. Submit a pull request
265+
266+
## License
267+
268+
This example is provided under the MIT-0 License. See the LICENSE file for details.

‎typescript/postgres-lambda/SUMMARY.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# PostgreSQL and Lambda Integration Example - Summary
2+
3+
This CDK example demonstrates the integration between AWS Lambda and Aurora PostgreSQL Serverless v2. It showcases two key integration patterns:
4+
5+
## 1. Lambda to PostgreSQL
6+
7+
The first pattern demonstrates how a Lambda function can connect to and interact with a PostgreSQL database:
8+
9+
- The Lambda function (`LambdaToPostgres`) retrieves database credentials from AWS Secrets Manager
10+
- It establishes a connection to the PostgreSQL database
11+
- It creates a table if it doesn't exist, inserts data, and queries the database
12+
- The function returns the query results
13+
14+
## 2. PostgreSQL to Lambda
15+
16+
The second pattern demonstrates how PostgreSQL can invoke a Lambda function:
17+
18+
- PostgreSQL uses the `aws_lambda` extension to call Lambda functions
19+
- The Lambda function (`PostgresFunction`) receives data from PostgreSQL
20+
- It processes the data based on the action specified in the event
21+
- It returns results that can be used in SQL queries
22+
23+
## Security Features
24+
25+
The example implements several security best practices:
26+
27+
- The database is deployed in a private subnet
28+
- Security groups restrict access to the database
29+
- Credentials are stored in AWS Secrets Manager
30+
- IAM roles limit permissions to only what's necessary
31+
32+
## Helper Scripts
33+
34+
The example includes several helper scripts:
35+
36+
- `test-lambda.sh`: For testing the Lambda functions
37+
- `connect-to-postgres.sh`: For connecting to the PostgreSQL database
38+
- `setup-postgres-lambda.sql`: For setting up the PostgreSQL database to call Lambda
39+
40+
## Deployment
41+
42+
The example can be deployed with standard CDK commands:
43+
44+
```bash
45+
npm install
46+
npm run build
47+
npx cdk deploy
48+
```
49+
50+
After deployment, users need to set up the PostgreSQL database to call Lambda by creating the `aws_lambda` extension and defining functions that invoke Lambda.

0 commit comments

Comments
 (0)