Skip to content

lab5#65

Merged
Alexeyzem merged 5 commits intomainfrom
zemliakov/lab5
Dec 18, 2025
Merged

lab5#65
Alexeyzem merged 5 commits intomainfrom
zemliakov/lab5

Conversation

@Alexeyzem
Copy link
Copy Markdown
Collaborator

No description provided.

Copilot AI review requested due to automatic review settings December 11, 2025 09:38
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR sets up an Apache Airflow environment for Lab 5, implementing an HR data pipeline that generates and processes employee data. The implementation includes Docker containerization with CeleryExecutor and PostgreSQL, along with three DAGs for schema initialization, data generation, and metric calculation.

Key Changes

  • Docker-based Airflow deployment with CeleryExecutor, Redis, and PostgreSQL
  • Three DAGs: schema initialization, random employee data generation (10,000 records), and monthly hiring metrics calculation
  • Custom Dockerfile with PostgreSQL provider dependencies

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
requirements.txt Adds PostgreSQL provider for Airflow database operations
docker-compose.yaml Configures multi-service Airflow cluster with webserver, scheduler, worker, and triggerer
init_schema.py Creates database tables for employees and monthly hiring metrics
init_data.py Generates 10,000 random employee records with Russian names
collect_data.py Calculates and stores monthly hiring statistics
Dockerfile Builds custom Airflow image with system dependencies and Python packages
.env Sets Airflow user ID for container permissions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Alexeyzem Alexeyzem merged commit 621fc92 into main Dec 18, 2025
1 check passed
@Alexeyzem Alexeyzem deleted the zemliakov/lab5 branch December 18, 2025 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants