3. Deploy as a CRON Job in Render
Note: 1 cron job for each pipeline
Step 1: Connect to GitHub
Step 2: Select GitHub Repo Step 3: Create CRON Job Name and select a Render Project (if applicable) (useful if there are many services in render under the same project) (corn, web service, etc)
Step 4: Set Schedule on the Schedule tab using a CRON expression * * * * *
1 0 * * * -- 12:01 AM daily
or
1 0 1 1 * -- 12:01 AM every January 1st
Step 5: Specify the command
python scripts/run_incremental_pipeline.py
or
export PYTHONPATH="/opt/render/project/src/src" && python scripts/run_full_pipeline.py
Step 6: Select Instance
Step 7: Add environment variables (can be uploaded from a .env file)
Step 8: Deploy CRON Job
Step 9: Optional -- Manually Trigger a CRON JOB by clicking Trigger Run on the Render Service Dashboard
