Skip to content

Commit 5a3852d

Browse files
RonaldRonnieRonaldRonnie
committed
Create GitHub Actions CI/CD pipeline
- Created .github/workflows/ci.yml with comprehensive CI/CD pipeline - Test job: Runs tests on Python 3.8, 3.9, 3.10, 3.11 - Lint job: Code quality checks with black and flake8 - Type-check job: Type checking with mypy - Security job: Security scanning with bandit and safety - Docker-build job: Builds and tests Docker image - All-checks job: Summary of all checks - Features: - Tests run on all supported Python versions (3.8-3.11) - Automated linting (black, flake8) - Automated type checking (mypy) - Security scanning (bandit, safety) - Docker image build and test - Code coverage reporting - Workflow runs on PR and push to main/develop - Added status badges to README: - CI/CD Pipeline status badge - Code Quality (Black) badge Fixes #49
1 parent 5f2ff5c commit 5a3852d

File tree

2 files changed

+233
-0
lines changed

2 files changed

+233
-0
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
name: CI/CD Pipeline
2+
3+
on:
4+
push:
5+
branches: [ main, develop ]
6+
pull_request:
7+
branches: [ main, develop ]
8+
9+
env:
10+
PYTHON_DEFAULT_VERSION: "3.11"
11+
12+
jobs:
13+
test:
14+
name: Test Python ${{ matrix.python-version }}
15+
runs-on: ubuntu-latest
16+
strategy:
17+
fail-fast: false
18+
matrix:
19+
python-version: ["3.8", "3.9", "3.10", "3.11"]
20+
21+
steps:
22+
- name: Checkout code
23+
uses: actions/checkout@v4
24+
25+
- name: Set up Python ${{ matrix.python-version }}
26+
uses: actions/setup-python@v5
27+
with:
28+
python-version: ${{ matrix.python-version }}
29+
cache: 'pip'
30+
31+
- name: Install system dependencies
32+
run: |
33+
sudo apt-get update
34+
sudo apt-get install -y gcc g++ curl git
35+
36+
- name: Upgrade pip, setuptools, and wheel
37+
run: |
38+
python -m pip install --upgrade pip setuptools wheel build
39+
40+
- name: Install PyTorch CPU (for faster CI)
41+
run: |
42+
pip install --no-cache-dir --default-timeout=600 \
43+
--extra-index-url https://download.pytorch.org/whl/cpu \
44+
torch==2.1.0+cpu \
45+
torchvision==0.16.0+cpu \
46+
torchaudio==2.1.0+cpu
47+
48+
- name: Install project dependencies
49+
run: |
50+
pip install --no-cache-dir -e ".[dev]"
51+
# Install remaining dependencies that aren't in dev extras
52+
pip install --no-cache-dir \
53+
transformers>=4.34.0 \
54+
scikit-learn>=1.3.0 \
55+
pandas>=2.1.1 \
56+
numpy>=1.26.0 \
57+
biopython>=1.81 \
58+
sentencepiece>=0.1.99 \
59+
accelerate>=0.24.0 \
60+
datasets>=2.14.0 \
61+
google-generativeai>=0.7.2 \
62+
tiktoken>=0.5.0 \
63+
litellm>=1.50.0 \
64+
paper-qa>=5.0.0 \
65+
requests>=2.31.0 \
66+
beautifulsoup4>=4.12.2 \
67+
lxml>=4.9.0 \
68+
openpyxl>=3.1.0 \
69+
xlrd>=2.0.1 \
70+
tqdm>=4.65.0 \
71+
python-dotenv>=1.0.0 \
72+
psutil>=5.9.0 \
73+
fastapi>=0.104.0 \
74+
"uvicorn[standard]>=0.23.2" \
75+
aiohttp>=3.8.6 \
76+
websockets>=11.0.3 \
77+
python-multipart>=0.0.5 \
78+
aiofiles>=0.7.0 \
79+
pydantic>=2.4.2 \
80+
typing-extensions>=3.10.0.2 \
81+
starlette>=0.31.1 \
82+
click>=8.0.1 \
83+
h11>=0.12.0 \
84+
httptools>=0.3.0 \
85+
PyYAML>=5.4.1 \
86+
"watchfiles[watchdog]>=1.0.0" \
87+
wsproto>=1.0.0 \
88+
tokenizers>=0.14.1 \
89+
pytz>=2023.3 \
90+
qdrant-client>=1.7.0
91+
92+
- name: Run tests
93+
env:
94+
# Set dummy API keys for testing (tests should handle missing keys gracefully)
95+
GEMINI_API_KEY: "test_key_1234567890abcdef"
96+
NCBI_API_KEY: "test_ncbi_key_1234567890"
97+
EMAIL: "test@example.com"
98+
run: |
99+
pytest tests/ -v --tb=short --cov=app --cov-report=xml --cov-report=term-missing
100+
101+
- name: Upload coverage to Codecov
102+
uses: codecov/codecov-action@v4
103+
with:
104+
file: ./coverage.xml
105+
flags: unittests
106+
name: codecov-umbrella
107+
fail_ci_if_error: false
108+
109+
lint:
110+
name: Lint Code
111+
runs-on: ubuntu-latest
112+
113+
steps:
114+
- name: Checkout code
115+
uses: actions/checkout@v4
116+
117+
- name: Set up Python
118+
uses: actions/setup-python@v5
119+
with:
120+
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
121+
cache: 'pip'
122+
123+
- name: Install linting dependencies
124+
run: |
125+
python -m pip install --upgrade pip
126+
pip install black>=23.0.0 flake8>=6.0.0
127+
128+
- name: Check code formatting with Black
129+
run: |
130+
black --check --diff app/ tests/ cli.py main.py
131+
132+
- name: Lint with flake8
133+
run: |
134+
flake8 app/ tests/ cli.py main.py --max-line-length=120 --extend-ignore=E203,W503 --exclude=__pycache__,*.pyc
135+
136+
type-check:
137+
name: Type Check
138+
runs-on: ubuntu-latest
139+
140+
steps:
141+
- name: Checkout code
142+
uses: actions/checkout@v4
143+
144+
- name: Set up Python
145+
uses: actions/setup-python@v5
146+
with:
147+
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
148+
cache: 'pip'
149+
150+
- name: Install type checking dependencies
151+
run: |
152+
python -m pip install --upgrade pip
153+
pip install mypy>=1.5.0 types-requests types-PyYAML
154+
155+
- name: Run mypy
156+
run: |
157+
mypy app/ --ignore-missing-imports --no-strict-optional --show-error-codes || true
158+
continue-on-error: true # Don't fail CI on type errors for now
159+
160+
security:
161+
name: Security Scan
162+
runs-on: ubuntu-latest
163+
164+
steps:
165+
- name: Checkout code
166+
uses: actions/checkout@v4
167+
168+
- name: Set up Python
169+
uses: actions/setup-python@v5
170+
with:
171+
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
172+
cache: 'pip'
173+
174+
- name: Install security scanning tools
175+
run: |
176+
python -m pip install --upgrade pip
177+
pip install bandit safety
178+
179+
- name: Run Bandit security scan
180+
run: |
181+
bandit -r app/ -f json -o bandit-report.json || true
182+
bandit -r app/ -ll || true
183+
continue-on-error: true
184+
185+
- name: Check dependencies with Safety
186+
run: |
187+
pip install -e ".[dev]"
188+
safety check --json || true
189+
safety check || true
190+
continue-on-error: true
191+
192+
docker-build:
193+
name: Docker Build Test
194+
runs-on: ubuntu-latest
195+
196+
steps:
197+
- name: Checkout code
198+
uses: actions/checkout@v4
199+
200+
- name: Set up Docker Buildx
201+
uses: docker/setup-buildx-action@v3
202+
203+
- name: Build Docker image
204+
uses: docker/build-push-action@v5
205+
with:
206+
context: .
207+
file: ./Dockerfile
208+
push: false
209+
tags: bioanalyzer-backend:test
210+
cache-from: type=gha
211+
cache-to: type=gha,mode=max
212+
213+
- name: Test Docker image
214+
run: |
215+
docker run --rm bioanalyzer-backend:test python -c "import app; print('Docker image works!')"
216+
217+
all-checks:
218+
name: All Checks Summary
219+
runs-on: ubuntu-latest
220+
needs: [test, lint, type-check, security, docker-build]
221+
if: always()
222+
223+
steps:
224+
- name: Check job status
225+
run: |
226+
echo "Test job: ${{ needs.test.result }}"
227+
echo "Lint job: ${{ needs.lint.result }}"
228+
echo "Type-check job: ${{ needs.type-check.result }}"
229+
echo "Security job: ${{ needs.security.result }}"
230+
echo "Docker-build job: ${{ needs.docker-build.result }}"
231+

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# BioAnalyzer Backend
22

3+
[![CI/CD Pipeline](https://github.com/waldronlab/bioanalyzer-backend/actions/workflows/ci.yml/badge.svg)](https://github.com/waldronlab/bioanalyzer-backend/actions/workflows/ci.yml)
34
[![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://python.org)
45
[![FastAPI](https://img.shields.io/badge/FastAPI-0.104+-green.svg)](https://fastapi.tiangolo.com)
56
[![Docker](https://img.shields.io/badge/Docker-20.0+-blue.svg)](https://docker.com)
67
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
8+
[![Code Quality](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
79

810
Backend system for analyzing scientific papers to identify curatable microbiome signatures. Extracts essential BugSigDB fields and retrieves full text from PubMed/PMC.
911

0 commit comments

Comments
 (0)