Skip to content

Commit 444cb53

Browse files
vishal-balavishal-bala
authored
ci: Enable GH Actions testing for forked PRs (#462)
# Changes - Adds smoke test to ensure all modules are importable with only required dependencies - Updates workflows to use latest versions of actions - Updates testing workflow to only test on latest versions of Redis images - Tests notebooks only on `redis-py==7.x` - Adds process for supporting testing workflows on PRs from forked repos - Adds `.github/workflows/test-fork-pr.yml` - Manual workflow for testing fork PRs with secrets - Adds `.github/actions/run-service-tests/action.yml` - Reusable composite action for service tests - Updates `.github/workflows/test.yml` to use the composite action and skip service tests on fork PRs # Fork PR Testing GitHub does not provide repository secrets to workflows triggered by `pull_request` events from forks, even when maintainers approve the workflow run. This prevents our service tests from running on fork PRs since they require API keys for OpenAI, Cohere, Mistral, Voyage, Azure OpenAI, AWS, Google Cloud, and HuggingFace. This PR adds a new `Test Fork PR` workflow that maintainers can manually trigger after reviewing fork PR code. The workflow accepts a PR number as input, validates that it's from a fork, checks out the fork's code at the exact commit SHA, runs the full test suite with secrets, and posts results as a "Service Tests" check on the PR using the GitHub Checks API. This satisfies branch protection requirements while maintaining security through explicit maintainer review and SHA pinning to prevent race conditions. The service test logic has been extracted into a composite action (`.github/actions/run-service-tests/action.yml`) to avoid duplication between the regular and fork PR workflows. ### Security Considerations **Important**: The fork PR workflow runs untrusted code from external contributors with full access to repository secrets. Maintainers must carefully review fork PR code before triggering the workflow to ensure: - No malicious code that could exfiltrate secrets (e.g., logging secrets, sending them to external endpoints) - No code that modifies workflow files or test infrastructure to expose secrets - No attempts to access or modify cloud resources using the provided credentials - Test code changes are legitimate and don't introduce backdoors The workflow pins to the exact commit SHA at trigger time, preventing attackers from pushing malicious commits after approval. However, the initial review is critical since the workflow executes arbitrary Python code with access to production API keys and cloud credentials. ### Maintainer Workflow for Fork PRs When a PR is opened from an external fork, the regular test workflow runs but skips the service tests. To run the full test suite: 1. Review the fork PR code for security issues 2. Navigate to **Actions** → **Test Fork PR** workflow 3. Click **Run workflow**, enter the PR number, and click **Run workflow** again 4. Test results will appear as a "Service Tests" check on the PR If the contributor pushes new commits, review the changes and re-trigger the workflow. Same-repository PRs continue to work as before with no manual intervention required.
1 parent ec3dbe3 commit 444cb53

File tree

7 files changed

+301
-63
lines changed

7 files changed

+301
-63
lines changed

‎.github/actions/run-service-tests/action.yml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: 'Run Service Tests'
2+
description: 'Run the full RedisVL service test suite with all API integrations'
3+
4+
inputs:
5+
python-version:
6+
description: 'Python version to use'
7+
required: true
8+
uv-version:
9+
description: 'uv version to use'
10+
required: true
11+
12+
runs:
13+
using: 'composite'
14+
steps:
15+
- name: Cache HuggingFace Models
16+
uses: actions/cache@v5
17+
with:
18+
path: hf_cache
19+
key: ${{ runner.os }}-hf-cache
20+
21+
- name: Set HuggingFace token
22+
shell: bash
23+
run: |
24+
mkdir -p ~/.huggingface
25+
echo '{"token":"$HF_TOKEN"}' > ~/.huggingface/token
26+
27+
- name: Install Python
28+
uses: actions/setup-python@v6
29+
with:
30+
python-version: ${{ inputs.python-version }}
31+
32+
- name: Install uv
33+
uses: astral-sh/setup-uv@v6
34+
with:
35+
version: ${{ inputs.uv-version }}
36+
enable-cache: true
37+
python-version: ${{ inputs.python-version }}
38+
cache-dependency-glob: |
39+
pyproject.toml
40+
uv.lock
41+
42+
- name: Install required dependencies
43+
shell: bash
44+
run: |
45+
uv sync
46+
47+
- name: Test module imports
48+
shell: bash
49+
run: |
50+
uv run python -m tests.test_imports redisvl
51+
52+
- name: Install all dependencies
53+
shell: bash
54+
run: |
55+
uv sync --all-extras
56+
57+
- name: Authenticate to Google Cloud
58+
uses: google-github-actions/auth@v1
59+
with:
60+
credentials_json: ${{ env.GOOGLE_CREDENTIALS }}
61+
62+
- name: Run full test suite
63+
shell: bash
64+
run: |
65+
make test-all

‎.github/workflows/claude.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
id-token: write
2626
steps:
2727
- name: Checkout repository
28-
uses: actions/checkout@v4
28+
uses: actions/checkout@v6
2929
with:
3030
fetch-depth: 1
3131

‎.github/workflows/lint.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,10 @@ jobs:
3131

3232
steps:
3333
- name: Check out repository
34-
uses: actions/checkout@v4
34+
uses: actions/checkout@v6
3535

3636
- name: Install Python
37-
uses: actions/setup-python@v5
37+
uses: actions/setup-python@v6
3838
with:
3939
python-version: ${{ matrix.python-version }}
4040

‎.github/workflows/release.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414

1515
steps:
1616
- name: Check out repository
17-
uses: actions/checkout@v4
17+
uses: actions/checkout@v6
1818

1919
- name: Install Python
20-
uses: actions/setup-python@v5
20+
uses: actions/setup-python@v6
2121
with:
2222
python-version: ${{ env.PYTHON_VERSION }}
2323

@@ -50,10 +50,10 @@ jobs:
5050

5151
steps:
5252
- name: Check out repository
53-
uses: actions/checkout@v4
53+
uses: actions/checkout@v6
5454

5555
- name: Install Python
56-
uses: actions/setup-python@v5
56+
uses: actions/setup-python@v6
5757
with:
5858
python-version: ${{ env.PYTHON_VERSION }}
5959

‎.github/workflows/test-fork-pr.yml‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
name: Test Fork PR
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
pr_number:
7+
description: 'Pull Request number to test'
8+
required: true
9+
type: number
10+
11+
permissions:
12+
contents: read
13+
checks: write
14+
pull-requests: read
15+
16+
env:
17+
PYTHON_VERSION: "3.11"
18+
UV_VERSION: "0.7.13"
19+
20+
jobs:
21+
setup:
22+
name: Validate PR
23+
runs-on: ubuntu-latest
24+
outputs:
25+
head_sha: ${{ steps.pr-info.outputs.head_sha }}
26+
head_repo: ${{ steps.pr-info.outputs.head_repo }}
27+
steps:
28+
- name: Get PR information
29+
id: pr-info
30+
uses: actions/github-script@v7
31+
with:
32+
script: |
33+
const pr = await github.rest.pulls.get({
34+
owner: context.repo.owner,
35+
repo: context.repo.repo,
36+
pull_number: ${{ inputs.pr_number }}
37+
});
38+
39+
if (pr.data.state !== 'open') {
40+
core.setFailed(`PR #${{ inputs.pr_number }} is not open`);
41+
return;
42+
}
43+
44+
if (pr.data.head.repo.full_name === `${context.repo.owner}/${context.repo.repo}`) {
45+
core.setFailed(`PR #${{ inputs.pr_number }} is not from a fork. Use the regular workflow.`);
46+
return;
47+
}
48+
49+
core.setOutput('head_sha', pr.data.head.sha);
50+
core.setOutput('head_repo', pr.data.head.repo.full_name);
51+
52+
console.log(`Testing PR #${{ inputs.pr_number }}`);
53+
console.log(` Head SHA: ${pr.data.head.sha}`);
54+
console.log(` Head Repo: ${pr.data.head.repo.full_name}`);
55+
56+
service-tests:
57+
name: Service Tests
58+
runs-on: ubuntu-latest
59+
needs: setup
60+
steps:
61+
- name: Create check run
62+
id: check
63+
uses: actions/github-script@v7
64+
with:
65+
script: |
66+
const check = await github.rest.checks.create({
67+
owner: context.repo.owner,
68+
repo: context.repo.repo,
69+
name: 'Service Tests',
70+
head_sha: '${{ needs.setup.outputs.head_sha }}',
71+
status: 'in_progress',
72+
started_at: new Date().toISOString()
73+
});
74+
core.setOutput('check_id', check.data.id);
75+
76+
- name: Check out fork PR code
77+
uses: actions/checkout@v6
78+
with:
79+
repository: ${{ needs.setup.outputs.head_repo }}
80+
ref: ${{ needs.setup.outputs.head_sha }}
81+
82+
- name: Run service tests
83+
uses: ./.github/actions/run-service-tests
84+
with:
85+
python-version: ${{ env.PYTHON_VERSION }}
86+
uv-version: ${{ env.UV_VERSION }}
87+
env:
88+
HF_HOME: ${{ github.workspace }}/hf_cache
89+
HF_TOKEN: ${{ secrets.HF_TOKEN }}
90+
GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
91+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
92+
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
93+
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
94+
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
95+
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
96+
VOYAGE_API_KEY: ${{ secrets.VOYAGE_API_KEY }}
97+
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
98+
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
99+
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
100+
OPENAI_API_VERSION: ${{ secrets.OPENAI_API_VERSION }}
101+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
102+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
103+
LANGCACHE_WITH_ATTRIBUTES_API_KEY: ${{ secrets.LANGCACHE_WITH_ATTRIBUTES_API_KEY }}
104+
LANGCACHE_WITH_ATTRIBUTES_CACHE_ID: ${{ secrets.LANGCACHE_WITH_ATTRIBUTES_CACHE_ID }}
105+
LANGCACHE_WITH_ATTRIBUTES_URL: ${{ secrets.LANGCACHE_WITH_ATTRIBUTES_URL }}
106+
LANGCACHE_NO_ATTRIBUTES_API_KEY: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_API_KEY }}
107+
LANGCACHE_NO_ATTRIBUTES_CACHE_ID: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_CACHE_ID }}
108+
LANGCACHE_NO_ATTRIBUTES_URL: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_URL }}
109+
110+
- name: Update check run (success)
111+
if: success()
112+
uses: actions/github-script@v7
113+
with:
114+
script: |
115+
await github.rest.checks.update({
116+
owner: context.repo.owner,
117+
repo: context.repo.repo,
118+
check_run_id: ${{ steps.check.outputs.check_id }},
119+
status: 'completed',
120+
conclusion: 'success',
121+
completed_at: new Date().toISOString()
122+
});
123+
124+
- name: Update check run (failure)
125+
if: failure()
126+
uses: actions/github-script@v7
127+
with:
128+
script: |
129+
await github.rest.checks.update({
130+
owner: context.repo.owner,
131+
repo: context.repo.repo,
132+
check_run_id: ${{ steps.check.outputs.check_id }},
133+
status: 'completed',
134+
conclusion: 'failure',
135+
completed_at: new Date().toISOString()
136+
});

‎.github/workflows/test.yml‎

Lines changed: 30 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -20,50 +20,20 @@ jobs:
2020
service-tests:
2121
name: Service Tests
2222
runs-on: ubuntu-latest
23-
env:
24-
HF_HOME: ${{ github.workspace }}/hf_cache
23+
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
2524
steps:
2625
- name: Check out repository
27-
uses: actions/checkout@v4
28-
29-
- name: Cache HuggingFace Models
30-
uses: actions/cache@v4
31-
with:
32-
path: hf_cache
33-
key: ${{ runner.os }}-hf-cache
34-
35-
- name: Set HuggingFace token
36-
run: |
37-
mkdir -p ~/.huggingface
38-
echo '{"token":"${{ secrets.HF_TOKEN }}"}' > ~/.huggingface/token
26+
uses: actions/checkout@v6
3927

40-
- name: Install Python
41-
uses: actions/setup-python@v5
28+
- name: Run service tests
29+
uses: ./.github/actions/run-service-tests
4230
with:
4331
python-version: ${{ env.PYTHON_VERSION }}
44-
45-
- name: Install uv
46-
uses: astral-sh/setup-uv@v6
47-
with:
48-
version: ${{ env.UV_VERSION }}
49-
enable-cache: true
50-
python-version: ${{ env.PYTHON_VERSION }} # sets UV_PYTHON
51-
cache-dependency-glob: |
52-
pyproject.toml
53-
uv.lock
54-
55-
- name: Install dependencies
56-
run: |
57-
uv sync --all-extras
58-
59-
- name: Authenticate to Google Cloud
60-
uses: google-github-actions/auth@v1
61-
with:
62-
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}
63-
64-
- name: Run full test suite and prime the HF cache
32+
uv-version: ${{ env.UV_VERSION }}
6533
env:
34+
HF_HOME: ${{ github.workspace }}/hf_cache
6635
HF_TOKEN: ${{ secrets.HF_TOKEN }}
36+
GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
6737
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
6838
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
6939
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
@@ -82,25 +52,22 @@ jobs:
8252
LANGCACHE_NO_ATTRIBUTES_API_KEY: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_API_KEY }}
8353
LANGCACHE_NO_ATTRIBUTES_CACHE_ID: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_CACHE_ID }}
8454
LANGCACHE_NO_ATTRIBUTES_URL: ${{ secrets.LANGCACHE_NO_ATTRIBUTES_URL }}
85-
run: |
86-
make test-all
8755

8856
test:
89-
name: Python ${{ matrix.python-version }} - redis-py ${{ matrix.redis-py-version }} [redis ${{ matrix.redis-version }}]
57+
name: Python ${{ matrix.python-version }} - redis-py ${{ matrix.redis-py-version }} [${{ matrix.redis-image }}]
9058
runs-on: ubuntu-latest
9159
needs: service-tests
9260
env:
9361
HF_HOME: ${{ github.workspace }}/hf_cache
9462
strategy:
9563
fail-fast: false
9664
matrix:
97-
# 3.11 tests are run in the service-tests job
98-
python-version: ["3.9", "3.10", "3.12", "3.13"]
65+
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
9966
redis-py-version: ["5.x", "6.x", "7.x"]
100-
redis-version: ["6.2.6-v9", "latest", "8.4.0"]
67+
redis-image: ["redis/redis-stack-server:latest", "redis:latest"]
10168
steps:
10269
- name: Check out repository
103-
uses: actions/checkout@v4
70+
uses: actions/checkout@v6
10471

10572
- name: Cache HuggingFace Models
10673
uses: actions/cache@v4
@@ -138,16 +105,7 @@ jobs:
138105
139106
- name: Set Redis image name
140107
run: |
141-
if [[ "${{ matrix.redis-version }}" == "8.4.0" ]]; then
142-
echo "REDIS_IMAGE=redis:${{ matrix.redis-version }}" >> $GITHUB_ENV
143-
else
144-
echo "REDIS_IMAGE=redis/redis-stack-server:${{ matrix.redis-version }}" >> $GITHUB_ENV
145-
fi
146-
147-
- name: Authenticate to Google Cloud
148-
uses: google-github-actions/auth@v1
149-
with:
150-
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}
108+
echo "REDIS_IMAGE=${{ matrix.redis-image }}" >> $GITHUB_ENV
151109
152110
- name: Run tests
153111
env:
@@ -156,8 +114,24 @@ jobs:
156114
run: |
157115
make test
158116
117+
- name: Authenticate to Google Cloud
118+
uses: google-github-actions/auth@v1
119+
if: (
120+
matrix.redis-py-version == '7.x' &&
121+
matrix.redis-image == 'redis/redis-stack-server:latest' &&
122+
matrix.python-version == '3.11' &&
123+
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
124+
)
125+
with:
126+
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}
127+
159128
- name: Run notebooks
160-
if: matrix.redis-py-version == '6.x' && matrix.redis-version == 'latest'
129+
if: (
130+
matrix.redis-py-version == '7.x' &&
131+
matrix.redis-image == 'redis/redis-stack-server:latest' &&
132+
matrix.python-version == '3.11' &&
133+
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
134+
)
161135
env:
162136
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
163137
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
@@ -180,7 +154,7 @@ jobs:
180154
runs-on: ubuntu-latest
181155
steps:
182156
- name: Check out repository
183-
uses: actions/checkout@v3
157+
uses: actions/checkout@v6
184158

185159
- name: Install Python
186160
uses: actions/setup-python@v5

0 commit comments

Comments
 (0)