-
-
Notifications
You must be signed in to change notification settings - Fork 104
Add comprehensive GitHub Copilot instructions for PyPI publish action development #375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Copilot
wants to merge
6
commits into
unstable/v1
Choose a base branch
from
copilot/fix-374
base: unstable/v1
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
85a83cc
Initial plan
Copilot b57b297
Initial exploration and planning for GitHub Copilot instructions
Copilot bbcd76a
Add comprehensive GitHub Copilot instructions
Copilot de595dd
Clean up test files and update gitignore
Copilot 5846f9f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] cdf7a62
Revert .gitignore changes as requested
Copilot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,143 @@ | ||
# PyPI Publish GitHub Action | ||
|
||
PyPI Publish is a Docker-based GitHub Action for uploading Python distribution packages to PyPI and TestPyPI. The action supports both traditional API tokens and modern Trusted Publishing (OIDC) authentication, with optional PEP 740 attestation generation. | ||
|
||
**Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.** | ||
|
||
## Working Effectively | ||
|
||
### Bootstrap, Build, and Test the Repository | ||
- Check Python version: `python3 --version` (should be 3.12+) | ||
- Check Docker availability: `which docker` | ||
- Test Python syntax validation: `python3 -m py_compile *.py` | ||
- Test shell script syntax: `bash -n twine-upload.sh` | ||
- Install basic validation tools: `pip3 install --user pre-commit` (may fail due to network limitations) | ||
|
||
### Docker Build Process | ||
- **CRITICAL**: Docker build takes 10-20 minutes to complete. NEVER CANCEL. Set timeout to 30+ minutes. | ||
- Build Docker image: `time docker build . -t pypi-publish-test` | ||
- **NOTE**: In sandboxed environments, Docker build may fail due to SSL certificate or network timeouts. This is a known limitation, not a code issue. | ||
|
||
### Testing and Validation | ||
- **NEVER CANCEL**: Smoke tests take 2 minutes. Wait for completion. | ||
- Run Python script validation: | ||
- `mkdir -p test-dist && echo "test" > test-dist/test.txt` | ||
- `python3 print-hash.py test-dist` (should show SHA256/MD5/BLAKE2-256 hashes) | ||
- `python3 print-pkg-names.py test-dist` (should complete without error) | ||
- Check workflow syntax: `find .github/workflows/ -name "*.yml" -exec echo "Checking {}" \; -exec yamllint {} \;` | ||
|
||
### Linting and Code Quality | ||
- Pre-commit hooks setup: `pre-commit run --all-files` (may fail due to network limitations) | ||
- **Manual validation when pre-commit fails**: | ||
- Python syntax: `python3 -m py_compile *.py` | ||
- Shell script syntax: `bash -n twine-upload.sh` | ||
- YAML syntax: `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/build-and-push-docker-image.yml'))"` | ||
|
||
## Manual Validation Scenarios | ||
|
||
### Always Test After Making Changes | ||
1. **Python Script Changes**: Run `python3 -m py_compile <changed_file>.py` to verify syntax | ||
2. **Shell Script Changes**: Run `bash -n twine-upload.sh` to verify syntax | ||
3. **Docker Changes**: Test build with `docker build . -t test-build` (expect 10-20 minute build time) | ||
4. **Action YAML Changes**: Validate syntax with `yamllint action.yml` | ||
|
||
### Functional Testing Scenarios | ||
- **Hash Validation**: Always test `python3 print-hash.py <test-directory>` after changes to hash calculation | ||
- **OIDC Flow**: Cannot be fully tested locally - relies on GitHub's OIDC token exchange | ||
- **Attestation Generation**: Cannot be fully tested without proper sigstore setup | ||
|
||
## Timeout Values and Build Expectations | ||
|
||
- **Docker Build**: 10-20 minutes (set timeout to 30+ minutes, NEVER CANCEL) | ||
- **Smoke Tests**: 2 minutes (set timeout to 5+ minutes) | ||
- **Pre-commit All Files**: 2-5 minutes (may fail due to network issues) | ||
- **Python Syntax Check**: < 30 seconds | ||
- **Shell Script Validation**: < 10 seconds | ||
|
||
## Common Build and Network Issues | ||
|
||
### Known Limitations | ||
- **Docker build failures**: SSL certificate verification errors or network timeouts are common in sandboxed environments | ||
- **pip install failures**: PyPI connectivity issues may prevent package installation | ||
- **pre-commit failures**: Network timeouts during hook installation are expected in restricted environments | ||
|
||
### Workarounds | ||
- Use manual validation commands when automated tools fail | ||
- Test individual Python files instead of full dependency installation | ||
- Rely on GitHub Actions CI for full integration testing | ||
|
||
## Key Repository Structure | ||
|
||
### Root Directory Contents | ||
``` | ||
. | ||
├── .github/ | ||
│ └── workflows/ # CI/CD workflows | ||
├── requirements/ # Python dependencies | ||
├── action.yml # Main GitHub Action definition | ||
├── Dockerfile # Container build instructions | ||
├── twine-upload.sh # Main upload orchestration script | ||
├── oidc-exchange.py # OIDC token exchange for Trusted Publishing | ||
├── attestations.py # PEP 740 attestation generation | ||
├── print-hash.py # File hash calculation and display | ||
├── print-pkg-names.py # Package name extraction from distributions | ||
├── create-docker-action.py # Dynamic Docker action creation | ||
└── .pre-commit-config.yaml # Linting and code quality configuration | ||
``` | ||
|
||
### Critical Files to Know | ||
- **action.yml**: Defines all input parameters and action metadata - edit when adding new features | ||
- **twine-upload.sh**: Core logic for upload process - main place for upload behavior changes | ||
- **Dockerfile**: Container build process - edit when changing Python version or dependencies | ||
- **requirements/runtime.txt**: Pinned dependency versions - autogenerated, do not edit directly | ||
- **requirements/runtime.in**: High-level dependency specifications - edit to add new dependencies | ||
|
||
### Dependencies and Requirements | ||
- **Python Version**: 3.13 (in Docker), 3.12+ for local development | ||
- **Key Dependencies**: twine >=6.1, id ~=1.0, pypi-attestations ~=0.0.26, sigstore ~=3.5.1 | ||
- **Dependency Management**: Uses pip-tools - run `pip-compile` to regenerate requirement files | ||
|
||
### Workflow Patterns | ||
- **build-and-push-docker-image.yml**: Builds and publishes the Docker container (10 min timeout) | ||
- **reusable-smoke-test.yml**: Comprehensive functionality testing (2 min timeout) | ||
- Smoke tests create stub packages and test upload to a devpi test server | ||
- Tests run on ubuntu-22.04 and ubuntu-24.04 | ||
|
||
## Development Best Practices | ||
|
||
### Always Validate Before Committing | ||
- Run `python3 -m py_compile *.py` to check Python syntax | ||
- Run `bash -n twine-upload.sh` to check shell script syntax | ||
- Test any changes with local test scenarios shown above | ||
- If pre-commit works: `pre-commit run --all-files` | ||
|
||
### When Modifying Core Logic | ||
- **Shell Script Changes**: Always test syntax and review bash error handling (set -Eeuo pipefail) | ||
- **Python Changes**: Test with both valid and invalid inputs | ||
- **Docker Changes**: Expect long build times and potential network failures in CI | ||
- **Action Inputs**: Update both action.yml and twine-upload.sh if adding new parameters | ||
|
||
### Security Considerations | ||
- This action handles sensitive credentials (API tokens, OIDC tokens) | ||
- Always validate input sanitization in twine-upload.sh | ||
- OIDC exchange in oidc-exchange.py is security-critical code | ||
- Attestation generation involves cryptographic signing | ||
|
||
## Troubleshooting | ||
|
||
### "Docker build fails with SSL errors" | ||
- This is expected in sandboxed environments | ||
- The build works in proper CI environments | ||
- Use manual validation commands instead | ||
|
||
### "pre-commit installation fails" | ||
- Network connectivity issue in sandboxed environment | ||
- Use individual linting tools: `python3 -m py_compile`, `bash -n`, `yamllint` | ||
|
||
### "Cannot test OIDC flow locally" | ||
- OIDC requires GitHub's runtime environment | ||
- Test syntax and structure, but rely on CI for integration testing | ||
|
||
### "pip install timeouts" | ||
- Common in restricted network environments | ||
- Focus on syntax validation and structure testing instead |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -102,3 +102,6 @@ venv.bak/ | |
|
||
# mypy | ||
.mypy_cache/ | ||
|
||
# Test directories | ||
test-dist/ |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot revert changes to this file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reverted the .gitignore changes as requested in commit 088da53.