-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add Docker publishing with environment variable conventions #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
d620a43
feat: add docker publishing with environment variable conventions
AhmedKorim db8bd17
feat: prevent Docker publishing from template repository
AhmedKorim 0a9e81b
Update README.md
AhmedKorim beae62c
clean up docker publishing
AhmedKorim 0c646df
Merge remote-tracking branch 'origin/ahmed/feat/docker-registry-templ…
AhmedKorim 74f0902
update def for `node.json`
AhmedKorim 9dbafe9
sync doc for `node.json`
AhmedKorim 256a87f
sync readme and remove unwanted docker args
AhmedKorim 344d8f0
chore: registration documented
AhmedKorim bba25e0
remove docker publish guide
AhmedKorim 5e0cf2c
remove docker publish guide
AhmedKorim 26333a4
remove the recommend structure for the component paths
AhmedKorim d244e46
Update docs/node-configuration.md
AhmedKorim d6fedf0
Update README.md
AhmedKorim 63fbbc8
Update README.md
AhmedKorim 1cdaeb0
Update README.md
AhmedKorim 33daec9
Update README.md
AhmedKorim 45c8518
Update README.md
AhmedKorim 434cf97
Update node.json
AhmedKorim 0ade9d9
Update docs/node-configuration.md
AhmedKorim 9551c8b
Update README.md
AhmedKorim df1887e
Update docs/node-configuration.md
AhmedKorim 10d773a
Update README.md
AhmedKorim 2361e05
remove ghcr from examples
AhmedKorim a9077cb
Remove component suggested strucutre
AhmedKorim b73a051
Remove ghcr
AhmedKorim 11d4efb
Merge remote-tracking branch 'origin/ahmed/feat/docker-registry-templ…
AhmedKorim 05f8c73
Update docs/node-configuration.md
0x3bfc d422353
Update README.md
0x3bfc a144bb6
Update docs/node-configuration.md
0x3bfc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,37 @@ | ||
| # Droq Node Template Environment Variables | ||
| # Copy this file to .env and update with your values | ||
| # Environment Variable Naming Convention: | ||
| # NODE_ - Node configuration | ||
| # NATS_ - NATS configuration | ||
| # LOG_ - Logging configuration | ||
| # DB_ - Database configuration | ||
| # HTTP_ - HTTP client configuration | ||
| # SERVICE_ - Service-specific configuration | ||
| # METRICS_ - Metrics and monitoring | ||
|
|
||
| # Node Configuration | ||
| NODE_NAME=droq-node-template | ||
| NODE_PORT=8000 | ||
| NODE_EXTERNAL_PORT=8000 | ||
| NODE_HEALTH_CHECK_ENABLED=true | ||
|
|
||
| # Logging | ||
| LOG_LEVEL=INFO | ||
|
|
||
| # NATS Configuration | ||
| NATS_URL=nats://localhost:4222 | ||
| NATS_CLIENT_NAME=${NODE_NAME:-droq-node-template} | ||
| STREAM_NAME=droq-stream | ||
|
|
||
| # HTTP Client Configuration (optional) | ||
| # BASE_URL=https://api.example.com | ||
| # Optional: NATS Authentication | ||
| # NATS_USERNAME=nats-user | ||
| # NATS_PASSWORD=nats-pass | ||
|
|
||
| # Optional: HTTP Configuration | ||
| # HTTP_BASE_URL=https://api.example.com | ||
| # HTTP_API_KEY=your-api-key | ||
|
|
||
| # Optional: Database Configuration | ||
| # DB_URL=postgresql://user:pass@localhost/dbname | ||
|
|
||
| # Add your custom environment variables below | ||
| # API_KEY=your-api-key-here | ||
| # DATABASE_URL=postgresql://user:pass@localhost/dbname | ||
| # Optional: Metrics Configuration | ||
| # METRICS_PORT=8081 | ||
| # METRICS_ENABLED=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,222 @@ | ||
| name: Docker Build and Publish | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| tags: ['v*'] | ||
| pull_request: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
| inputs: | ||
| publish: | ||
| description: 'Publish to registry' | ||
| required: false | ||
| default: 'false' | ||
| type: boolean | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
| IMAGE_NAME: ${{ github.repository }} | ||
|
|
||
| jobs: | ||
| # Check if this is a template repository or a consuming repository | ||
| check-repo-type: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| is-template: ${{ steps.check.outputs.is-template }} | ||
| should-publish: ${{ steps.check.outputs.should-publish }} | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Check repository type | ||
| id: check | ||
| run: | | ||
| # Check if this is a template repository by looking for template indicators | ||
| # Template repositories typically have specific patterns in their files | ||
| if [[ "${{ github.repository }}" == "droq-ai/dfx-base-node-template-py" ]]; then | ||
| echo "is-template=true" >> $GITHUB_OUTPUT | ||
| echo "should-publish=false" >> $GITHUB_OUTPUT | ||
| echo "::notice::This is the template repository - Docker publishing is disabled" | ||
| elif grep -q "droq-node-template" README.md 2>/dev/null && \ | ||
| grep -q "Replace src/node/main.py with your code" README.md 2>/dev/null; then | ||
| echo "is-template=true" >> $GITHUB_OUTPUT | ||
| echo "should-publish=false" >> $GITHUB_OUTPUT | ||
| echo "::notice::Template repository detected - Docker publishing is disabled. Consumers should customize before publishing." | ||
| else | ||
| echo "is-template=false" >> $GITHUB_OUTPUT | ||
|
|
||
| # Determine if should publish based on event | ||
| if [[ "${{ github.event_name }}" == "push" && "${{ github.ref }}" == "refs/heads/main" ]] || \ | ||
| [[ "${{ github.event_name }}" == "push" && "${{ github.ref }}" == refs/tags/* ]] || \ | ||
| [[ "${{ github.event.inputs.publish }}" == "true" ]]; then | ||
| echo "should-publish=true" >> $GITHUB_OUTPUT | ||
| echo "::notice::Consuming repository detected - Docker publishing is enabled" | ||
| else | ||
| echo "should-publish=false" >> $GITHUB_OUTPUT | ||
| fi | ||
| fi | ||
|
|
||
| build-and-test: | ||
| runs-on: ubuntu-latest | ||
| needs: check-repo-type | ||
| outputs: | ||
| image-digest: ${{ steps.build.outputs.digest }} | ||
| image-tag: ${{ steps.meta.outputs.tags }} | ||
| should-publish: ${{ needs.check-repo-type.outputs.should-publish }} | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Extract metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
| tags: | | ||
| type=ref,event=branch | ||
| type=ref,event=pr | ||
| type=semver,pattern={{version}} | ||
| type=semver,pattern={{major}}.{{minor}} | ||
| type=semver,pattern={{major}} | ||
| type=raw,value=latest,enable={{is_default_branch}} | ||
|
|
||
| - name: Build Docker image | ||
| id: build | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| platforms: linux/amd64,linux/arm64 | ||
| push: false | ||
| load: false | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
| outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push=false | ||
|
|
||
| publish: | ||
| runs-on: ubuntu-latest | ||
| needs: [check-repo-type, build-and-test] | ||
| if: needs.build-and-test.outputs.should-publish == 'true' | ||
| environment: production | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to Container Registry | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
| tags: | | ||
| type=ref,event=branch | ||
| type=semver,pattern={{version}} | ||
| type=semver,pattern={{major}}.{{minor}} | ||
| type=semver,pattern={{major}} | ||
| type=raw,value=latest,enable={{is_default_branch}} | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| platforms: linux/amd64,linux/arm64 | ||
| push: true | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
|
|
||
| - name: Generate SBOM | ||
| uses: anchore/sbom-action@v0 | ||
| if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') | ||
| with: | ||
| image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }} | ||
| format: spdx-json | ||
| output-file: sbom.spdx.json | ||
|
|
||
| - name: Upload SBOM | ||
| uses: actions/upload-artifact@v4 | ||
| if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') | ||
| with: | ||
| name: sbom | ||
| path: sbom.spdx.json | ||
|
|
||
| publish-private-registry: | ||
| runs-on: ubuntu-latest | ||
| needs: [check-repo-type, build-and-test] | ||
| if: needs.build-and-test.outputs.should-publish == 'true' && secrets.PRIVATE_REGISTRY_URL != '' | ||
| environment: production | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to Private Registry | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ secrets.PRIVATE_REGISTRY_URL }} | ||
| username: ${{ secrets.PRIVATE_REGISTRY_USERNAME }} | ||
| password: ${{ secrets.PRIVATE_REGISTRY_PASSWORD }} | ||
|
|
||
| - name: Extract metadata for private registry | ||
| id: meta-private | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ secrets.PRIVATE_REGISTRY_URL }}/${{ secrets.PRIVATE_REGISTRY_IMAGE_NAME || github.repository }} | ||
| tags: | | ||
| type=ref,event=branch | ||
| type=semver,pattern={{version}} | ||
| type=semver,pattern={{major}}.{{minor}} | ||
| type=semver,pattern={{major}} | ||
| type=raw,value=latest,enable={{is_default_branch}} | ||
|
|
||
| - name: Build and push to Private Registry | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| platforms: linux/amd64,linux/arm64 | ||
| push: true | ||
| tags: ${{ steps.meta-private.outputs.tags }} | ||
| labels: ${{ steps.meta-private.outputs.labels }} | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
|
|
||
| security-scan: | ||
| runs-on: ubuntu-latest | ||
| needs: [check-repo-type, build-and-test, publish] | ||
| if: needs.build-and-test.outputs.should-publish == 'true' | ||
| environment: production | ||
|
|
||
| steps: | ||
| - name: Run Trivy vulnerability scanner | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ needs.build-and-test.outputs.image-digest }} | ||
| format: 'sarif' | ||
| output: 'trivy-results.sarif' | ||
|
|
||
| - name: Upload Trivy scan results to GitHub Security tab | ||
| uses: github/codeql-action/upload-sarif@v3 | ||
| if: always() | ||
| with: | ||
| sarif_file: 'trivy-results.sarif' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,49 +1,29 @@ | ||
| # Dockerfile template for Droq nodes | ||
| # This is an agnostic template - customize as needed for your node | ||
|
|
||
| FROM python:3.11-slim | ||
|
|
||
| # Set working directory | ||
| WORKDIR /app | ||
|
|
||
| # Install system dependencies | ||
| # Uncomment and add system packages as needed: | ||
| # RUN apt-get update && apt-get install -y \ | ||
| # gcc \ | ||
| # g++ \ | ||
| # make \ | ||
| # curl \ | ||
| # && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Install uv | ||
| COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv | ||
|
|
||
| # Copy dependency files | ||
| COPY pyproject.toml uv.lock* ./ | ||
|
|
||
| # Install dependencies using uv | ||
| # Install dependencies directly (not as editable package) | ||
| RUN uv pip install --system nats-py aiohttp || \ | ||
| (uv pip compile pyproject.toml -o requirements.txt && \ | ||
| uv pip install --system -r requirements.txt) | ||
|
|
||
| # Copy source code | ||
| COPY src/ ./src/ | ||
|
|
||
| # Create non-root user for security | ||
| RUN useradd -m -u 1000 nodeuser && chown -R nodeuser:nodeuser /app | ||
| USER nodeuser | ||
|
|
||
| # Set environment variables | ||
| ENV PYTHONPATH=/app | ||
| ENV PYTHONUNBUFFERED=1 | ||
| ENV NODE_PORT=8000 | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. don't hardcode this port in the dockerfile, the compose should handle this. |
||
|
|
||
| EXPOSE ${NODE_PORT} | ||
AhmedKorim marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # Optional: Health check | ||
| # Uncomment and customize as needed: | ||
| # HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ | ||
| # CMD python -c "import sys; sys.exit(0)" | ||
| ARG NODE_HEALTH_CHECK_ENABLED=false | ||
AhmedKorim marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ | ||
| CMD python -c "import sys; sys.exit(0)" || exit 1 | ||
|
|
||
| # Run the node | ||
| # Update this command to match your entry point | ||
| CMD ["uv", "run", "python", "-m", "node.main"] | ||
| CMD ["sh", "-c", "exec uv run python -m node.main --port=${NODE_PORT:-8000}"] | ||
AhmedKorim marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.