Skip to content

Commit 4f9292a

Browse files
scottschreckengaustscottschreckengaust
authored
docs: add developer's guide for running CodeBuild locally (#94)
* docs: add developer's guide for running CodeBuild locally * feat: updated for running codebuild locally * Revise prerequisites and script options in guide Updated prerequisites and options for the codebuild_build.sh script in the developer guide. * Fix formatting of script options in developer guide * Update developers guide with buildspec-override note Added note about editing the buildspec-override value in codebuild.yml. * Update DEVELOPERS_GUIDE.md * fix: update documentation and run codebuild Signed-off-by: Scott Schreckengaust <scottschreckengaust@users.noreply.github.com> * fix: running codebuild Signed-off-by: Scott Schreckengaust <scottschreckengaust@users.noreply.github.com> * Update codebuild command for architecture handling * feat: fixing for act locally Signed-off-by: Scott Schreckengaust <345885+scottschreckengaust@users.noreply.github.com> * feat(doc): the act use Signed-off-by: Scott Schreckengaust <345885+scottschreckengaust@users.noreply.github.com> --------- Signed-off-by: Scott Schreckengaust <scottschreckengaust@users.noreply.github.com> Signed-off-by: Scott Schreckengaust <345885+scottschreckengaust@users.noreply.github.com> Co-authored-by: Scott Schreckengaust <345885+scottschreckengaust@users.noreply.github.com>
1 parent cdb5817 commit 4f9292a

File tree

3 files changed

+167
-32
lines changed

3 files changed

+167
-32
lines changed

‎.github/workflows/codebuild.yml‎

Lines changed: 85 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -62,9 +62,10 @@ jobs:
6262
with:
6363
role-to-assume: ${{ secrets.AWS_CODEBUILD_ROLE_ARN }}
6464
aws-region: ${{ vars.AWS_REGION || 'us-east-1' }}
65-
role-duration-seconds: 7200
65+
role-duration-seconds: ${{ vars.ROLE_DURATION_SECONDS || 7200 }}
6666
role-session-name: GitHubActions${{ github.run_id }}
6767
mask-aws-account-id: true
68+
retry-max-attempts: 0
6869

6970
- name: Run CodeBuild
7071
if: steps.cache-check.outputs.cache-hit != 'true'
@@ -77,61 +78,90 @@ jobs:
7778
version: 0.2
7879
env:
7980
variables:
80-
TEST_ONE: "1"
81+
GH_TOKEN: ${{ github.token }}
8182
phases:
8283
install:
8384
commands:
84-
- echo "install ${TEST_ONE}" | tee --append ./codebuild.out
85-
- dnf install -y lshw || echo "dnf install failed"
85+
- dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo || echo "dnf config-manager"
86+
- dnf install -y 'dnf-command(config-manager)' gh || echo "dnf install failed"
87+
- curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH=$HOME/.local/bin:$PATH || "echo uv failed"
8688
pre_build:
8789
commands:
88-
- echo "pre_build ${TEST_ONE}" | tee --append ./codebuild.out
89-
- echo "=== OS ==="
90-
- cat /etc/os-release
91-
- echo "=== Kernel ==="
92-
- uname -a
93-
- echo "=== CPU ==="
94-
- lscpu
95-
- echo "=== Memory ==="
96-
- free -h
97-
- echo "=== Kisk ==="
98-
- df -h
99-
- echo "=== Block Devices ==="
100-
- lsblk
101-
- echo "=== Hardward Summary ==="
102-
- lshw -short || echo "lshw failed"
90+
- echo "pre_build"
91+
- mkdir -p .codebuild
92+
- touch .codebuild/codebuild.out
93+
- git config --global --add safe.directory "/codebuild/output/srcDownload/src" # for running AWS CodeBuild locally
10394
build:
10495
commands:
105-
- echo "build ${TEST_ONE}" | tee --append ./codebuild.out
106-
- ls -alR
96+
- DEFAULT_BRANCH=$(gh repo view --json defaultBranchRef --jq '.defaultBranchRef.name')
97+
- CURRENT_BRANCH=$(git symbolic-ref --short HEAD 2>/dev/null || echo "")
98+
- CURRENT_TAG=$(git describe --tags --exact-match 2>/dev/null || echo "")
99+
- IS_RELEASE=$([[ -n "$CURRENT_TAG" ]] && echo "true" || echo "false")
100+
- IS_PRE_RELEASE=$([[ "$CURRENT_BRANCH" == "$DEFAULT_BRANCH" ]] && echo "true" || echo "false")
101+
- IS_PRE_MERGE=$([[ -z "$CURRENT_TAG" && "$CURRENT_BRANCH" != "$DEFAULT_BRANCH" ]] && echo "true" || echo "false")
102+
- if [[ "$IS_RELEASE" == "true" ]]; then echo "This is a release"; fi;
103+
- if [[ "$IS_PRE_RELEASE" == "true" ]]; then echo "This is a pre-release"; fi;
104+
- if [[ "$IS_PRE_MERGE" == "true" ]]; then echo "This is a pre-merge"; fi;
105+
- mkdir -p .codebuild/evaluation
106+
- mkdir -p .codebuild/trend
107+
- mkdir -p .codebuild/missing
108+
- touch .codebuild/evaluation/evaluation_report.html
109+
- touch .codebuild/evaluation/metrics.yml
110+
- touch .codebuild/trend/trend_report.html
107111
post_build:
108112
commands:
109-
- echo "post_build ${TEST_ONE}" | tee --append ./codebuild.out
110113
- echo "Build completed with status $CODEBUILD_BUILD_SUCCEEDING"
111-
- cat ./codebuild.out
114+
- cat ./.codebuild/codebuild.out
112115
artifacts:
113116
files:
114-
- '**/codebuild.out'
115-
discard-paths: yes
117+
- '**/*'
118+
discard-paths: no
119+
base-directory: .codebuild
120+
secondary-artifacts:
121+
evaluation:
122+
files:
123+
- '**/*'
124+
name: evaluation
125+
discard-paths: yes
126+
base-directory: .codebuild/evaluation
127+
trend:
128+
files:
129+
- '**/*'
130+
name: trend
131+
discard-paths: yes
132+
base-directory: .codebuild/trend
116133
117134
- name: Build ID
118135
if: always() && steps.cache-check.outputs.cache-hit != 'true'
119136
run: echo "CodeBuild Build ID ${{ steps.codebuild.outputs.aws-build-id }}"
120137

121-
- name: Download CodeBuild artifact
138+
- name: Download CodeBuild artifacts
122139
if: steps.cache-check.outputs.cache-hit != 'true'
123140
run: |
124-
ARTIFACT_LOCATION=$(aws codebuild batch-get-builds \
141+
DOWNLOADS="${ACT_CODEBUILD_DIR:-${GITHUB_WORKSPACE}/.codebuild/downloads}"
142+
mkdir -p "$DOWNLOADS"
143+
PRIMARY_ARTIFACT_LOCATION=$(aws codebuild batch-get-builds \
125144
--ids "${{ steps.codebuild.outputs.aws-build-id }}" \
126145
--query 'builds[0].artifacts.location' \
127146
--output text)
128-
aws s3 cp "s3://${ARTIFACT_LOCATION#arn:aws:s3:::}" ./${{ env.CODEBUILD_PROJECT_NAME }}.zip
147+
aws s3 cp "s3://${PRIMARY_ARTIFACT_LOCATION#arn:aws:s3:::}" "$DOWNLOADS/${{ env.CODEBUILD_PROJECT_NAME }}.zip"
148+
SECONDARY_ARTIFACT_LOCATIONS=$(aws codebuild batch-get-builds \
149+
--ids "${{ steps.codebuild.outputs.aws-build-id }}" \
150+
--query 'builds[0].secondaryArtifacts[*].[artifactIdentifier, location]' \
151+
--output json)
152+
echo "$SECONDARY_ARTIFACT_LOCATIONS" | jq -r '.[] | @tsv' | while IFS=$'\t' read -r NAME LOCATION; do
153+
echo "Downloading secondary artifact: $NAME"
154+
aws s3 cp "s3://${LOCATION#arn:aws:s3:::}" "$DOWNLOADS/${NAME}.zip"
155+
done
129156
130157
- name: List CodeBuild artifacts
131158
if: steps.cache-check.outputs.cache-hit != 'true'
132159
run: |
133-
ls -alR
134-
unzip -l ${{ env.CODEBUILD_PROJECT_NAME }}.zip
160+
DOWNLOADS="${ACT_CODEBUILD_DIR:-${GITHUB_WORKSPACE}/.codebuild/downloads}"
161+
ls -alR "$DOWNLOADS"
162+
unzip -l "$DOWNLOADS/${{ env.CODEBUILD_PROJECT_NAME }}.zip"
163+
unzip -l "$DOWNLOADS/evaluation.zip"
164+
unzip -l "$DOWNLOADS/trend.zip"
135165
136166
- name: Clean old report caches
137167
if: steps.cache-check.outputs.cache-hit != 'true'
@@ -147,5 +177,29 @@ jobs:
147177
if: steps.cache-check.outputs.cache-hit != 'true'
148178
uses: actions/cache/save@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
149179
with:
150-
path: ${{ env.CODEBUILD_PROJECT_NAME }}.zip
180+
path: ${{ github.workspace }}/.codebuild/downloads/${{ env.CODEBUILD_PROJECT_NAME }}.zip
151181
key: ${{ env.CODEBUILD_PROJECT_NAME }}-${{ github.ref_name }}-${{ github.sha }}
182+
183+
- name: Upload CodeBuild primary artifact
184+
if: ${{ !env.ACT }} # incompatability with v6 of upload-artifact and act
185+
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
186+
with:
187+
name: ${{ env.CODEBUILD_PROJECT_NAME }}.zip
188+
path: ${{ github.workspace }}/.codebuild/downloads/${{ env.CODEBUILD_PROJECT_NAME }}.zip
189+
if-no-files-found: error
190+
191+
- name: Upload CodeBuild secondary artifact - evaluation
192+
if: ${{ !env.ACT }} # incompatability with v6 of upload-artifact and act
193+
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
194+
with:
195+
name: evaluation.zip
196+
path: ${{ github.workspace }}/.codebuild/downloads/evaluation.zip
197+
if-no-files-found: error
198+
199+
- name: Upload CodeBuild secondary artifact - trend
200+
if: ${{ !env.ACT }} # incompatability with v6 of upload-artifact and act
201+
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
202+
with:
203+
name: trend.zip
204+
path: ${{ github.workspace }}/.codebuild/downloads/trend.zip
205+
if-no-files-found: error

‎.gitignore‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
11
.DS_Store
22
.amazonq/**
3-
.claude/**
3+
.claude/**
4+
.codebuild/**
5+
.vscode/**
6+
.env
7+
buildspec.yml
8+
codebuild_build.sh

‎docs/DEVELOPERS_GUIDE.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# Developer's Guide
2+
3+
## Running CodeBuild Locally
4+
5+
You can run AWS CodeBuild builds locally using the [CodeBuild local agent](https://docs.aws.amazon.com/codebuild/latest/userguide/use-codebuild-agent.html). This is useful for testing buildspec changes without pushing to the remote.
6+
7+
### Prerequisites
8+
9+
- Docker installed and running
10+
- The `codebuild_build.sh` script:
11+
12+
### Basic Usage
13+
14+
1. Setup
15+
- Download the local CodeBuild script and make it executable.
16+
- Send the `GH_TOKEN` environmental GitHub Personal Access Token (PAT) into a `./.env` file
17+
18+
```bash
19+
if [ ! -f codebuild_build.sh ]; then
20+
curl -O https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/local_builds/codebuild_build.sh && chmod +x codebuild_build.sh;
21+
fi;
22+
echo "GH_TOKEN=${GH_TOKEN:-ghp_notset}" > "./.env";
23+
```
24+
25+
2. Iterate
26+
27+
- _Optionally edit the `buildspec-override` value in the `.github/workflows/codebuild.yml` GitHub workflow_
28+
- Update `./buildspec.yml` based on the workflow contents to a local file
29+
- Run AWS CodeBuild build locally with images based on the machine architecture
30+
31+
```bash
32+
cat .github/workflows/codebuild.yml \
33+
| uvx yq -r '.jobs.build.steps[] | select(.id == "codebuild") | .with["buildspec-override"]' \
34+
> buildspec.yml
35+
./codebuild_build.sh \
36+
-i "public.ecr.aws/codebuild/amazonlinux-$([ "$(arch)" = "arm64" -o "$(arch)" = "aarch64" ] && echo "aarch64" || echo "x86_64")-standard:$([ "$(arch)" = "arm64" -o "$(arch)" = "aarch64" ] && echo "3.0" || echo "5.0")" \
37+
-a "./.codebuild/artifacts/" \
38+
-l "public.ecr.aws/codebuild/local-builds:$([ "$(arch)" = "arm64" -o "$(arch)" = "aarch64" ] && echo "aarch64" || echo "latest")" \
39+
-c \
40+
-e "./.env"
41+
```
42+
43+
### All Script Options
44+
45+
| Flag | Required | Description |
46+
|--------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
47+
| `-i IMAGE` | Yes | Customer build container image (e.g. `aws/codebuild/standard:5.0`) |
48+
| `-a DIR` | Yes | Artifact output directory |
49+
| `-b FILE` | No | Buildspec override file. Defaults to `buildspec.yml` in the source directory |
50+
| `-s DIR` | No | Source directory. First `-s` is the primary source; additional `-s` flags use `<sourceIdentifier>:<sourceLocation>` format for secondary sources. Defaults to the current working directory |
51+
| `-l IMAGE` | No | Override the default local agent image |
52+
| `-r DIR` | No | Report output directory |
53+
| `-c` | No | Use AWS configuration and credentials from your local host (`~/.aws` and `AWS_*` environment variables) |
54+
| `-p PROFILE` | No | AWS CLI profile to use (requires `-c`) |
55+
| `-e FILE` | No | File containing environment variables (`VAR=VAL` format, one per line) |
56+
| `-m` | No | Mount the source directory into the build container directly |
57+
| `-d` | No | Run the build container in Docker privileged mode |
58+
59+
60+
## Running GitHub Actions locally
61+
62+
_NOTE: This uses the [`act`](https://github.com/nektos/act) tool and assumes access to a valid AWS CodeBuild project `codebuild-project` in "us-east-1"_
63+
64+
```shell
65+
act --platform ubuntu-latest=-self-hosted \
66+
--job build \
67+
--workflows .github/workflows/codebuild.yml \
68+
--env-file .env \
69+
--var CODEBUILD_PROJECT_NAME=codebuild-project \
70+
--var AWS_REGION=us-east-1 \
71+
--var ROLE_DURATION_SECONDS=7200 \
72+
--artifact-server-path=$PWD/.codebuild/artifacts \
73+
--cache-server-path=$PWD/.codebuild/artifacts \
74+
--env ACT_CODEBUILD_DIR=$PWD/.codebuild/downloads \
75+
--bind
76+
```

0 commit comments

Comments
 (0)