Skip to content

Commit 034eec9

Browse files
lukaszcllukaszcl
authored
Merge branch 'main' into refactor-flakeguard-report
2 parents 75f07e4 + 1c9eb82 commit 034eec9

File tree

11 files changed

+1047
-47
lines changed

11 files changed

+1047
-47
lines changed

‎.github/workflows/generate-go-docs.yaml‎

Lines changed: 150 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -9,70 +9,177 @@ on:
99

1010
jobs:
1111
generate_docs_new_pr:
12-
if: ${{ contains(github.event.issue.labels.*.name, 'generate_go_docs') }}
12+
if: ${{ contains(github.event.pull_request.labels.*.name, 'generate_go_docs') }}
1313
runs-on: ubuntu-latest
1414
permissions:
15-
contents: read
16-
pull-requests: write
17-
repository-projects: read
15+
id-token: write
16+
contents: read
1817

1918
steps:
19+
- name: Setup GitHub Token for reading Generate Go Doc Repo
20+
id: setup-github-token-read
21+
uses: smartcontractkit/.github/actions/setup-github-token@9e7cc0779934cae4a9028b8588c9adb64d8ce68c # [email protected]
22+
with:
23+
aws-role-arn: ${{ secrets.AWS_ROLE_ARN_READ_GENERATE_GO_DOC_REPO }}
24+
aws-lambda-url: ${{ secrets.GATI_LAMBDA_TT_URL }}
25+
aws-region: ${{ secrets.AWS_REGION }}
26+
27+
- name: Configure git for private repository and install go tools
28+
env:
29+
GOPRIVATE: github.com/smartcontractkit/generate-go-function-docs
30+
run: |
31+
git config --global url."https://x-access-token:${{ steps.setup-github-token-read.outputs.access-token }}@github.com/".insteadOf "https://github.com/"
32+
go install github.com/smartcontractkit/[email protected]
33+
go install github.com/jmank88/[email protected]
34+
go install golang.org/x/tools/gopls@latest
35+
2036
- name: Checkout current branch
2137
uses: actions/checkout@v3
2238
with:
2339
fetch-depth: 0
2440

25-
- name: Setup Git
26-
run: |
27-
git config user.name "github-actions[bot]"
28-
git config user.email "github-actions[bot]@users.noreply.github.com"
29-
30-
- name: Get PR details
31-
id: pr-details
32-
run: |
33-
echo "pr_number=$(jq -r .pull_request.number <<< "$GITHUB_EVENT_PATH")" >> $GITHUB_OUTPUT
34-
echo "pr_branch=$(jq -r .pull_request.head.ref <<< "$GITHUB_EVENT_PATH")" >> $GITHUB_OUTPUT
35-
echo "base_commit=${{ github.event.pull_request.base.sha }}" >> $GITHUB_OUTPUT
36-
echo "head_commit=${{ github.event.pull_request.head.sha }}" >> $GITHUB_OUTPUT
41+
- name: Detect changes related to current PR
42+
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
43+
id: changes
44+
with:
45+
filters: |
46+
seth:
47+
- 'seth/**/*.go'
48+
wasp:
49+
- 'wasp/**/*.go'
50+
havoc:
51+
- 'havoc/**/*.go'
52+
lib:
53+
- 'lib/**/*.go'
54+
k8s-test-runner:
55+
- 'k8s-test-runner/**/*.go'
56+
framework:
57+
- 'framework/**/*.go'
58+
# later add tools here or, if possible, make this filter dynamic so that it's created based on the go modules in the repository
3759
38-
- name: Install go doc generator
60+
- name: Find all go modules in the repository and filter the ones that changed
3961
shell: bash
62+
id: go-modules
63+
env:
64+
FILTERS: ${{ steps.changes.outputs.changes }}
4065
run: |
41-
go install github.com/smartcontractkit/generate-go-function-docs@latest
66+
PATH=$PATH:$(go env GOPATH)/bin
67+
export PATH
4268
43-
- name: Install gopls
44-
shell: bash
45-
run: |
46-
go install golang.org/x/tools/gopls@latest
69+
# Find all go projects
70+
gomods_output=$(gomods 2>&1)
4771
48-
- name: Create a new branch
49-
run: |
50-
git checkout -b ${{ steps.pr-details.outputs.pr_branch }}-docs
72+
# Extract the parent directories of go.mod files
73+
parent_folders=$(echo "$gomods_output" | grep 'go\.mod$' | sed 's/\/go\.mod//' | sed 's/^[ \t]*//;s/[ \t]*$//' | xargs -n 1)
74+
75+
# Convert parent directories into a JSON matrix
76+
echo "$parent_folders" | jq -R -s 'split("\n") | map(select(length > 0)) | map({folder: .})' > all_folders.json
5177
52-
- name: Generate go docs
78+
# Filter the directories that did not changeß
79+
jq --argjson filters "$FILTERS" 'map(select(.folder as $folder | $filters | index($folder)))' all_folders.json > filtered_folders.json
80+
81+
echo "Filtered folder List JSON"
82+
cat filtered_folders.json
83+
84+
rm all_folders.json
85+
86+
- name: Generate go docs for changed projects
5387
shell: bash
5488
env:
5589
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
5690
run: |
57-
generate-go-function-docs diff -b ${{ steps.pr-details.outputs.base_commit }} -c ${{ steps.pr-details.outputs.head_commit }} -g fake_slash
91+
# Add go binary to PATH
92+
PATH=$PATH:$(go env GOPATH)/bin
93+
export PATH
94+
cat filtered_folders.json | jq -c '.[]' | while read -r item; do
95+
folder=$(echo "$item" | jq -r '.folder')
96+
echo "Processing folder: $folder"
97+
generate-go-function-docs diff -b ${{ github.event.pull_request.base.sha }} -c ${{ github.event.pull_request.head.sha }} --saveCosts --generator chatgpt --generatorSubType ${{ vars.GO_DOC_GEN_CHATGPT_MODEL }} --folder "$folder"
98+
done
99+
rm filtered_folders.json
58100
59-
- name: Make sure code still compiles
60-
run: |
61-
go mod tidy
62-
go mod vendor
63-
go build ./...
101+
- name: Upload costs as artifact
102+
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
103+
with:
104+
name: generation-costs
105+
path: ./costs
64106

65-
- name: Commit changes
66-
run: |
67-
git add .
68-
git commit -m "Add automatically generated go documentation"
69-
git push origin ${{ steps.pr-details.outputs.pr_branch }}-docs
107+
- name: Remove costs before committing
108+
shell: bash
109+
run: rm -rf costs
110+
111+
- name: Setup GitHub Token for creating a new PR
112+
id: setup-github-token-write
113+
uses: smartcontractkit/.github/actions/setup-github-token@9e7cc0779934cae4a9028b8588c9adb64d8ce68c # [email protected]
114+
with:
115+
aws-role-arn: ${{ secrets.AWS_ROLE_ARN_CREATE_PR }}
116+
aws-lambda-url: ${{ secrets.GATI_LAMBDA_TT_URL }}
117+
aws-region: ${{ secrets.AWS_REGION }}
70118

71119
- name: Create a new PR targeting current PR
72-
uses: peter-evans/create-pull-request@v5
120+
uses: peter-evans/create-pull-request@5e914681df9dc83aa4e4905692ca88beb2f9e91f # v7.0.5
121+
id: create-pr
122+
with:
123+
token: ${{ steps.setup-github-token-write.outputs.access-token }}
124+
branch: ${{ github.head_ref }}-docs
125+
base: ${{ github.head_ref }}
126+
title: "Go docs for PR#${{ github.event.pull_request.number }}"
127+
body: "This PR contains automatically generated go documentation for the PR#${{ github.event.pull_request.number }}. Please review the changes."
128+
commit-message: "[Bot] Add automatically generated go documentation"
129+
130+
- name: Find comment with PR link
131+
uses: peter-evans/find-comment@3eae4d37986fb5a8592848f6a574fdf654e61f9e # v3.1.0
132+
id: fc
133+
with:
134+
issue-number: ${{ github.event.pull_request.number }}
135+
body-includes: Go doc generation
136+
137+
- name: Create comment in the original PR
138+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
139+
if: steps.fc.outputs.comment-id == 0
140+
with:
141+
token: ${{ steps.setup-github-token-write.outputs.access-token }}
142+
issue-number: ${{ github.event.pull_request.number }}
143+
body: |
144+
## Go doc generation
145+
Hey @${{ github.actor }}, you can check generated Go function documentation [here](${{ steps.create-pr.outputs.pull-request-url }}). Please review them and merge to this PR once you're satisfied with them.
146+
147+
- name: Send Slack notification
148+
uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
149+
if: failure()
150+
id: slack
151+
env:
152+
SLACK_BOT_TOKEN: ${{ secrets.QA_SLACK_API_KEY }}
73153
with:
74-
token: ${{ secrets.GITHUB_TOKEN }}
75-
base: ${{ env.pr_branch }}
76-
branch: ${{ steps.pr-details.outputs.pr_branch }}-docs
77-
title: "Go docs for PR#${{ env.pr_number }}"
78-
body: "This PR contains automatically generated go documentation for the PR#${{ env.pr_number }}. Please review the changes."
154+
channel-id: 'C049X3353K2'
155+
payload: |
156+
{
157+
"attachments": [
158+
{
159+
"color": "#C62828",
160+
"blocks": [
161+
{
162+
"type": "section",
163+
"text": {
164+
"type": "mrkdwn",
165+
"text": "Go doc generation - Failed :x:"
166+
}
167+
},
168+
{
169+
"type": "section",
170+
"text": {
171+
"type": "mrkdwn",
172+
"text": "<@U060CGGPY8H>, please have a look."
173+
}
174+
},
175+
{
176+
"type": "section",
177+
"text": {
178+
"type": "mrkdwn",
179+
"text": "<${{ github.server_url }}/${{ github.repository }}/pull/${{ github.event.pull_request.number }}|PR#${{ github.event.pull_request.number }}> | <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run Details>"
180+
}
181+
}
182+
]
183+
}
184+
]
185+
}

‎book/src/framework/observability/metrics.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,18 @@
22

33
We use Prometheus to collect metrics of Chainlink nodes and other services.
44

5+
Check [Prometheus UI](http://localhost:9090/query).
6+
57
Check [Grafana](http://localhost:3000/explore?panes=%7B%22gGs%22:%7B%22datasource%22:%22PBFA97CFB590B2093%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bjob%3D%5C%22ctf%5C%22%7D%22,%22range%22:true,%22instant%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22PBFA97CFB590B2093%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22__auto%22%7D%5D,%22range%22:%7B%22from%22:%22now-5m%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1) example query.
68

79
Queries:
810
- All metrics of all containers
911
```json
1012
{job="ctf"}
11-
```
13+
```
14+
15+
## Docker Resources
16+
17+
We are using [cadvisor](https://github.com/google/cadvisor) to monitor resources of test [containers](http://localhost:3000/d/pMEd7m0Mz/cadvisor-exporter?orgId=1).
18+
19+
Cadvisor UI can be found [here](http://localhost:8085/containers/)

‎framework/.changeset/v0.3.5.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
- Add cadvisor support to "ctf obs up"
2+
- Add cadvisor dashboard

‎framework/.changeset/v0.3.6.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
- Add cadvisor support to obs stack
2+
- Add cadvisor dashboard

‎framework/cmd/observability/compose/conf/prometheus.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,8 @@ scrape_configs:
1111
- source_labels: [ __meta_docker_port_private ]
1212
regex: '6688'
1313
action: keep
14+
- job_name: cadvisor
15+
scrape_interval: 10s
16+
static_configs:
17+
- targets:
18+
- cadvisor:8080

0 commit comments

Comments
 (0)