Skip to content

Update logs grepping scripts and add a script to find the first API log entry in AWS logs #4880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lukaszgryglicki merged 3 commits into from
Nov 27, 2025
+29 −2
Merged

Update logs grepping scripts and add a script to find the first API log entry in AWS logs #4880

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
808ee41
Update logs grepping scripts and add a script to find the first API l…
lukaszgryglicki Nov 27, 2025
50996d1
Update utils/find_1st_log_entry.sh
lukaszgryglicki Nov 27, 2025
a4de28d
Update utils/find_1st_log_entry.sh
lukaszgryglicki Nov 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions utils/calculate_api_stats.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ then
exit 1
fi
export DTFROM="${1}"
REGION=us-east-1 NO_ECHO=1 DTTO='1 second ago' OUT="api-logs-${STAGE}-1.json" ./utils/search_aws_logs.sh 'LG:api-request-path'
REGION=us-east-2 NO_ECHO=1 DTTO='1 second ago' OUT="api-logs-${STAGE}-2.json" ./utils/search_aws_logs.sh 'LG:api-request-path'
INCL_LOGS='githubactivity,githubinstall,apiv1,apiv2,api-v3-lambda' REGION=us-east-1 NO_ECHO=1 DTTO='1 second ago' OUT="api-logs-${STAGE}-1.json" ./utils/search_aws_logs.sh 'LG:api-request-path'
INCL_LOGS='api-v4-lambda' REGION=us-east-2 NO_ECHO=1 DTTO='1 second ago' OUT="api-logs-${STAGE}-2.json" ./utils/search_aws_logs.sh 'LG:api-request-path'
jq -s 'add' "api-logs-${STAGE}-1.json" "api-logs-${STAGE}-2.json" > "api-logs-${STAGE}.json" && rm -f "api-logs-${STAGE}-1.json" "api-logs-${STAGE}-2.json"
./utils/count_apis.sh "api-logs-${STAGE}.json" > "api-logs-${STAGE}.log" && cat "api-logs-${STAGE}.log"
cat "api-logs-${STAGE}.json" | grep '"logGroupName": "' | sort | uniq
26 changes: 26 additions & 0 deletions utils/find_1st_log_entry.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash
# Copyright The Linux Foundation and each contributor to LFX.
# SPDX-License-Identifier: MIT
if [ -z "${STAGE}" ]
then
export STAGE=dev
fi
# Array of log group configurations: "region log_group_name"
log_groups=(
"us-east-1 /aws/lambda/cla-backend-${STAGE}-api-v3-lambda"
"us-east-1 /aws/lambda/cla-backend-${STAGE}-apiv1"
"us-east-1 /aws/lambda/cla-backend-${STAGE}-apiv2"
"us-east-1 /aws/lambda/cla-backend-${STAGE}-githubactivity"
"us-east-1 /aws/lambda/cla-backend-${STAGE}-githubinstall"
"us-east-2 /aws/lambda/cla-backend-go-api-v4-lambda"
)

for entry in "${log_groups[@]}"; do
region=$(echo "$entry" | awk '{print $1}')
log_group=$(echo "$entry" | cut -d' ' -f2-)
aws --region "$region" --profile "lfproduct-${STAGE}" logs filter-log-events \
--log-group-name "$log_group" \
--start-time 0 --limit 1 --filter-pattern "\"LG:api-request-path\"" \
| jq -r '.events.[0].timestamp' \
| awk '{print strftime("%Y-%m-%d %H:%M:%S", $1/1000)}'
done
Loading