Skip to content

log-collector-script checks for netfilter tables & updates to network info scrape #2514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
abeowlu wants to merge 44 commits into
base: main
Choose a base branch
from
Open

log-collector-script checks for netfilter tables & updates to network info scrape #2514

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
05f49d7
cri: containerd for cluster req kubelet<v1.24
abeowlu Oct 11, 2022
1325e31
nftables kubeproxy mode
abeowlu Nov 14, 2025
ce2ac5b
shfmt lint
abeowlu Nov 14, 2025
63d6f90
update netwk commd name
abeowlu Nov 14, 2025
e17f863
update netwk commd name
abeowlu Nov 14, 2025
bf1f01e
shfmrt & suppress error
abeowlu Nov 14, 2025
585f018
changes
abeowlu Nov 14, 2025
ec59839
chore(docs): update CHANGELOG.md for release v20251112
web-flow Nov 16, 2025
85f99e7
chore: update dependencies
dependabot[bot] Nov 17, 2025
3783df3
chore(build,al2023): remove containerd 1.7 override for 1.30 (#2517)
wwvela Nov 17, 2025
f2103c2
feat(workflows): add stale issue and PR workflow (#2519)
mselim00 Nov 19, 2025
4c1c316
fix(al2023): fix ci e2e test flakiness (#2520)
shvbsle Nov 19, 2025
221a1ac
ci: fix eni max pods syntax (#2521)
mselim00 Nov 19, 2025
acac4a3
fix(ci/max-pods): remove duplicate create PR step (#2524)
mselim00 Nov 20, 2025
9191481
chore: enhance .gitignore to filter out irrelevant python files (#2522)
shvbsle Nov 20, 2025
2da0f0e
chore: update eni-max-pods.txt
web-flow Nov 20, 2025
34aa041
chore(docs): update CHANGELOG.md for release v20251120
web-flow Nov 22, 2025
9161128
chore: update eni-max-pods.txt
web-flow Nov 22, 2025
a3c0b86
updated readme for NodeLogs instructions (#2537)
viveksb007 Nov 26, 2025
9b273bd
feat(al2): added a provisioner to setup Neuron support for AL2 AMI
Youssef-Beltagy Nov 20, 2025
2ccad3c
feat(al2): added a provisioner to setup EFA support for AL2 AMI
Youssef-Beltagy Nov 20, 2025
74286eb
feat(al2): added a provisioner to reduce cpu sleep for accelerated AL…
Youssef-Beltagy Nov 20, 2025
c5c6969
feat(al2): added a provisioner to setup dependencies for accelerated …
Youssef-Beltagy Nov 20, 2025
80a2068
feat(al2): added a provisioner to setup Nvidia support for AL2 AMI
Youssef-Beltagy Nov 21, 2025
17ea3bd
doc(al2): updated AL2 docs with accelerated AMI parameters
Youssef-Beltagy Nov 26, 2025
2ce245c
Deprecation update
tucktuck9 Nov 26, 2025
66bf585
Revise AL2 documentation for EKS AL2 AMI changes
tucktuck9 Nov 26, 2025
00a8771
chore: update dependencies
dependabot[bot] Nov 24, 2025
d20909b
chore: bump actions/dependency-review-action from 4.8.1 to 4.8.2
dependabot[bot] Nov 24, 2025
cb3186c
chore: bump actions/stale from 5 to 10
dependabot[bot] Nov 24, 2025
c51507b
chore: bump peter-evans/create-pull-request from 7.0.8 to 7.0.9
dependabot[bot] Nov 24, 2025
426c67e
chore: bump actions/setup-go from 6.0.0 to 6.1.0
dependabot[bot] Nov 24, 2025
0de9a4e
feat(log-collector): collect mountpoint-s3-csi-driver logs
dannycjones Nov 26, 2025
aad4ef1
chore: bump actions/checkout from 5.0.0 to 6.0.0
dependabot[bot] Nov 28, 2025
706d99c
chore: update dependencies
dependabot[bot] Dec 1, 2025
8b5c28c
fix(ci): honor never-stale label for PR/issue cleanup
mselim00 Dec 1, 2025
ff4add1
chore: bump aws-actions/configure-aws-credentials from 5.1.0 to 5.1.1
dependabot[bot] Dec 1, 2025
3bf54ab
chore: update eni-max-pods.txt
web-flow Dec 3, 2025
a990351
cri: containerd for cluster req kubelet<v1.24
abeowlu Oct 11, 2022
b8d918b
pull in deleted al2 files
abeowlu Dec 7, 2025
0e21f0e
replay log-script changes on latest
abeowlu Dec 7, 2025
3cbd08c
using script pattern
abeowlu Dec 7, 2025
c8a6bbe
confirm script to pattern
abeowlu Dec 7, 2025
bf77abf
confirm script to pattern
abeowlu Dec 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 31 additions & 10 deletions log-collector-script/linux/eks-log-collector.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ collect() {
get_modinfo
get_mounts_info
get_selinux_info
get_iptables_info
get_network_tables_info
get_pkglist
get_system_services
get_containerd_info
Expand Down Expand Up @@ -376,7 +376,7 @@ get_selinux_info() {
ok
}

get_iptables_info() {
get_network_tables_info() {
if ! command -v iptables > /dev/null 2>&1; then
echo "IPtables not installed" | tee -a iptables.txt
else
Expand All @@ -398,19 +398,29 @@ get_iptables_info() {
else
# check that ip_vs module is loaded in get_modinfo()
try "collect ipvs information"
ipvsadm --save | tee "${COLLECT_DIR}"/networking/ipvsadm.txt
ok -e "\n" | tee -a "${COLLECT_DIR}"/networking/ipvsadm.txt
ipvsadm --list --numeric --rate | tee -a "${COLLECT_DIR}"/networking/ipvsadm.txt
ok -e "\n" | tee -a "${COLLECT_DIR}"/networking/ipvsadm.txt
ipvsadm --list --numeric --stats --exact | tee -a "${COLLECT_DIR}"/networking/ipvsadm.txt
ipvsadm --save >> "${COLLECT_DIR}"/networking/ipvsadm.txt
ok -e "\n" >> "${COLLECT_DIR}"/networking/ipvsadm.txt
ipvsadm --list --numeric --rate >> "${COLLECT_DIR}"/networking/ipvsadm.txt
ok -e "\n" >> "${COLLECT_DIR}"/networking/ipvsadm.txt
ipvsadm --list --numeric --stats --exact >> "${COLLECT_DIR}"/networking/ipvsadm.txt
Comment on lines +403 to +407
Copy link
Copy Markdown
Contributor

@mselim00 mselim00 Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's keep these as they were. if you can share details on why appending would be more appropriate here, we can evaluate this for a separate change

Copy link
Copy Markdown
Contributor Author

@abeowlu abeowlu Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • my local repo script does not output/tee ipvs info collected to stdout, but the testing the script on main does...
        This is version 0.7.8. New versions can be found at https://github.com/awslabs/amazon-eks-ami/blob/main/log-collector-script/

Trying to collect common operating system logs...
Trying to collect kernel logs...
Trying to collect modinfo... Trying to collect mount points and volume information...
Trying to collect SELinux status...
Trying to collect iptables information... /sbin/ipvsadm
Trying to collect ipvs information... -A -t 198.19.10.0:webcache -s rr
-a -t 198.19.10.0:webcache -r 100.64.0.9:webcache -m -w 1
-A -t 198.19.20.0:webcache -s rr
-a -t 198.19.20.0:webcache -r 100.64.0.9:webcache -m -w 1
-A -t 198.19.30.0:webcache -s rr
-a -t 198.19.30.0:webcache -r 100.64.0.9:webcache -m -w 1
sed: -e expression #1, char 18: unknown option to `s'
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port                 CPS    InPPS   OutPPS    InBPS   OutBPS
  -> RemoteAddress:Port
TCP  198.19.10.0:8080                    0        0        0        0        0
  -> 100.64.0.9:8080                     0        0        0        0        0
TCP  198.19.20.0:8080                    0        0        0        0        0
  -> 100.64.0.9:8080                     0        0        0        0        0
TCP  198.19.30.0:8080                    0        0        0        0        0
  -> 100.64.0.9:8080                     0        0        0        0        0

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  198.19.10.0:8080                    0        0        0        0        0
...
  • can become unnecessarily verbose. I can update to ipvsadm --save | tee ${COLLECT_DIR}"/networking/... >> ${COLLECT_DIR}"/networking/... like in my local repo, but I think this redundant so the shortened to append to the log file. Script output now is;
        This is version 0.7.9. New versions can be found at https://github.com/awslabs/amazon-eks-ami/blob/main/log-collector-script/

Trying to collect common operating system logs...
Trying to collect kernel logs...
Trying to collect modinfo... Trying to collect mount points and volume information...
Trying to collect SELinux status...
Trying to collect iptables information...
Trying to collect ipvs information...

Trying to collect nftables information... skipping nftable info: if using nftables install 'nft' utility and try again
Trying to collect installed packages...
Trying to collect active system services...

fi

if ! command -v ipset > /dev/null 2>&1; then
echo "ipset not installed" | tee "${COLLECT_DIR}"/networking/ipset.txt
else
ipset --list | tee "${COLLECT_DIR}"/networking/ipset.txt
ok -e "\n" | tee -a "${COLLECT_DIR}"/networking/ipset.txt
ipset --save | tee -a "${COLLECT_DIR}"/networking/ipset.txt
ipset --list >> "${COLLECT_DIR}"/networking/ipset.txt
ok -e "\n" >> "${COLLECT_DIR}"/networking/ipset.txt
ipset --save >> "${COLLECT_DIR}"/networking/ipset.txt
fi

if lsmod | grep nf_tables > /dev/null 2>&1; then
if ! command -v nft --version > /dev/null 2>&1; then
try "install nftables"
yum install nftables -y > /dev/nulll 2>&1
get_nftables
fi
else
get_nftables
fi

ok
Expand Down Expand Up @@ -478,6 +488,17 @@ get_modinfo() {
lsmod | grep -e ip_vs -e nf_conntrack > "${COLLECT_DIR}/modinfo/ip_vs"
}

get_nftables() {
try "collect nftables information"

nft list tables | grep -E '^(table|flush)' | while read -r _ family name; do
FILENAME="${family}_${name}.txt"
nft list table "$family" "$name" > "${COLLECT_DIR}"/networking/$FILENAME
done

ok
}

get_docker_logs() {
try "collect Docker daemon logs"

Expand Down
24 changes: 0 additions & 24 deletions templates/al2/runtime/kubelet.service
View file
Open in desktop

This file was deleted.

286 changes: 0 additions & 286 deletions templates/al2/template.json
View file
Open in desktop

This file was deleted.