feat(log-collector-script): journal log with precise timestamp#2151
feat(log-collector-script): journal log with precise timestamp#2151guessi wants to merge 1 commit intoawslabs:mainfrom
Conversation
| readonly JOURNALCTL_CMD='/usr/bin/journalctl --output=short-iso-precise' | ||
|
|
||
| # Collect log start from 10 days ago, could be '-10d (10 days ago)', '-10h (10 hours ago)', '-10m (10 minutes ago)' | ||
| readonly JOURNALCTL_SINCE='-10d' |
There was a problem hiding this comment.
If we need to gather log since 30 days ago, it could easily be done by setting the value as -30d.
| readonly LOG_DIR="/var/log" | ||
| readonly COLLECT_DIR="/tmp/eks-log-collector" | ||
| readonly CURRENT_TIME=$(date --utc +%Y-%m-%d_%H%M-%Z) | ||
| readonly DAYS_10=$(date -d "-10 days" '+%Y-%m-%d %H:%M') |
There was a problem hiding this comment.
Before change, it is confused to have hard-coded DAYS_10 here. It's just a variable, would better to have no explicitly days defined in variable name.
|
|
||
| timeout 75 journalctl --unit=nodeadm-run --since "${DAYS_10}" > "${COLLECT_DIR}"/nodeadm/nodeadm-run.log | ||
| timeout 75 ${JOURNALCTL_CMD} --unit=nodeadm-config > "${COLLECT_DIR}"/nodeadm/nodeadm-config.log | ||
| timeout 75 ${JOURNALCTL_CMD} --unit=nodeadm-run > "${COLLECT_DIR}"/nodeadm/nodeadm-run.log |
There was a problem hiding this comment.
No need to add --since for the nodadm-{config,run} services.
guessi
force-pushed
the
fine-tune-log-collector
branch
from
e909a99 to
d275801
Compare
| timeout 75 killall -sUSR1 containerd | ||
| timeout 75 containerd config dump > "${COLLECT_DIR}"/containerd/containerd-config.txt 2>&1 || echo -e "\tTimed out, ignoring \"containerd info output \" " | ||
| timeout 75 journalctl -u containerd > "${COLLECT_DIR}"/containerd/containerd-log.txt 2>&1 || echo -e "\tTimed out, ignoring \"containerd info output \" " | ||
| timeout 75 ${JOURNALCTL_CMD} --unit=containerd --since "${JOURNALCTL_SINCE}" > "${COLLECT_DIR}"/containerd/containerd-log.txt 2>&1 || echo -e "\tTimed out, ignoring \"containerd info output \" " |
There was a problem hiding this comment.
Add --since flag to aligned with log gathering command for docker.
|
|
||
| # Unify the out output with precise ISO timestamp (e.g. 2025-02-12T06:25:29.149489+0000) | ||
| # It might be needed to have nanosecond timestamp for advanced troubleshooting. | ||
| readonly JOURNALCTL_CMD='/usr/bin/journalctl --output=short-iso-precise' |
There was a problem hiding this comment.
EKS node might have customized timezone set, precisely log the timestamp would be very helpful for troubleshooting.
|
This pull request is stale because it has been open for 60 days with no activity. Remove the stale label or comment to avoid closure in 14 days |
guessi
force-pushed
the
fine-tune-log-collector
branch
from
d275801 to
28453f0
Compare
|
Not stale, just rebase again and resolved conflict. |
1 participant
Issue #, if available:
Description of changes:
Feb 12 06:25:29timezoneinfo (it might have different timezone setup at node bootstrap).secondonly, no that precise.2025-02-12T06:25:29.149489+0000timezoneinfo included.nanosecond.journalctlcommands inside log-collector script.${JOURNALCTL_CMD}defined, try to unify the command with--outputflag set.timeout->journalctl->--unit-->--since (if needed)--since $(date -d "-10 days" '+%Y-%m-%d %H:%M')--since "-10d"Testing Done