[debug] check kernel logs

[kolyshkin]

Debugging #1999

[gemini-code-assist]

Summary of Changes

Hello @kolyshkin, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request implements a targeted debugging enhancement to the e2e test suite. The primary goal is to improve diagnostic capabilities by ensuring that relevant kernel logs are consistently collected after test runs, regardless of their outcome. This change is specifically aimed at investigating a reported issue, likely involving filesystem interactions within the container runtime.

Highlights

• Enhanced Debugging Workflow: Modified the run-tests.sh script to ensure that kernel logs are always captured after the ginkgo e2e tests, even if the tests themselves fail. This is achieved by appending || true to the test command.
• Kernel Log Inspection for OverlayFS: Introduced a new command, dmesg | grep overlay, to be executed after the e2e tests. This command specifically filters kernel messages for entries related to 'overlay', aiding in the diagnosis of issues potentially linked to overlay filesystem operations, which are common in container environments.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• tests/podman/run-tests.sh
  • The ginkgo e2e test command was modified to always return a successful exit code (0) by appending || true. This ensures that subsequent commands in the script will execute, even if the tests fail.
  • A new command, dmesg | grep overlay, was added to the script. This command will execute after the ginkgo tests and output kernel messages filtered for 'overlay' related content.

Activity

• No specific activity (comments, reviews, etc.) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request appears to be for debugging purposes, adding a command to check kernel logs after tests run. However, the current implementation uses || true which will cause the test suite to always report success, even if there are test failures. This is a critical issue for CI. I've provided a review comment with a suggested code change to ensure the kernel logs are printed without suppressing the actual test results.

[gemini-code-assist]

Using || true will suppress the exit code of the test suite, causing the CI to pass even if tests fail. This is a critical issue as it can hide genuine test failures. To ensure kernel logs are captured on failure without suppressing the test result, you should capture the exit code of the test command and then exit with it after printing the logs.

Suggested change

	-timeout=50m -cover -flake-attempts 3 -progress -trace -no-color test/e2e/. || true
	dmesg | grep overlay
	-timeout=50m -cover -flake-attempts 3 -progress -trace -no-color test/e2e/.
	result=$?
	dmesg | grep overlay
	exit $result

[packit-as-a-service]

TMT tests failed. @containers/packit-build please check.

[kolyshkin]

2026-02-08T23:10:04.2433378Z [ 1145.466923] overlay: filesystem on /root/go/src/github.com/containers/podman/test/e2e/buildah-context-819518267/overlay/1575444993/upper not supported as upperdir
2026-02-08T23:10:04.2434531Z [ 1145.707035] overlay: filesystem on /root/go/src/github.com/containers/podman/test/e2e/buildah-context-336131691/overlay/2370125604/upper not supported as upperdir
2026-02-08T23:10:04.2435591Z [ 1145.949283] overlay: filesystem on /root/go/src/github.com/containers/podman/test/e2e/buildah-context-2195722604/overlay/1913652838/upper not supported as upperdir

Usually upperdir is either under /var/lib/containers or /var/tmp, both are bind-mounted from the host here:

crun/.github/workflows/test.yaml

Line 133 in 0bc3050

sudo docker run --cgroupns=host --privileged --rm -v /var/tmp:/var/tmp:rw -v /var/lib/containers:/var/lib/containers:rw -v /sys/fs/cgroup:/sys/fs/cgroup:rw,rslave -v ${PWD}:/crun crun-podman

and therefore are not on overlayfs.

We need to exclude the test.