Fix container ignoring max bytes when parsing CRI partial messages by belimawr · Pull Request #49743 · elastic/beats

belimawr · 2026-03-27T16:54:25Z

Proposed commit message

See title

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
I have added an entry in ./changelog/fragments using the changelog tool.

~~## Disruptive User Impact~~
~~## Author's Checklist~~

How to test this PR locally

1. Run the tests

go test -count=1 ./libbeat/reader/parser ./libbeat/reader/readjson ./filebeat/input/log

The test added by this PR is TestDockerJSONMaxBytes in libbeat/reader/readjson/docker_json_test.go.

2. Manual testing

Follow the 'steps to reproduce' from #49259, you'll have to add allow_deprecated_use: true to the input configuration to run Filebeat >= 9.0.0.

Or build the docker image then run the following script.
To build:

SNAPSHOT=true PACKAGES=docker PLATFORMS=linux/amd64 mage -v package

If the test fails at the end of the build, don't worry, that's likely because your Docker version is >= 29. Just ensure the image docker.elastic.co/beats/filebeat-oss:9.4.0-SNAPSHOT was built.

#!/bin/bash

# Generate a CRI log with 1000 partial lines (~65KB each, ~63MB logical line)
mkdir -p /tmp/fb-oom-test/containers /tmp/fb-oom-test/registry
CHUNK=$(head -c 65000 /dev/zero | tr '\0' 'A')
{
  echo '2024-01-01T00:00:00.000000000Z stdout F {"message":"normal"}'
  for i in $(seq 1 1000); do
    printf '2024-01-01T00:00:00.%09dZ stdout P %s\n' "$i" "$CHUNK"
  done
  echo '2024-01-01T00:01:00.000000000Z stdout F end'
} > /tmp/fb-oom-test/containers/test_default_main-abcdef123456.log

# Create minimal filebeat config
cat > /tmp/fb-oom-test/filebeat.yml << 'CONF'
filebeat.inputs:
  - type: container
    max_bytes: 65536
    allow_deprecated_use: true
    paths:
      - /var/log/containers/*.log
output.file:
  path: /tmp
  filename: fb-out
  codec.format:
    string: ''
setup:
  ilm.enabled: false
  template.enabled: false
logging:
  level: warning
  to_stderr: true
CONF

echo "Running Filebeat 9.3.2 - latest release"
docker run --rm --name fb-oom-test --user=root \
  --network=none --memory=64m \
  -v /tmp/fb-oom-test/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro \
  -v /tmp/fb-oom-test/containers:/var/log/containers:ro \
  -v /tmp/fb-oom-test/registry:/usr/share/filebeat/data \
  docker.elastic.co/beats/filebeat:9.3.2 \
  filebeat -e --strict.perms=false

exit_code=$?

echo
echo

if [ $exit_code -eq 137 ]; then
    echo "Error: Process killed by OOM (Out of Memory) Killer (Exit Code: 137)"
elif [ $exit_code -ne 0 ]; then
    echo "Process failed with exit code: $exit_code"
fi

echo
echo

echo "Running Filebeat local built"
echo

sudo rm -rf /tmp/fb-oom-test/registry

docker run --rm --name fb-oom-test \
  --network=none --memory=64m --user=root \
  -v /tmp/fb-oom-test/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro \
  -v /tmp/fb-oom-test/containers:/var/log/containers:ro \
  -v /tmp/fb-oom-test/registry:/usr/share/filebeat/data \
  docker.elastic.co/beats/filebeat-oss:9.4.0-SNAPSHOT \
  filebeat -e --strict.perms=false

exit_code=$?

if [ $exit_code -eq 137 ]; then
    echo "Error: Process killed by OOM (Out of Memory) Killer (Exit Code: 137)"
elif [ $exit_code -ne 0 ]; then
    echo "Process failed with exit code: $exit_code"
fi

It will print Error: Process killed by OOM (Out of Memory) Killer (Exit Code: 137) for the latest release and will just 'hang' (well, keep executing) with the currently build Filebeat

Related issues

Fixes container input: CRI partial line reassembly ignores max_bytes, causing OOM #49259

~~## Use cases~~
~~## Screenshots~~
~~## Logs~~

GenAI-Assisted: Yes Human-Reviewed: Yes Tool: Claude-CLI, Model: Sonet 4.6

github-actions · 2026-03-27T16:54:37Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

…noring-max-bytes

Copilot

Pull request overview

This PR addresses an OOM risk in Filebeat’s container/CRI log parsing by enforcing the max_bytes limit during CRI partial-line reassembly (instead of only after reassembly), and adds a regression test plus a changelog fragment.

Changes:

Add maxBytes support to DockerJSONReader and enforce it while joining CRI partial (P) chunks.
Plumb max_bytes from the harvester/parser pipeline into the Docker/CRI JSON reader.
Add a unit test and a changelog fragment for the fix.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`libbeat/reader/readjson/docker_json.go`	Introduces `maxBytes` and truncation logic during partial reassembly.
`libbeat/reader/readjson/docker_json_test.go`	Updates constructor usage and adds a new max-bytes regression test.
`libbeat/reader/parser/parser.go`	Passes configured `MaxBytes` into the container parser.
`filebeat/input/log/harvester.go`	Passes `MaxBytes` into the Docker JSON reader; normalizes some error strings.
`changelog/fragments/1774630083-fix-container-max-bytes.yaml`	Adds release note entry for the bug fix.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libbeat/reader/readjson/docker_json.go

libbeat/reader/readjson/docker_json_test.go

changelog/fragments/1774630083-fix-container-max-bytes.yaml

libbeat/reader/readjson/docker_json.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

elasticmachine · 2026-03-27T21:27:35Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libbeat/reader/readjson/docker_json.go

changelog/fragments/1774630083-fix-container-max-bytes.yaml

coderabbitai · 2026-03-27T21:32:26Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c3e065f5-6f37-4dbf-97c8-b9ab9c9bf663

📥 Commits

Reviewing files that changed from the base of the PR and between 57cb928 and a1af03a.

📒 Files selected for processing (1)

libbeat/reader/readjson/docker_json.go

🚧 Files skipped from review as they are similar to previous changes (1)

libbeat/reader/readjson/docker_json.go

📝 Walkthrough

Walkthrough

Adds a changelog fragment and wires maxBytes through container log parsing. DockerJSONReader gains a maxBytes field and updated constructors; Next() enforces the limit during CRI partial reassembly by truncating appended content, setting a truncated flag, and draining remaining partial fragments. The container parser and harvester now pass MaxBytes into the JSON reader. Some harvester error messages were lowercased. Tests were added/updated to validate truncation and draining behavior.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR fully addresses issue `#49259` by implementing max_bytes enforcement in CRI partial line reassembly, adding truncation logic, and introducing comprehensive tests.
Out of Scope Changes check	✅ Passed	All changes directly support the max_bytes enforcement objective. Minor error message capitalization updates are reasonable refactoring within scope.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

🛠️ Update Documentation: Commit on current branch
🛠️ Update Documentation: Create PR

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@changelog/fragments/1774630083-fix-container-max-bytes.yaml`:
- Line 12: In the changelog fragment, update the YAML key value for "kind" from
"bug" to the exact expected token "bug-fix" so the changelog tool recognizes
this as a fix; locate the "kind" entry in the fragment (the line currently
reading kind: bug) and replace its value with kind: bug-fix.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: beda5ed9-4c33-44f8-8ccf-568533f875b4

📥 Commits

Reviewing files that changed from the base of the PR and between 12e4652 and 79d8c39.

📒 Files selected for processing (5)

changelog/fragments/1774630083-fix-container-max-bytes.yaml
filebeat/input/log/harvester.go
libbeat/reader/parser/parser.go
libbeat/reader/readjson/docker_json.go
libbeat/reader/readjson/docker_json_test.go

changelog/fragments/1774630083-fix-container-max-bytes.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

belimawr added 2 commits March 27, 2026 11:50

Add max bytes support to DockerJSONReader

f9be708

GenAI-Assisted: Yes Human-Reviewed: Yes Tool: Claude-CLI, Model: Sonet 4.6

Add changelog

5455e5a

belimawr self-assigned this Mar 27, 2026

belimawr added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team bugfix backport-active-all Automated backport with mergify to all the active branches labels Mar 27, 2026

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Mar 27, 2026

belimawr added 2 commits March 27, 2026 16:43

Fix lint warnings

090d3ef

Merge branch 'main' of github.com:elastic/beats into fix-container-ig…

e8e0a88

…noring-max-bytes

belimawr requested a review from Copilot March 27, 2026 20:44

Copilot started reviewing on behalf of belimawr March 27, 2026 20:45 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

belimawr and others added 3 commits March 27, 2026 17:22

Update changelog/fragments/1774630083-fix-container-max-bytes.yaml

b6f9398

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update libbeat/reader/readjson/docker_json.go

1590c7d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Drain truncated lines

79d8c39

belimawr requested a review from Copilot March 27, 2026 21:27

belimawr marked this pull request as ready for review March 27, 2026 21:27

belimawr requested a review from a team as a code owner March 27, 2026 21:27

belimawr requested review from andrzej-stencel and orestisfl March 27, 2026 21:27

Copilot started reviewing on behalf of belimawr March 27, 2026 21:27 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

libbeat/reader/readjson/docker_json.go Outdated Show resolved Hide resolved

changelog/fragments/1774630083-fix-container-max-bytes.yaml Outdated Show resolved Hide resolved

coderabbitai bot reviewed Mar 27, 2026

View reviewed changes

changelog/fragments/1774630083-fix-container-max-bytes.yaml Outdated Show resolved Hide resolved

belimawr and others added 2 commits March 27, 2026 17:38

Fix changelog kind

57cb928

Update libbeat/reader/readjson/docker_json.go

a1af03a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix container ignoring max bytes when parsing CRI partial messages#49743

Fix container ignoring max bytes when parsing CRI partial messages#49743
belimawr wants to merge 9 commits intoelastic:mainfrom
belimawr:fix-container-ignoring-max-bytes

belimawr commented Mar 27, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai bot commented Mar 27, 2026 •

edited

Loading

Walkthrough

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

belimawr commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Related issues

Uh oh!

github-actions bot commented Mar 27, 2026

🤖 GitHub comments

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

coderabbitai bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

belimawr commented Mar 27, 2026 •

edited

Loading

coderabbitai bot commented Mar 27, 2026 •

edited

Loading