[filebeat][ABS] - Fix CSV decoder JSON escaping in azure-blob-storage input

[ShourieG]

Type of change

• Bug

Proposed commit message

x-pack/filebeat/input/azureblobstorage: Fix CSV decoder JSON escaping in azure-blob-storage input

The azure-blob-storage input's decode path only matched the
decoder.Decoder interface, calling Decode() which builds JSON via
string concatenation without escaping field values. CSV values
containing double quotes (e.g. RFC 2045 MIME type parameters)
produce malformed JSON, causing downstream ingest pipeline failures.

Added a decoder.ValueDecoder switch case ahead of decoder.Decoder,
matching the pattern already used by the GCS input. ValueDecoder's
DecodeValue() uses json.Marshal which handles escaping correctly.

NOTE

We changed an existing test case because after adding decodeValue it builds a map[string]any, then calls json.Marshal. Go's json.Marshal sorts map keys alphabetically so the expected test results needed to align accordingly.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 698ec0e5-78cb-4161-bd77-af97a10ffbc5

📥 Commits

Reviewing files that changed from the base of the PR and between 15d1529 and 44a6169.

📒 Files selected for processing (1)

• x-pack/filebeat/input/azureblobstorage/job.go

🚧 Files skipped from review as they are similar to previous changes (1)

• x-pack/filebeat/input/azureblobstorage/job.go

---

📝 Walkthrough

Walkthrough

Fixes a CSV-decoding bug in the Azure Blob Storage input that produced malformed JSON when CSV fields contained double quotes. The decoder now escapes values via json.Marshal. job.decode gains a decoder.ValueDecoder branch that iterates with Next()/DecodeValue(), creates events with createEvent(string(msg), offset) and sets the last flag from !dec.More(). Adds a csv_quoted_values test and txn_quoted.json fixture; updates mock CSV JSON record field ordering.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ShourieG]

/test

[github-actions]

@Mergifyio backport 8.19 9.3 9.4

[mergify]

backport 8.19 9.3 9.4

✅ Backports have been created

Details

• #50108 [8.19](backport #50097) [filebeat][ABS] - Fix CSV decoder JSON escaping in azure-blob-storage input has been created for branch 8.19 but encountered conflicts

Cherry-pick of 34634a3 has failed:

On branch mergify/bp/8.19/pr-50097
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 34634a3e8.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1776095974-fix-csv-decoder-json-escaping-abs-input.yaml
	modified:   x-pack/filebeat/input/azureblobstorage/decoding_test.go
	modified:   x-pack/filebeat/input/azureblobstorage/mock/data_files.go
	new file:   x-pack/filebeat/input/azureblobstorage/testdata/txn_quoted.csv
	new file:   x-pack/filebeat/input/azureblobstorage/testdata/txn_quoted.json

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   x-pack/filebeat/input/azureblobstorage/job.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #50109 [9.3](backport #50097) [filebeat][ABS] - Fix CSV decoder JSON escaping in azure-blob-storage input has been created for branch 9.3
• #50110 [9.4](backport #50097) [filebeat][ABS] - Fix CSV decoder JSON escaping in azure-blob-storage input has been created for branch 9.4