Skip to content

filter_kubernetes: add owner references information into record.#9719

Merged
edsiper merged 1 commit intofluent:masterfrom
shuaich:master
Feb 28, 2025
Merged

filter_kubernetes: add owner references information into record.#9719
edsiper merged 1 commit intofluent:masterfrom
shuaich:master

Conversation

@shuaich
Copy link
Contributor

@shuaich shuaich commented Dec 12, 2024 •
edited
Loading

This change add owner references information into record when Owner_References parameter is set On.

This PR implements the feature request in #9631.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
filters:
  - name:                kubernetes
          Match:                       kube_*
          ...
          Owner_References: On
  • Debug log output from testing the change
"kubernetes"=>{"pod_name"=>"fluentbit-gke-2p6b5", "namespace_name"=>"kube-system", "pod_id"=>"c759a5f5-xxxx-xxxx-9117-8a1dc0b1f907", "labels"=>{"component"=>"fluentbit-gke", "controller-revision-hash"=>"77665fff9", "k8s-app"=>"fluentbit-gke", "kubernetes.io/cluster-service"=>"true", "pod-template-generation"=>"1"}, "ownerReferences"=>[{"apiVersion"=>"apps/v1", "kind"=>"DaemonSet", "name"=>"fluentbit-gke", "uid"=>"1a12c3e2-d6c4-4a8a-b877-dd3c857d1aea", "controller"=>true, "blockOwnerDeletion"=>true}], "host"=>"xxx-2a9c049c-qgw3", "pod_ip"=>"10.128.0.111", "container_name"=>"fluentbit", "docker_id"=>"2acc16e490e8d6xxxacbb45fe6c519c4f55593e00cd59ff3178b3a92965c88", "container_hash"=>"xxx", "container_image"=>"sha256:5163d704420bb702ba4df39cc774c2961b4f5450ab843aa85f0e92aeb2321ea2"}}
  • Attached Valgrind output that shows no leaks or memory corruption was found
valgrind ./bin/flb-rt-filter_kubernetes
==1967769== Memcheck, a memory error detector
==1967769== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==1967769== Using Valgrind-3.20.0 and LibVEX; rerun with -h for copyright info
==1967769== Command: ./bin/flb-rt-filter_kubernetes
==1967769==
Test kube_core_base...                          [ OK ]
Test kube_core_no_meta...                       [ OK ]
Test kube_core_unescaping_text...               [ OK ]
Test kube_core_unescaping_json...               [ OK ]
Test kube_core_base_with_namespace_labels_and_annotations... [ OK ]
Test kube_core_base_with_owner_references...    [ OK ]
Test kube_options_use-kubelet_enabled_json...   [ OK ]
Test kube_options_use-kubelet_disabled_json...  [ OK ]
Test kube_options_merge_log_enabled_text...     [ OK ]
Test kube_options_merge_log_enabled_json...     [ OK ]
Test kube_options_merge_log_enabled_invalid_json... [ OK ]
Test kube_options_merge_log_disabled_json...    [ OK ]
Test kube_options_merge_log_trim_enabled_json... [ OK ]
Test kube_options_merge_log_trim_disabled_json... [ OK ]
Test kube_options_merge_log_key_json...         [ OK ]
Test kube_options_keep_log_enabled_json...      [ OK ]
Test kube_options_keep_log_disabled_json...     [ OK ]
Test kube_options_k8s_logging_parser_disabled_text_stdout... [ OK ]
Test kube_options_k8s_logging_parser_disabled_text_stderr... [ OK ]
Test kube_options_k8s_logging_exclude_disabled_text_stdout... [ OK ]
Test kube_options_k8s_logging_exclude_disabled_text_stderr... [ OK ]
Test kube_annotations_invalid_text...           [ OK ]
Test kube_annotations_parser_regex_with_time_text... [ OK ]
Test kube_annotations_parser_regex_with_time_invalid_text_1... [ OK ]
Test kube_annotations_parser_json_with_time_json... [ OK ]
Test kube_annotations_parser_json_with_time_invalid_json_1... [ OK ]
Test kube_annotations_parser_invalid_text_stdout... [ OK ]
Test kube_annotations_parser_invalid_text_stderr... [ OK ]
Test kube_annotations_parser_stdout_text_stdout... [ OK ]
Test kube_annotations_parser_stdout_text_stderr... [ OK ]
Test kube_annotations_parser_stderr_text_stdout... [ OK ]
Test kube_annotations_parser_stderr_text_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_1_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_1_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_2_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_2_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_3_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_3_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_4_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_4_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_5_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_5_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_1_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_1_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_2_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_2_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_3_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_3_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_4_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_4_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_5_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_5_stderr... [ OK ]
Test kube_annotations_exclude_default_text...   [ OK ]
Test kube_annotations_exclude_invalid_text_stdout... [ OK ]
Test kube_annotations_exclude_invalid_text_stderr... [ OK ]
Test kube_annotations_exclude_stdout_text_stdout... [ OK ]
Test kube_annotations_exclude_stdout_text_stderr... [ OK ]
Test kube_annotations_exclude_stderr_text_stdout... [ OK ]
Test kube_annotations_exclude_stderr_text_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_4_stderr... [ OK ]
SUCCESS: All unit tests have passed.
==1967769==
==1967769== HEAP SUMMARY:
==1967769==     in use at exit: 0 bytes in 0 blocks
==1967769==   total heap usage: 298,782 allocs, 298,782 frees, 106,224,699 bytes allocated
==1967769==
==1967769== All heap blocks were freed -- no leaks are possible
==1967769==
==1967769== For lists of detected and suppressed errors, rerun with: -s
==1967769== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Need to backport to branch 3.2.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@JeffLuoo
Copy link
Contributor

JeffLuoo commented Dec 20, 2024

LGTM

@shuaich
Copy link
Contributor Author

shuaich commented Jan 8, 2025

Gentle ping. May I have the feedback on this PR?

edsiper
edsiper requested changes Jan 9, 2025
View reviewed changes
lib/cprofiles/src/cprof_profile.c
profile->comments_count++;

return 0;
} No newline at end of file
Copy link
Member

@edsiper edsiper Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please do not touch this file

Copy link
Contributor Author

@shuaich shuaich Jan 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I had to manually apply local changes to cprof files to make the build and run the unit test. I then manually reverted those changes but still got the one line diff, which I don't still understand.

Do maintainers have the permission to revert this file change in this PR?

Copy link
Contributor Author

@shuaich shuaich Jan 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out there is an additional newline character appended by my vim editor. :set noeol fixes the one line diff. See https://stackoverflow.com/questions/1050640/how-to-stop-vim-from-adding-a-newline-at-end-of-file

@shuaich
filter_kubernetes: add owner references information into record.
7ca3670 
Signed-off-by: shuaichen <shuaichen@google.com>
@edsiper edsiper added this to the Fluent Bit v4.0.0 milestone Jan 10, 2025
@shuaich
Copy link
Contributor Author

shuaich commented Jan 16, 2025

@edsiper Could we backport this PR to v3.2+?

@shuaich
Copy link
Contributor Author

shuaich commented Jan 17, 2025

#9849 to merge to branch 3.2.

@shuaich shuaich mentioned this pull request Jan 20, 2025
Merged
7 tasks
@patrick-stephens patrick-stephens added the ok-package-test Run PR packaging tests label Feb 27, 2025
@patrick-stephens
Copy link
Contributor

patrick-stephens commented Feb 27, 2025

We need a docs PR too

@edsiper edsiper merged commit 69b1ead into fluent:master Feb 28, 2025
115 of 125 checks passed
This was referenced Mar 1, 2025
Closed
Merged
@shuaich
Copy link
Contributor Author

shuaich commented Mar 1, 2025

Documentation PR: fluent/fluent-bit-docs#1577

@BrewTestBot BrewTestBot mentioned this pull request Apr 1, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@leonardo-albertovich leonardo-albertovich Awaiting requested review from leonardo-albertovich

@fujimotos fujimotos Awaiting requested review from fujimotos

@koleini koleini Awaiting requested review from koleini

@edsiper edsiper Awaiting requested review from edsiper edsiper is a code owner

Assignees

No one assigned

Labels

docs-required ok-package-test Run PR packaging tests

Projects

None yet

Milestone

Fluent Bit v4.0.0

Development

Successfully merging this pull request may close these issues.

4 participants

@shuaich @JeffLuoo @patrick-stephens @edsiper