Skip to content

feat(libsinsp): add multivalue transformer support#2776

Merged
poiana merged 8 commits intofalcosecurity:masterfrom
therealbobo:therealbobo-multivalue-transformers
Feb 3, 2026
Merged

feat(libsinsp): add multivalue transformer support#2776
poiana merged 8 commits intofalcosecurity:masterfrom
therealbobo:therealbobo-multivalue-transformers

Conversation

@therealbobo
Copy link
Contributor

@therealbobo therealbobo commented Jan 8, 2026

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind test

/kind feature

/kind sync

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:

Refactor filter transformers to use interfaces

  1. Parser Changes (filter/parser.cpp, filter/parser.h):
  • Extended the parser to support function-call syntax for transformers: transformer(arg1, arg2, ...)
  • Transformers can now take multiple arguments, not just a single value
  1. AST Changes (filter/ast.h, filter/ast.cpp):
  • Added new transformer_list_expr AST node for tuple/list arguments like (fd.name, fd.directory)
  • Changed field_transformer_expr to hold a vector of values instead of a single value
  • Added visitor methods for the new expression type
  1. Transformer Interface (sinsp_filter_transformer.h, sinsp_filter_transformer.cpp):
  • Added interface for querying transformer properties
  • Added transformer_str_to_type() and filter_transformer_type_str() for type conversion

Add multivalue transformer support

  1. New Classes (sinsp_filtercheck_multivalue_transformer.h/.cpp):
  • sinsp_filter_multivalue_transformer: Base class for multivalue transformers
    • Holds multiple sinsp_filter_check arguments
    • Provides argument_types() for compile-time type validation
    • Provides result_type() to describe output type
  • multivalue_transformer_filter_check: Wrapper that makes multivalue transformers usable as regular filter checks
  • sinsp_filter_multivalue_transformer_join: First multivalue transformer implementation (joins a list with a separator)
  1. New Helper Classes (sinsp_filtercheck_rawstring.h/.cpp):
  • rawstring_check: Filter check that returns a raw string literal
  • rawnum_check: Filter check that returns a numeric literal (delegates to appropriate type-specific filter check)
  • list_check: Filter check that returns a list of string literals
  1. Compiler Changes (filter.cpp, filter.h):
  • Added sinsp_extractor_compiler: New compiler for compiling field/transformer expressions (not full filters)
  • Handles compilation of transformer_list_expr nodes into multivalue transformers
  • Integrates with both filter compilation and event formatting
  1. Event Formatter (eventformatter.cpp, eventformatter.h):
  • Added formatter_visitor to properly handle transformer expressions in format strings
  • Uses sinsp_extractor_compiler for compiling field expressions

Which issue(s) this PR fixes:

Fixes #1925

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@github-actions
Copy link

github-actions bot commented Jan 8, 2026

Perf diff from master - unit tests

    12.54%     +1.76%  [.] std::__shared_count<(__gnu_cxx::_Lock_policy)2>::_M_get_use_count() const
     5.19%     -0.69%  [.] thread_group_info::get_first_thread() const
     9.88%     -0.61%  [.] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_add_ref_lock_nothrow()
     9.92%     -0.54%  [.] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
    12.06%     -0.44%  [.] sinsp_threadinfo::get_main_thread()
     3.56%     +0.27%  [.] sinsp_thread_manager::create_thread_dependencies(std::shared_ptr<sinsp_threadinfo> const&)
     6.51%     +0.24%  [.] std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count(std::__weak_count<(__gnu_cxx::_Lock_policy)2> const&, std::nothrow_t)
    15.35%     +0.21%  [.] std::__shared_ptr<sinsp_threadinfo, (__gnu_cxx::_Lock_policy)2>::__shared_ptr(std::__weak_ptr<sinsp_threadinfo, (__gnu_cxx::_Lock_policy)2> const&, std::nothrow_t)
     0.32%     -0.17%  [.] void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag)
     9.14%     +0.16%  [.] sinsp_threadinfo::update_main_fdtable()

Heap diff from master - unit tests

peak heap memory consumption: -8.16K
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: -20.35K
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            -0.0264         -0.0265           239           232           239           232
BM_sinsp_split_median                                          -0.0247         -0.0245           238           232           238           232
BM_sinsp_split_stddev                                          -0.3902         -0.3934             2             1             2             1
BM_sinsp_split_cv                                              -0.3737         -0.3768             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  -0.0516         -0.0516            72            69            72            69
BM_sinsp_concatenate_paths_relative_path_median                -0.0369         -0.0371            72            69            72            69
BM_sinsp_concatenate_paths_relative_path_stddev                -0.2222         -0.2271             1             1             1             1
BM_sinsp_concatenate_paths_relative_path_cv                    -0.1799         -0.1851             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     +0.1089         +0.1093            40            44            40            44
BM_sinsp_concatenate_paths_empty_path_median                   +0.1157         +0.1160            39            44            39            44
BM_sinsp_concatenate_paths_empty_path_stddev                   -0.9591         -0.9601             1             0             1             0
BM_sinsp_concatenate_paths_empty_path_cv                       -0.9631         -0.9640             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.0020         +0.0020            68            69            68            69
BM_sinsp_concatenate_paths_absolute_path_median                -0.0002         -0.0002            68            68            68            68
BM_sinsp_concatenate_paths_absolute_path_stddev                +0.6050         +0.6078             0             1             0             1
BM_sinsp_concatenate_paths_absolute_path_cv                    +0.6018         +0.6046             0             0             0             0

@therealbobo therealbobo force-pushed the therealbobo-multivalue-transformers branch from 1b38b80 to 0f77b85 Compare January 8, 2026 20:34
@codecov
Copy link

codecov bot commented Jan 8, 2026

Codecov Report

❌ Patch coverage is 78.12971% with 145 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.52%. Comparing base (7250ae9) to head (44dc78c).
⚠️ Report is 11 commits behind head on master.

Files with missing lines Patch % Lines
userspace/libsinsp/filter.cpp 61.47% 47 Missing ⚠️
userspace/libsinsp/eventformatter.cpp 68.65% 21 Missing ⚠️
userspace/libsinsp/filter/ast.cpp 41.66% 21 Missing ⚠️
...sinsp/sinsp_filtercheck_multivalue_transformer.cpp 73.07% 21 Missing ⚠️
userspace/libsinsp/sinsp_filtercheck_rawstring.cpp 82.14% 10 Missing ⚠️
userspace/libsinsp/test/filter_transformer.ut.cpp 91.20% 8 Missing ⚠️
userspace/libsinsp/filter/ast.h 72.00% 7 Missing ⚠️
...p_filter_transformers/sinsp_filter_transformer.cpp 0.00% 4 Missing ⚠️
userspace/libsinsp/filter/ppm_codes.cpp 57.14% 3 Missing ⚠️
userspace/libsinsp/filter/parser.cpp 96.77% 2 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2776      +/-   ##
==========================================
- Coverage   74.53%   74.52%   -0.01%     
==========================================
  Files         292      294       +2     
  Lines       29987    30541     +554     
  Branches     4660     4757      +97     
==========================================
+ Hits        22350    22762     +412     
- Misses       7637     7779     +142     
Flag Coverage Δ
libsinsp 74.52% <78.12%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@therealbobo therealbobo force-pushed the therealbobo-multivalue-transformers branch 2 times, most recently from 0323726 to 7c68537 Compare January 12, 2026 12:13
@therealbobo therealbobo marked this pull request as ready for review January 12, 2026 12:13
@poiana poiana requested a review from terror96 January 12, 2026 12:13
ekoops
ekoops previously approved these changes Jan 16, 2026
Copy link
Contributor

@ekoops ekoops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana
Copy link
Contributor

poiana commented Jan 16, 2026

LGTM label has been added.

DetailsGit tree hash: 02ddb028b028fba91545cd8982af7adecb01e972

@ekoops ekoops added this to the 0.24.0 milestone Jan 16, 2026
@poiana poiana removed the lgtm label Jan 19, 2026
@poiana poiana requested a review from ekoops January 19, 2026 14:42
@therealbobo therealbobo force-pushed the therealbobo-multivalue-transformers branch 2 times, most recently from 442d99a to c17e840 Compare January 19, 2026 15:10
@therealbobo therealbobo requested a review from irozzo-1A January 19, 2026 15:23
ekoops
ekoops previously approved these changes Jan 29, 2026
Copy link
Contributor

@ekoops ekoops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana poiana added the lgtm label Jan 29, 2026
@poiana
Copy link
Contributor

poiana commented Jan 29, 2026

LGTM label has been added.

DetailsGit tree hash: 9f4a35ae055a555a436a16741f3bccef02f7f5b0

Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
…former

Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
Signed-off-by: Roberto Scolaro <roberto.scolaro21@gmail.com>
@therealbobo therealbobo force-pushed the therealbobo-multivalue-transformers branch from c17e840 to 44dc78c Compare February 1, 2026 11:52
@poiana poiana removed the lgtm label Feb 1, 2026
@poiana poiana requested a review from ekoops February 1, 2026 11:52
@poiana poiana added the lgtm label Feb 2, 2026
@poiana
Copy link
Contributor

poiana commented Feb 2, 2026

LGTM label has been added.

DetailsGit tree hash: c235f5e8c38a5514a36bbf1e83587ac595a15e01

@leogr
Copy link
Member

leogr commented Feb 2, 2026

cc @falcosecurity/libs-maintainers

@leogr leogr requested a review from a team February 2, 2026 14:22
Copy link
Contributor

@irozzo-1A irozzo-1A left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@github-project-automation github-project-automation bot moved this from Todo to In progress in Falco Roadmap Feb 3, 2026
@poiana
Copy link
Contributor

poiana commented Feb 3, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ekoops, irozzo-1A, leogr, therealbobo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [ekoops,irozzo-1A,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana merged commit 64da2bd into falcosecurity:master Feb 3, 2026
48 checks passed
@github-project-automation github-project-automation bot moved this from In progress to Done in Falco Roadmap Feb 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

New transformer: join(<list>, <sep>)

5 participants