Skip to content

Commit 1d5b0de

Browse files
authored
parser_json: use JSON as fallback parser instead of Yajl for performance (#4813)
**Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: Recently, Ruby's json has incredible performance improvements. It might be faster than oj gem. So, I think json is a suitable as fallback. This is similar with #4759 Here is easily benchmark. (I used same log file in #4759) * Before * It spent 90.50963662 sec to handle 10 GB file * After * It spent 74.624230077 sec to handle 10 GB file * config ``` <source> @type tail path "#{File.expand_path '~/tmp/access*.log'}" pos_file "#{File.expand_path '~/tmp/fluentd/access.log.pos'}" tag log read_from_head true <parse> @type json </parse> </source> <match **> @type file path "#{File.expand_path '~/tmp/fluentd/log'}" </match> ``` FYI) * https://byroot.github.io/ruby/json/2024/12/15/optimizing-ruby-json-part-1.html * https://byroot.github.io/ruby/json/2024/12/18/optimizing-ruby-json-part-2.html * https://byroot.github.io/ruby/json/2024/12/27/optimizing-ruby-json-part-3.html * https://byroot.github.io/ruby/json/2024/12/29/optimizing-ruby-json-part-4.html * https://byroot.github.io/ruby/json/2025/01/04/optimizing-ruby-json-part-5.html * https://byroot.github.io/ruby/json/2025/01/12/optimizing-ruby-json-part-6.html * https://byroot.github.io/ruby/json/2025/01/14/optimizing-ruby-json-part-7.html **Docs Changes**: fluent/fluentd-docs-gitbook#560 **Release Note**: Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
1 parent 24f08be commit 1d5b0de

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

lib/fluent/plugin/parser_json.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,8 @@ def configure_json_parser(name)
5252
when :oj
5353
return [Oj.method(:load), Oj::ParseError] if Fluent::OjOptions.available?
5454

55-
log&.info "Oj is not installed, and failing back to Yajl for json parser"
56-
configure_json_parser(:yajl)
55+
log&.info "Oj is not installed, and failing back to JSON for json parser"
56+
configure_json_parser(:json)
5757
when :json then [JSON.method(:load), JSON::ParserError]
5858
when :yajl then [Yajl.method(:load), Yajl::ParseError]
5959
else

test/plugin/test_parser_json.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,12 +28,12 @@ def test_fall_back_oj_to_yajl_if_oj_not_available
2828

2929
result = @parser.instance.configure_json_parser(:oj)
3030

31-
assert_equal [Yajl.method(:load), Yajl::ParseError], result
31+
assert_equal [JSON.method(:load), JSON::ParserError], result
3232
logs = @parser.logs.collect do |log|
3333
log.gsub(/\A\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} [-+]\d{4} /, "")
3434
end
3535
assert_equal(
36-
["[info]: Oj is not installed, and failing back to Yajl for json parser\n"],
36+
["[info]: Oj is not installed, and failing back to JSON for json parser\n"],
3737
logs
3838
)
3939
end

0 commit comments

Comments
 (0)