Skip to content

refactor(query): refactor the implementation of join cardinality estimation #18501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

refactor(query): refactor the implementation of join cardinality estimation #18501

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
543d207
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
b6b507b
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
d10c3ab
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
d290889
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
037755a
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
0fdf17b
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
d9e3306
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 8, 2025
d00327f
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 9, 2025
273fed8
refactor(query): refactor the implementation of join cardinality esti…
zhang2014 Aug 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions ...planner/optimizer/data/results/obfuscated/01_multi_join_avg_case_expression_optimized.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,28 @@ Exchange(Merge)
├── probe keys: [a.a0m (#9)]
├── other filters: []
├── Join(Inner)
│ ├── build keys: [b.a0k (#48), b.a0n (#50)]
│ ├── probe keys: [a.a0k (#7), a.a0n (#10)]
│ ├── other filters: [lte(b.a2c (#52), a.a0d (#0)), gt(b.a2k (#61), a.a0d (#0))]
│ ├── build keys: [d.a5t (#151)]
│ ├── probe keys: [a.a0l (#8)]
│ ├── other filters: []
│ ├── Join(Inner)
│ │ ├── build keys: [d.a5t (#151)]
│ │ ├── probe keys: [a.a0l (#8)]
│ │ ├── other filters: []
│ │ ├── build keys: [b.a0k (#48), b.a0n (#50)]
│ │ ├── probe keys: [a.a0k (#7), a.a0n (#10)]
│ │ ├── other filters: [lte(b.a2c (#52), a.a0d (#0)), gt(b.a2k (#61), a.a0d (#0))]
│ │ ├── Scan
│ │ │ ├── table: default.a0c (#0)
│ │ │ ├── filters: [gte(a0c.a0d (#0), '20240526'), lte(a0c.a0d (#0), '20250525')]
│ │ │ ├── order by: []
│ │ │ └── limit: NONE
│ │ └── Exchange(Broadcast)
│ │ └── Scan
│ │ ├── table: default.a5r (#3)
│ │ ├── filters: [eq(substring(a5r.a5w (#156), 1, 1), '1')]
│ │ ├── table: default.a1z (#1)
│ │ ├── filters: [eq(a1z.a2t (#70), '624100')]
│ │ ├── order by: []
│ │ └── limit: NONE
│ └── Exchange(Broadcast)
│ └── Scan
│ ├── table: default.a1z (#1)
│ ├── filters: [eq(a1z.a2t (#70), '624100')]
│ ├── table: default.a5r (#3)
│ ├── filters: [eq(substring(a5r.a5w (#156), 1, 1), '1')]
│ ├── order by: []
│ └── limit: NONE
└── Exchange(Broadcast)
Expand Down
62 changes: 31 additions & 31 deletions .../planner/optimizer/data/results/obfuscated/01_multi_join_avg_case_expression_physical.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,27 @@ Exchange
└── EvalScalar
├── output columns: [sell_mnt = 0 (#170)]
├── expressions: [t.sell_mnt (#169) = 0]
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
└── EvalScalar
├── output columns: [sell_mnt (#169)]
├── expressions: [sum(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#167) / CAST(if(CAST(count(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#168) = 0 AS Boolean NULL), 1, count(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#168)) AS UInt64 NULL) + 3]
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
└── AggregateFinal
├── output columns: [sum(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#167), count(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#168), a.a0d (#0), a.a0k (#7), a.a0m (#9), c.a5m (#144)]
├── group by: [a0d, a0k, a0m, a5m]
├── aggregate functions: [sum(sum_arg_0), count()]
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
└── Exchange
├── output columns: [sum(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#167), count(CASE WHEN d.a1v = '603020' THEN 1 ELSE 0 END) (#168), a.a0d (#0), a.a0k (#7), a.a0m (#9), c.a5m (#144)]
├── exchange type: Hash(0, 1, 2, 3)
└── AggregatePartial
├── group by: [a0d, a0k, a0m, a5m]
├── aggregate functions: [sum(sum_arg_0), count()]
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
└── EvalScalar
├── output columns: [a.a0d (#0), a.a0k (#7), a.a0m (#9), c.a5m (#144), sum_arg_0 (#166)]
├── expressions: [if(d.a1v (#154) = '603020', 1, 0)]
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
└── HashJoin
├── output columns: [a.a0d (#0), a.a0k (#7), a.a0m (#9), d.a1v (#154), c.a5m (#144)]
├── join type: INNER
Expand All @@ -34,7 +34,7 @@ Exchange
├── filters: []
├── build join filters:
│ └── filter id:3, build key:c.a0m (#149), probe key:a.a0m (#9), filter type:inlist,min_max
├── estimated rows: 7119376617326129446912.00
├── estimated rows: 889922077165.77
├── Exchange(Build)
│ ├── output columns: [c.a5m (#144), a0m (#149)]
│ ├── exchange type: Broadcast
Expand All @@ -54,48 +54,48 @@ Exchange
└── HashJoin(Probe)
├── output columns: [a.a0d (#0), a.a0k (#7), a.a0m (#9), d.a1v (#154)]
├── join type: INNER
├── build keys: [b.a0k (#48), b.a0n (#50)]
├── probe keys: [a.a0k (#7), a.a0n (#10)]
├── keys is null equal: [false, false]
├── filters: [b.a2c (#52) <= a.a0d (#0), b.a2k (#61) > a.a0d (#0)]
├── build keys: [d.a5t (#151)]
├── probe keys: [a.a0l (#8)]
├── keys is null equal: [false]
├── filters: []
├── build join filters:
│ ├── filter id:1, build key:b.a0k (#48), probe key:a.a0k (#7), filter type:inlist,min_max
│ └── filter id:2, build key:b.a0n (#50), probe key:a.a0n (#10), filter type:inlist,min_max
├── estimated rows: 111635169056257280.00
│ └── filter id:2, build key:d.a5t (#151), probe key:a.a0l (#8), filter type:inlist,min_max
├── estimated rows: 2790879226.41
├── Exchange(Build)
│ ├── output columns: [b.a0k (#48), b.a0n (#50), b.a2c (#52), b.a2k (#61)]
│ ├── output columns: [d.a5t (#151), d.a1v (#154)]
│ ├── exchange type: Broadcast
│ └── TableScan
│ ├── table: default.default.a1z
│ ├── output columns: [a0k (#48), a0n (#50), a2c (#52), a2k (#61)]
│ ├── table: default.default.a5r
│ ├── output columns: [a5t (#151), a1v (#154)]
│ ├── read rows: 0
│ ├── read size: 0
│ ├── partitions total: 0
│ ├── partitions scanned: 0
│ ├── push downs: [filters: [is_true(a1z.a2t (#70) = '624100')], limit: NONE]
│ └── estimated rows: 45493.85
│ ├── push downs: [filters: [is_true(substr(a5r.a5w (#156), 1, 1) = '1')], limit: NONE]
│ └── estimated rows: 806.60
└── HashJoin(Probe)
├── output columns: [a.a0d (#0), a.a0k (#7), a.a0m (#9), a.a0n (#10), d.a1v (#154)]
├── output columns: [a.a0d (#0), a.a0k (#7), a.a0l (#8), a.a0m (#9)]
├── join type: INNER
├── build keys: [d.a5t (#151)]
├── probe keys: [a.a0l (#8)]
├── keys is null equal: [false]
├── filters: []
├── build keys: [b.a0k (#48), b.a0n (#50)]
├── probe keys: [a.a0k (#7), a.a0n (#10)]
├── keys is null equal: [false, false]
├── filters: [b.a2c (#52) <= a.a0d (#0), b.a2k (#61) > a.a0d (#0)]
├── build join filters:
│ └── filter id:0, build key:d.a5t (#151), probe key:a.a0l (#8), filter type:inlist,min_max
├── estimated rows: 2453851765646.43
│ ├── filter id:0, build key:b.a0k (#48), probe key:a.a0k (#7), filter type:inlist,min_max
│ └── filter id:1, build key:b.a0n (#50), probe key:a.a0n (#10), filter type:inlist,min_max
├── estimated rows: 692010718.18
├── Exchange(Build)
│ ├── output columns: [d.a5t (#151), d.a1v (#154)]
│ ├── output columns: [b.a0k (#48), b.a0n (#50), b.a2c (#52), b.a2k (#61)]
│ ├── exchange type: Broadcast
│ └── TableScan
│ ├── table: default.default.a5r
│ ├── output columns: [a5t (#151), a1v (#154)]
│ ├── table: default.default.a1z
│ ├── output columns: [a0k (#48), a0n (#50), a2c (#52), a2k (#61)]
│ ├── read rows: 0
│ ├── read size: 0
│ ├── partitions total: 0
│ ├── partitions scanned: 0
│ ├── push downs: [filters: [is_true(substr(a5r.a5w (#156), 1, 1) = '1')], limit: NONE]
│ └── estimated rows: 806.60
│ ├── push downs: [filters: [is_true(a1z.a2t (#70) = '624100')], limit: NONE]
│ └── estimated rows: 45493.85
└── TableScan(Probe)
├── table: default.default.a0c
├── output columns: [a0d (#0), a0k (#7), a0l (#8), a0m (#9), a0n (#10)]
Expand All @@ -104,6 +104,6 @@ Exchange
├── partitions total: 0
├── partitions scanned: 0
├── push downs: [filters: [and_filters(a0c.a0d (#0) >= '20240526', a0c.a0d (#0) <= '20250525')], limit: NONE]
├── apply join filters: [#3, #1, #2, #0]
├── apply join filters: [#3, #2, #0, #1]
└── estimated rows: 3042216421.58

20 changes: 10 additions & 10 deletions ...planner/optimizer/data/results/obfuscated/01_multi_join_sum_case_expression_optimized.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,28 +24,28 @@ Exchange(Merge)
│ └── EvalScalar
│ ├── scalars: [a.a0d (#0) AS (#0), a.a0k (#7) AS (#7), a.a0m (#9) AS (#9), if(eq(d.a1v (#154), '603020'), 1, 0) AS (#166), b.a2t (#70) AS (#169), d.a5w (#156) AS (#171)]
│ └── Join(Inner)
│ ├── build keys: [b.a0k (#48), b.a0n (#50)]
│ ├── probe keys: [a.a0k (#7), a.a0n (#10)]
│ ├── other filters: [lte(b.a2c (#52), a.a0d (#0)), gt(b.a2k (#61), a.a0d (#0))]
│ ├── build keys: [d.a5t (#151)]
│ ├── probe keys: [a.a0l (#8)]
│ ├── other filters: []
│ ├── Join(Inner)
│ │ ├── build keys: [d.a5t (#151)]
│ │ ├── probe keys: [a.a0l (#8)]
│ │ ├── other filters: []
│ │ ├── build keys: [b.a0k (#48), b.a0n (#50)]
│ │ ├── probe keys: [a.a0k (#7), a.a0n (#10)]
│ │ ├── other filters: [lte(b.a2c (#52), a.a0d (#0)), gt(b.a2k (#61), a.a0d (#0))]
│ │ ├── Scan
│ │ │ ├── table: default.a0c (#0)
│ │ │ ├── filters: [gte(a0c.a0d (#0), '20240526'), lte(a0c.a0d (#0), '20250525')]
│ │ │ ├── order by: []
│ │ │ └── limit: NONE
│ │ └── Exchange(Broadcast)
│ │ └── Scan
│ │ ├── table: default.a5r (#3)
│ │ ├── filters: [eq(substring(a5r.a5w (#156), 1, 1), '1')]
│ │ ├── table: default.a1z (#1)
│ │ ├── filters: [eq(a1z.a2t (#70), '624100')]
│ │ ├── order by: []
│ │ └── limit: NONE
│ └── Exchange(Broadcast)
│ └── Scan
│ ├── table: default.a1z (#1)
│ ├── filters: [eq(a1z.a2t (#70), '624100')]
│ ├── table: default.a5r (#3)
│ ├── filters: [eq(substring(a5r.a5w (#156), 1, 1), '1')]
│ ├── order by: []
│ └── limit: NONE
└── Exchange(Broadcast)
Expand Down
Loading
Loading