2024-01-04 kernel meeting notes

[zachschuermann]

summary

Happy new year! Lightweight check-in, still many missing from break. Will pick up next week!

action items

from last-last time

• Nick design doc
• new hackathon/FFI code review
• kernel write draft review

attendees

@tdas @roeap @zachschuermann @ryan-johnson-databricks @MrPowers

notes stream

@MrPowers

• goal: 10x number of users of delta lake. proxy metric for this is dowloads.
• chatted with clickhouse. we have a prototype C++ FFI, definitely eager to jump on integration next quarter.
  • note: we had the call with them and then they were concerned that WE were the ones going to do optimization. This is not the case, we obviously want to allow for all of their optimization. Learning: need a better pitch/better docs to make our intentions/etc. clear.
• docs: delta-rs docs
• delta-rs API (Matthew/Robert/Niko talking about ReplaceWhere)
• Matthew working towards 1.0 delta-rs
• Prioritize API stability
• Ask: quick API reviews from Ryan/TD. Also get Bart/Serge involved.
• Bizarre unusable API: some of the delta rust apis (DNF low usability)