Support for Change Data Feed (CDF)

[larssjesnendk]

I would like to request support for Change Data Feed (CDF).

Currently, it seems that CDF functionality is not available in this library, which limits my ability to efficiently capture and process incremental changes from Delta tables.

Use Case: CDF is critical for scenarios where downstream systems need to consume only the changes (inserts, updates, deletes) rather than scanning entire tables. This feature would enable:

• Real-time or near-real-time data pipelines.
• Efficient synchronization with external systems.
• Reduced compute and storage costs compared to full table scans.

Expected Behavior:

• Ability to read changes from Delta tables using CDF APIs.
• Support for specifying version ranges or timestamps for change queries.

[mightyshazam]

I believe this feature is in the kernel already, so I'll look into what it takes.

[larssjesnendk]

Sounds great @mightyshazam! would it be possible for you to provide a timeline? 😊

[mightyshazam]

I don't have a timeline at the moment. There are potential kernel changes that I have been waiting on before moving forward with this project, but I'll have to see if this feature is easy enough to add without those changes.