RisingWave

[amotl]

About

RisingWave is a stream processing and management platform designed to offer the simplest and most cost-effective way to process, analyze, and manage real-time event data — with built-in support for the Apache Iceberg™ open table format. It provides both a Postgres-compatible SQL interface and a DataFrame-style Python interface.

Resources

• https://github.com/risingwavelabs/risingwave
• https://www.risingwave.com/

[amotl]

@hlcianfagna made a start presenting CrateDB with RisingWave. Thanks!

• RisingWave: Tutorial about Apache Iceberg -> CrateDB cratedb-guide#207

[amotl]

@hlcianfagna also tried the native way to connect external targets, per CREATE SINK operation. However, this failed.

• The PostgreSQL connector hits Add support for the special regtype data type crate#17833
• The JDBC connector hits Ignore SET pkfield=excluded.pkfield in INSERT ON CONFLICT crate#17834

The negative outcome yielded a workaround in form of a Python processor program using the on_change event handler to call a function to write data to CrateDB.

@hlcianfagna says:

The procedure using on_change works, but with a few considerations due to issues on the RisingWave side.