Starting from RocksDB 7.0.0, the HDFS support was removed from the main repository and provided instead as a plugin. Since Pegasus currently uses RocksDB 8.5.3, the HDFS plugin needs to be integrated into RocksDB to achieve the following goals:
- Support maintaining the underlying RocksDB data on HDFS in the Pegasus server, as well as accessing HDFS directly through the RocksDB interfaces.
- Support generating RocksDB SST files on HDFS via the Java APIs, so that they can be ingested into the Pegasus server using the bulk load tool.
Task list: