Skip to content

Feature: introduce the HDFS plugin into RocksDB to support accessing data on HDFS #2361

@empiredan

Description

@empiredan

Starting from RocksDB 7.0.0, the HDFS support was removed from the main repository and provided instead as a plugin. Since Pegasus currently uses RocksDB 8.5.3, the HDFS plugin needs to be integrated into RocksDB to achieve the following goals:

  1. Support maintaining the underlying RocksDB data on HDFS in the Pegasus server, as well as accessing HDFS directly through the RocksDB interfaces.
  2. Support generating RocksDB SST files on HDFS via the Java APIs, so that they can be ingested into the Pegasus server using the bulk load tool.

Task list:

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions