Skip to content

feat(nifi): Add dependencies for ADLS and GCP  #942

@Jimvin

Description

@Jimvin

The Stackable docker image for Apache NiFi does not contain the Hadoop Azure ADLS or Google GCS libraries. Consequently NiFi cannot write to Hive/Iceberg tables stored on ADLS/ABFS or GCS. Trying to write to an Iceberg table using the PutIceberg processor produces the following error:

PutIceberg[id=5e413c4e-7d2f-1e5a-ffff-fffffe36b8d5] Failed to load table from catalog: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem not found
- Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem not found

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions