Skip to content

Account-scoped ADLS properties from Iceberg/Polaris are silently ignored #6357

@firecast

Description

@firecast

Problem

When using Polaris catalog with ADLS vended credentials, PyIceberg returns account-scoped property keys like adls.sas-token.<account>.dfs.core.windows.net instead of plain adls.sas-token. Daft's _convert_iceberg_file_io_properties_to_io_config only does exact key matching, so these credentials are silently ignored, causing authentication failures.

Expected Behavior

Daft should recognize account-scoped ADLS property keys (e.g. adls.sas-token.<account>.dfs.core.windows.net, adls.account-key.<account>.dfs.core.windows.net) and extract both the credential value and the account name from the key suffix.

Reproduction

Configure a Polaris catalog with ADLS storage. When loading an Iceberg table, the file IO properties will contain scoped keys like:

{
    "adls.sas-token.myaccount.dfs.core.windows.net": "<token>"
}

Daft fails to authenticate because it doesn't match these scoped keys.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions