[Feature] Add filesystem support for Microsoft Azure Storage

[qining-mj]

Purpose

Adding support for Microsoft Azure Storage
Linked issue: close #xxx

Tests

API and Format

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java?tabs=powershell%2Cmanaged-identity%2Croles-azure-portal%2Csign-in-azure-cli&pivots=blob-storage-quickstart-scratch

Documentation

This is a new feature.

A catalog is created in a similar way the S3 catalog is being created

CREATE CATALOG my_azure_catalog WITH (
'type' = 'paimon',
'fs.azure.account.key.ACCOUNT.blob.core.windows.net' = 'secret key',
'warehouse' = 'wasb://test@ACCOUNT.blob.core.windows.net/',
);

[qining-mj]

[JingsongLi]

+1

[JingsongLi]

@qining-mj Can you also update documentation?

[qining-mj]

@JingsongLi Sure, I can update the documentation as well. I'll make sure to keep it in sync with the code changes.

[qining-mj]

@JingsongLi Since this PR is focused on the code changes, could you consider merging this PR first? I'll create a separate PR for the documentation updates afterward. This way, we can ensure the code is integrated promptly, and I can focus on the documentation without blocking the current changes.

[JingsongLi]

@JingsongLi Since this PR is focused on the code changes, could you consider merging this PR first? I'll create a separate PR for the documentation updates afterward. This way, we can ensure the code is integrated promptly, and I can focus on the documentation without blocking the current changes.

You can just add documentation in this PR, if the doc is not so complex, doc can be merged into codes changing PR.

[qining-mj]

Azure

Download [paimon-Azure-< version >.jar.

Flink

If you have already configured Azure access through Spark (Via Hadoop FileSystem), here you can skip the following configuration.

Put [paimon-azure-< version >.jar into lib directory of your Flink home, and create catalog:

CREATE CATALOG my_catalog WITH (
   'type' = 'paimon',
   'warehouse' = 'wasb://,<container>@<account>.blob.core.windows.net/<path>',
   'fs.azure.account.key.Account.blob.core.windows.net' = 'yyy'
);

Spark

If you have already configured Azure access through Spark (Via Hadoop FileSystem), here you can skip the following configuration.

Place paimon-azure-{{< version >}}.jar together with paimon-spark-{{< version >}}.jar under Spark's jars directory, and start like

spark-sql \ 
  --conf spark.sql.catalog.paimon=org.apache.paimon.spark.SparkCatalog \
  --conf spark.sql.catalog.paimon.warehouse=wasb://,<container>@<account>.blob.core.windows.net/<path> \
  --conf fs.azure.account.key.Account.blob.core.windows.net=yyy \

[qining-mj]

@JingsongLi I have made preliminary updates to the documentation and plan to continue refining it. Thank you for your guidance。
I will continue to update and improve the document in the future.

[JingsongLi]

You can modify filesystems.md directly.

[qining-mj]

@JingsongLi Azure documentation has been added to filesystems.md，Thank you for your guidance。