-
Notifications
You must be signed in to change notification settings - Fork 499
2505: added documentation #2507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
486b2cf
cf375cb
aa2cdb3
e6ded09
1c0babe
d378f9e
c3765df
1b24e95
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| --- | ||
| title: Azure Blob Storage | ||
| sidebar_position: 5 | ||
| --- | ||
|
|
||
| # Azure Blob Storage | ||
|
|
||
| [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) (Azure Blob Storage) is a massively scalable and secure object storage for cloud-native workloads, archives, data lakes, HPC, and machine learning. | ||
|
|
||
| ## Install Azure FS Plugin Manually | ||
|
|
||
| Azure Blob Storage support is not included in the default Fluss distribution. To enable Azure Blob Storage support, you need to manually install the filesystem plugin into Fluss. | ||
|
|
||
| 1. **Prepare the plugin JAR**: | ||
|
|
||
| - Download the `fluss-fs-azure-$FLUSS_VERSION$.jar` from the [Maven Repository](https://repo1.maven.org/maven2/org/apache/fluss/fluss-fs-azure/$FLUSS_VERSION$/fluss-fs-azure-$FLUSS_VERSION$.jar). | ||
|
|
||
| 2. **Place the plugin**: Place the plugin JAR file in the `${FLUSS_HOME}/plugins/azure/` directory: | ||
| ```bash | ||
| mkdir -p ${FLUSS_HOME}/plugins/azure/ | ||
| cp fluss-fs-azure-$FLUSS_VERSION$.jar ${FLUSS_HOME}/plugins/azure/ | ||
| ``` | ||
|
|
||
| 3. Restart Fluss if the cluster is already running to ensure the new plugin is loaded. | ||
|
|
||
| ## Configurations setup | ||
|
|
||
| To enabled Azure Blob Storage as remote storage, there are some required configurations that must be added to Fluss' `server.yaml`: | ||
|
|
||
| ```yaml | ||
| # The dir that used to be as the remote storage of Fluss, use the Azure Data Lake Storage URI | ||
| remote.data.dir: abfs://flus@flussblob.dfs.core.windows.net/path | ||
| # the access key for the azure blob storage account | ||
| fs.azure.account.key: 09a295d5-3da5-4435-a660-f438b331ade8 | ||
| # The oauth account provider type for Token-based Authentication | ||
| fs.azure.account.oauth.provider.type: org.apache.fluss.fs.abfs.token.DynamicTemporaryAzureCredentialsProvider | ||
gkatzioura marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| # The oauth2 client id for Token-based Authentication | ||
| fs.azure.account.oauth2.client.id: ed953f8a-d5e9-481c-b355-62794f178f66 | ||
| # The oauth2 client secret for Token-based Authentication | ||
| fs.azure.account.oauth2.client.secret: ec29f904-64f6-4372-831a-dc28ec818683 | ||
| # The oauth2 endpoint to generate access tokens for Token-based Authentication | ||
| fs.azure.account.oauth2.client.endpoint: https://login.microsoftonline.com/154b1d91-2d07-4e3a-beb6-9261ab4926ab/oauth2/token | ||
|
Comment on lines
+38
to
+42
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are the tokens real? Would be better to use some mock tokens to avoid security leak.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They are mock generated uuids |
||
| ``` | ||
Uh oh!
There was an error while loading. Please reload this page.