HDDS-9858. Add comprehensive user guide documentation.

cspell.yaml

@@ -69,6 +69,7 @@ words:
 - ASF
 - PMC
 # Ozone specific words
+- HCFS
 - HDDS
 - ratis
 - OM
@@ -125,6 +126,27 @@ words:
 - UX
 - devs
 - CLI
+- rebalancing
+- frontmatter
+- backporting
+- Metastore
+- Kerberized
+- Ambari
+- Lakehouse
 - lakehouse
+- Acyclical
+- CDP
+- RDD
+- classpaths
+- ajv
+- keytab
+- keytabs
+- Boto
+- Tez
 - Flink
-- rebalancing
+- configmap
+- df
+- INPATH
+- ozonecluster
+- pyspark
+- setacl

docs/04-user-guide/01-client-interfaces/01-o3.md

@@ -1,7 +1,225 @@
 ---
-sidebar_label: o3 (Ozone Shell)
+sidebar_label: Ozone Shell
 ---
 
-# o3: Ozone Shell
+# Ozone Shell (`ozone sh`)
 
-**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section.
+The Ozone Shell, invoked via the `ozone sh` command, is a primary command-line interface (CLI) for interacting with Apache Ozone. It allows administrators and users to manage resources and perform basic data operations directly against the Ozone Manager (OM) using Ozone's native RPC protocol.
+
+## Overview
+
+The Ozone Shell provides a structured way to manage the Ozone namespace and related entities. It's essential for administrative tasks and useful for basic data interaction and scripting.
+
+**Key Capabilities:**
+
+- **Volume Management:** Create, delete, list volumes; get volume information; set/clear owner and quotas (space and namespace).
+- **Bucket Management:** Create, delete, list buckets; get bucket information; set/clear properties like versioning, replication, layout, encryption key, owner, and quotas; link buckets.
+- **Key (Object) Management:** Put (upload), get (download), delete, list keys; get key information; manage multipart uploads (though less common via shell).
+- **ACL Management:** Set, get, add, and remove Access Control Lists (ACLs) for volumes, buckets, keys, and prefixes.
+- **Snapshot Management:** Create, delete, list, rename snapshots; get snapshot info; manage snapshot diffs.
+- **S3 Credentials Management:** Manage S3 access keys and secrets (primarily for multi-tenancy).
+- **Administrative Actions:** Perform cluster-level administrative tasks (e.g., decommissioning, safemode management - though often under `ozone admin`).
+
+## Syntax
+
+The general syntax follows a consistent pattern:
+
+```bash
+ozone sh <resource> <action> [options] <path_or_arguments>
+```
+
+- `<resource>`: The type of Ozone entity you are interacting with (e.g., `volume`, `bucket`, `key`, `snapshot`, `s3`, `prefix`).
+- `<action>`: The specific operation to perform on the resource (e.g., `create`, `list`, `info`, `delete`, `put`, `get`, `setacl`, `setquota`).
+- `[options]`: Command-specific flags that modify the action's behavior (e.g., `--quota`, `--layout`, `--acl`, `--type`, `--replication`, `-k`, `-r`).
+- `<path_or_arguments>`: The Ozone URI path to the target resource (e.g., `/volumeName`, `/volumeName/bucketName`, `/volumeName/bucketName/keyName`) or other required arguments depending on the command.
+
+**Getting Help:**
+
+You can get help for any command level:
+
+```bash
+# Help for all shell commands
+ozone sh --help
+
+# Help for volume commands
+ozone sh volume --help
+
+# Help for creating a volume
+ozone sh volume create --help
+```
+
+## Common Commands and Examples
+
+Here are examples grouped by resource type, drawing from common use cases and test scenarios.
+
+### Volume Operations
+
+```bash
+# Create a volume
+ozone sh volume create /data-vol
+
+# Create a volume with quotas
+ozone sh volume create --space-quota 5GB --namespace-quota 10000 /quota-vol
+
+# List volumes
+ozone sh volume list /
+# Example Output:
+# [ {
+#   "metadata" : { },
+#   "name" : "data-vol",
+#   "admin" : "hadoop",
+#   "owner" : "hadoop",
+#   "creationTime" : "2024-04-11T18:00:00.000Z",
+#   "modificationTime" : "2024-04-11T18:00:00.000Z",
+#   "quotaInBytes" : -1,
+#   "quotaInNamespace" : -1,
+#   "usedNamespace" : 0
+# }, ... ]
+
+# Get volume information
+ozone sh volume info /data-vol
+
+# Set volume quota
+ozone sh volume setquota --space-quota 10GB /data-vol
+
+# Clear volume namespace quota
+ozone sh volume clrquota --namespace-quota /data-vol
+
+# Delete an empty volume
+ozone sh volume delete /data-vol
+
+# Recursively delete a volume and its contents (USE WITH CAUTION!)
+ozone sh volume delete -r /quota-vol
+```
+
+### Bucket Operations
+
+```bash
+# Create a bucket (defaults: OBS layout, RATIS/THREE replication)
+ozone sh bucket create /data-vol/raw-data
+
+# Create an FSO bucket
+ozone sh bucket create --layout FILE_SYSTEM_OPTIMIZED /data-vol/fso-data
+
+# Create an EC bucket (requires pre-defined EC policy 'rs-6-3-1024k')
+ozone sh bucket create --replication rs-6-3-1024k --type EC /data-vol/ec-data
+
+# Create an encrypted bucket (requires KMS setup and key 'my-bek')
+ozone sh bucket create -k my-bek /data-vol/secure-data
+
+# List buckets in a volume
+ozone sh bucket list /data-vol
+
+# Get bucket information
+ozone sh bucket info /data-vol/raw-data
+
+# Set bucket quota
+ozone sh bucket setquota --namespace-quota 50000 /data-vol/raw-data
+
+# Set bucket replication config (change existing bucket to EC)
+ozone sh bucket set-replication-config /data-vol/raw-data --type EC --replication rs-3-2-1024k
+
+# Link an existing bucket into another path (e.g., S3 volume)
+ozone sh bucket link /data-vol/raw-data /s3v/linked-raw-data
+
+# Delete an empty bucket
+ozone sh bucket delete /data-vol/raw-data
+
+# Recursively delete a bucket (USE WITH CAUTION!)
+ozone sh bucket delete -r /data-vol/fso-data
+```
+
+### Key (Object) Operations
+
+```bash
+# Upload (put) a local file to Ozone
+# Note the argument order: <destination_ozone_path> <source_local_path>
+ozone sh key put /data-vol/raw-data/logs/app.log /local/path/app.log
+
+# Upload a file with specific EC replication
+ozone sh key put /data-vol/ec-data/archive.zip /local/path/archive.zip --type EC --replication rs-6-3-1024k
+
+# Download (get) a key from Ozone to a local directory
+ozone sh key get /data-vol/raw-data/logs/app.log /tmp/
+
+# Get key information (metadata)
+ozone sh key info /data-vol/raw-data/logs/app.log
+# Example Output:
+# {
+#   "volumeName" : "data-vol",
+#   "bucketName" : "raw-data",
+#   "name" : "logs/app.log",
+#   "dataSize" : 10240,
+#   "creationTime" : "2024-04-11T18:15:00.000Z",
+#   "modificationTime" : "2024-04-11T18:15:00.000Z",
+#   "replicationConfig" : {
+#     "replicationType" : "RATIS",
+#     "requiredNodes" : 3,
+#     "replication" : "THREE"
+#   },
+#   "ozoneKeyLocations" : [ ... ],
+#   "metadata" : { },
+#   "fileEncryptionInfo" : null,
+#   "updateID" : 12345,
+#   "file" : false
+# }
+
+# List keys within a bucket (or prefix)
+ozone sh key list /data-vol/raw-data/logs/
+
+# Delete a key
+ozone sh key delete /data-vol/raw-data/logs/app.log
+```
+
+### ACL Operations
+
+```bash
+# Set ACLs (replaces existing ACLs) on a bucket
+ozone sh bucket setacl /data-vol/secure-data --acl user:dev_user:rw,group:dev_group:r
+
+# Get ACLs of a bucket
+ozone sh bucket getacl /data-vol/secure-data
+
+# Add an ACL (appends to existing ACLs) to a volume
+ozone sh volume addacl /data-vol --acl user:test_user:rl
+
+# Remove a specific ACL permission from a key
+ozone sh key removeacl /data-vol/raw-data/logs/app.log --acl user:test_user:w
+
+# Add a default ACL to a bucket (applied to new keys created within)
+ozone sh bucket addacl /data-vol/fso-data --acl default:group:analytics:r
+```
+
+### Snapshot Operations
+
+```bash
+# Create a snapshot of a bucket
+ozone sh snapshot create /data-vol/raw-data my-snapshot-`date +%Y%m%d`
+
+# List snapshots for a bucket
+ozone sh snapshot list /data-vol/raw-data
+
+# Get info about a specific snapshot
+ozone sh snapshot info /data-vol/raw-data my-snapshot-20240411
+
+# Delete a snapshot
+ozone sh snapshot delete /data-vol/raw-data my-snapshot-20240411
+
+# Get differences between two snapshots
+ozone sh snapshot diff /data-vol/raw-data my-snapshot-20240410 my-snapshot-20240411
+```
+
+## When to Use
+
+The Ozone Shell (`ozone sh`) is the recommended tool for:
+
+- **Cluster Administrators:** Performing setup, configuration changes, managing volumes/buckets, setting quotas and ACLs, managing snapshots.
+- **Scripting:** Automating administrative or basic data management tasks.
+- **Basic Data Interaction:** Simple uploads, downloads, listings, and deletions for exploration or troubleshooting.
+- **Accessing Ozone-Specific Features:** Managing features not exposed via other interfaces (e.g., certain snapshot operations, detailed ACL management).
+
+It is generally **not** the ideal interface for:
+
+- High-performance, parallel data access within applications (use the [Java API](./06-java-client-api.md) or `ofs`/`s3a`).
+- Complex filesystem operations requiring strong atomicity guarantees (use [`ofs`](./02-ofs.md) with FSO buckets).
+- Interfacing with applications designed for the S3 API (use the [S3 Gateway](./03-s3.md) or [`s3a`](./04-s3a.md)).