checkpoint

Author: kerneltime

docs/03-core-concepts/01-architecture.md

@@ -165,6 +165,7 @@ The typical write sequence follows these steps:
 1. **Namespace Operations**: The client contacts the Ozone Manager to create or locate the key in the namespace
 2. **Block Allocation**: The Ozone Manager requests blocks from the Storage Container Manager
 3. **Data Transfer**: The client directly writes data to the selected Datanodes according to the replication pipeline
+4. **Key Commit**: After successful data transfer, the client commits the key to the Ozone Manager
 
 #### Read Path Sequence
 

docs/03-core-concepts/01-namespace/01-volumes/01-overview.md

@@ -14,6 +14,29 @@ Volumes are the top-level entity in the Apache Ozone namespace hierarchy. They s
 - **Resource Management**: Volumes support quota enforcement for storage space and object counts
 - **Command Line Access**: Volumes can be created and managed through the Ozone shell (`ozone sh`)
 
+## Volume Name Limitations
+
+When creating volumes in Ozone, the following limitations apply to volume names:
+
+- **Length**: Volume names must be between 3 and 63 characters long
+- **Allowed Characters**: Volume names can only contain:
+  - Lowercase letters (a-z)
+  - Numbers (0-9)
+  - Hyphens (-)
+  - Periods (.)
+  - Underscore (_) when not in S3-strict mode
+- **Formatting Rules**:
+  - Cannot start with a period or dash
+  - Cannot end with a period or dash
+  - Cannot contain two consecutive periods
+  - Cannot have a period following a dash
+  - Cannot have a dash following a period
+  - Cannot contain uppercase letters
+  - Cannot be an IPv4 address format or consist of only numbers
+  - Cannot contain other special characters
+
+These limitations ensure that volume names are compatible with DNS naming conventions and maintain consistency across the Ozone ecosystem.
+
 
 ## Use Cases for Volumes
 
@@ -31,5 +54,5 @@ Volumes can be accessed through multiple Ozone interfaces:
 
 - **Ozone Shell**: `ozone sh volume ...` commands
 - **Ozone File System (OFS)**: `ofs://om-service-id/volume/bucket/key`
-- **S3 Gateway**: Volume information is not directly exposed in the S3 protocol
+- **S3 Gateway**: Volume information is not directly exposed in the S3 protocol. A default volume `s3v` is automatically created for S3 operations.
 - **Programmatic Access**: Through the Java Client API

docs/03-core-concepts/01-namespace/02-buckets/01-overview.md

@@ -24,6 +24,45 @@ In Apache Ozone, a **Bucket** is the primary container for storing objects (keys
 
 If a Volume is like a top-level user directory or tenant space, a Bucket can be thought of as a specific project folder or data category within that space. Keys within the bucket are the actual files or objects belonging to that project or category.
 
+## Bucket Name Limitations
+
+Bucket names in Ozone must adhere to specific naming conventions:
+
+* Length: 3-63 characters
+* Must start and end with a lowercase letter or number
+* Can contain lowercase letters, numbers, hyphens (-), and periods (.)
+* Cannot start or end with a hyphen or period
+* Cannot have two consecutive periods
+* Cannot have a period adjacent to a hyphen
+* Cannot be an IPv4 address or all numeric
+
+By default, Ozone adheres strictly to S3 naming conventions, which is important for S3 API compatibility. In this "strict S3" mode:
+
+* Uppercase letters are not allowed
+* Underscores (_) are not allowed
+
+### Hadoop Compatibility Trade-offs
+
+When using Ozone with Hadoop-compatible file systems (HCFS/OFS), you may want to use underscores in bucket names, which are common in Hadoop environments but not allowed in S3. 
+
+Ozone provides a configuration option to relax the S3 naming restrictions:
+
+```xml
+<property>
+  <name>ozone.om.namespace.s3.strict</name>
+  <value>false</value>
+</property>
+```
+
+When set to `false`, this configuration allows:
+* Underscores (_) in bucket names
+* More flexibility for working with Hadoop ecosystem tools
+
+**Trade-offs:**
+* Setting `ozone.om.namespace.s3.strict` to `false` enhances Hadoop compatibility
+* However, buckets with underscores will not be accessible through the S3 API
+* Choose based on your primary access pattern (S3 vs HCFS/OFS)
+
 ## Usage
 
 Buckets are created using Ozone tools (CLI, Java API) or compatible interfaces (S3 API, HCFS API).

docs/03-core-concepts/01-namespace/02-buckets/04-layouts/01-object-store.md

@@ -32,6 +32,7 @@ You can specify the OBS layout during bucket creation using the Ozone CLI:
 
 ```bash
 ozone sh bucket create --layout OBJECT_STORE /volumeName/bucketName
+ozone sh bucket create --layout obs /volumeName/bucketName
 ```
 
 Alternatively, you can set OBS as the default layout for all new buckets in the Ozone Manager configuration (`ozone-site.xml`):
@@ -47,9 +48,5 @@ Alternatively, you can set OBS as the default layout for all new buckets in the
 </property>
 ```
 
-## Choosing OBS vs. FSO
-
-*   Choose **OBS** if your primary need is **S3 compatibility** and a flat object namespace.
-*   Choose **FSO** if your primary need is **HCFS/OFS compatibility**, filesystem semantics (atomic renames/deletes via OFS), and integration with traditional Hadoop ecosystem tools (Spark, Hive, Impala).
 
 See the [FSO documentation](../file-system-optimized) for more details on the alternative layout.

docs/03-core-concepts/01-namespace/02-buckets/04-layouts/02-file-system-optimized.md

@@ -49,7 +49,8 @@ FSO is one of the recommended layouts for new deployments, alongside OBS.
 You can specify the FSO layout during bucket creation using the Ozone CLI:
 
 ```bash
-ozone sh bucket create --layout FILE_SYSTEM_OPTIMIZED /volumeName/bucketName
+ozone sh bucket create --layout fso /volumeName/bucketName
+ozone sh bucket create --layout <FILE_SYSTEM_OPTIMIZED|fso> /volumeName/bucketName
 ```
 
 Alternatively, you can set FSO as the default layout for all new buckets in the Ozone Manager configuration (`ozone-site.xml`):
@@ -65,20 +66,4 @@ Alternatively, you can set FSO as the default layout for all new buckets in the
 </property>
 ```
 
-## Choosing FSO vs. OBS
-
-*   **Choose FSO (File System Optimized) when:**
-    *   Your primary access pattern involves **HCFS/OFS compatible tools** like Spark, Hive, Impala, Presto, Flink, etc.
-    *   You require **fast, atomic directory operations** (renames, deletes) for tasks like `INSERT OVERWRITE`, job commit phases, or large-scale directory restructuring.
-    *   You need **strong consistency** for directory modifications, especially in concurrent environments.
-    *   You are migrating workloads from HDFS and want similar directory semantics and performance characteristics.
-    *   Workloads involve frequent metadata operations or operations on large directories (thousands/millions of files).
-
-*   **Choose OBS (Object Store) when:**
-    *   Your primary access pattern is through the **S3 API** using S3-native tools, SDKs, or applications.
-    *   You prioritize **strict S3 compatibility** and behavior (e.g., for cloud-native applications).
-    *   Your workload involves mostly **object-level GET/PUT/DELETE** operations rather than frequent directory manipulations.
-    *   You are storing large amounts of unstructured data like media files, backups, or logs where filesystem hierarchy is less important than S3 API access.
-    *   You need features specific to object storage paradigms, such as object versioning or complex bucket policies managed via S3 APIs.
-
 See the [OBS documentation](../object-store) for more details on the alternative layout.

docs/03-core-concepts/01-namespace/02-buckets/04-layouts/README.mdx

@@ -6,25 +6,27 @@ Ozone's flexibility in bucket layouts allows a single cluster to serve as both a
 
 ## Available Layouts
 
-Ozone provides the following bucket layouts:
-
-1.  **[Object Store (OBS)](object-store):**
-    *   **Namespace:** Flat (like S3).
-    *   **Compatibility:** Optimized for strict S3 compatibility.
-    *   **Use Case:** S3-native applications, cloud-native workloads, unstructured data storage (media, backups).
-    *   **HCFS Access (`ofs://`):** Not supported.
-
-2.  **[File System Optimized (FSO)](file-system-optimized):**
-    *   **Namespace:** Hierarchical (like HDFS).
-    *   **Compatibility:** Optimized for HCFS compatibility (`ofs://`, `o3fs://`). Supports atomic directory operations via HCFS.
-    *   **Use Case:** HDFS replacement, traditional analytics (Spark, Hive, Impala), workloads requiring filesystem semantics.
-    *   **S3 Access:** Supported, but directory operations are not atomic via S3.
-
-3.  **Legacy:**
-    *   This was the original layout before OBS and FSO were introduced.
-    *   It provides a flat namespace.
-    *   Its behavior, especially regarding S3 access and filesystem path interpretation, can depend on the `ozone.om.enable.filesystem.paths` configuration setting.
-    *   **Recommendation:** For all new buckets and use cases, prefer either **OBS** or **FSO** over the Legacy layout.
+Ozone provides three bucket layouts, each optimized for different use cases and access patterns:
+
+| Feature                             | **[Object Store (OBS)](object-store)**                                                                                   | **[File System Optimized (FSO)](file-system-optimized)**                                                                | **Legacy**                                              |
+|-------------------------------------|--------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------|
+| **Namespace Structure**             | Flat (like S3)                                                                                                           | Hierarchical (like HDFS)                                                                                                | Flat                                                    |
+| **Primary Compatibility**           | S3 API                                                                                                                   | HCFS API (`ofs://`, `o3fs://`)                                                                                          | Mixed                                                   |
+| **S3 Access**                       | ✅ Native and fully optimized                                                                                             | ✅ Supported, but with limitations                                                                                       | ✅ Depends on config                                     |
+| **OFS/HCFS Access**                 | ❌ Not supported                                                                                                          | ✅ Native and fully optimized                                                                                            | ✅ Limited                                               |
+| **Atomic Directory Operations**     | ❌ Not applicable (flat namespace)                                                                                        | ✅ Via HCFS only (not via S3)                                                                                            | ❌ Limited                                               |
+| **Creation Command**                | `ozone sh bucket create --layout obs /volume/bucket`                                                                     | `ozone sh bucket create --layout fso /volume/bucket`                                                                    | `ozone sh bucket create --layout legacy /volume/bucket` |
+| **Best For**                        | • Cloud-native applications<br/>• S3-compatible workloads<br/>• Unstructured data storage<br/>• Media, backups, archives | • HDFS replacement<br/>• Analytics (Spark, Hive, Impala)<br/>• Directory-centric operations<br/>• File system semantics | • Legacy applications<br/>• Backwards compatibility     |
+| **Performance Characteristics**     | • Optimized for object operations<br/>                                                                                   | • Fast directory operations via HCFS<br/>• Atomic renames via HCFS<br/>• Enhanced metadata operations                   | Deprecated                                              |
+| **Recommended For New Deployments** | ✅ Yes, for S3-centric workloads                                                                                          | ✅ Yes, for HDFS-centric workloads                                                                                       | ❌ Deprecated, prefer OBS or FSO                         |
+
+### Key Differences
+
+- **OBS (Object Store)** provides a true S3-compatible object store experience with a flat namespace. It cannot be accessed through Hadoop filesystem interfaces.
+
+- **FSO (File System Optimized)** offers filesystem-like semantics with directories and atomic operations when accessed via `ofs://` or `o3fs://`. While it supports S3 access, directory operations through S3 are not atomic and may be significantly slower.
+
+- **Legacy** was the original bucket layout before OBS and FSO were introduced. Its behavior depends on configuration settings. For all new deployments, we recommend using either OBS or FSO.
 
 ## Choosing the Right Layout
 

docs/03-core-concepts/01-namespace/README.mdx

@@ -11,24 +11,9 @@ Ozone organizes data within a hierarchical namespace. Understanding this structu
 
 <div className="container" style={{marginTop: '2rem'}}>
   <div className="row">
-    {/* Overview Card */}
-    <div className="col col--4" style={{textAlign: 'center', padding: '1rem'}}>
-      <Link to="./namespace/overview" style={{textDecoration: 'none', color: 'inherit'}}>
-        <img
-          src={useBaseUrl('/img/namespace/overview.svg')}
-          alt="Namespace Overview"
-          width="80"
-          height="80"
-          style={{marginBottom: '1rem'}}
-        />
-        <h3>Overview</h3>
-      </Link>
-      <p>General concepts and the overall structure of the Ozone namespace.</p>
-    </div>
-
     {/* Volumes Card */}
-    <div className="col col--4" style={{textAlign: 'center', padding: '1rem'}}>
-      <Link to="./namespace/volumes/" style={{textDecoration: 'none', color: 'inherit'}}>
+    <div className="col col--6" style={{textAlign: 'center', padding: '1rem'}}>
+      <Link to="./volumes/" style={{textDecoration: 'none', color: 'inherit'}}>
         <img
           src={useBaseUrl('/img/namespace/hierarchy.svg')}
           alt="Ozone Volumes"
@@ -42,8 +27,8 @@ Ozone organizes data within a hierarchical namespace. Understanding this structu
     </div>
 
     {/* Buckets Card */}
-    <div className="col col--4" style={{textAlign: 'center', padding: '1rem'}}>
-      <Link to="./namespace/buckets/" style={{textDecoration: 'none', color: 'inherit'}}>
+    <div className="col col--6" style={{textAlign: 'center', padding: '1rem'}}>
+      <Link to="./buckets/" style={{textDecoration: 'none', color: 'inherit'}}>
         <img
           src={useBaseUrl('/img/namespace/bucket.svg')}
           alt="Ozone Buckets"