Merge pull request ceph#64809 from bill-scales/ec_docs

Doc: Erasure Coding enhancements for tentacle

Author: anthonyeleven

doc/cephfs/createfs.rst

@@ -137,5 +137,11 @@ You may use Erasure Coded pools as CephFS data pools as long as they have overwr
     
 Note that EC overwrites are only supported when using OSDs with the BlueStore backend.
 
+If you are storing lots of small files or are frequently modifying files you can improve performance by enabling EC optimizations, which is done as follows:
+
+.. code:: bash
+
+    ceph osd pool set my_ec_pool allow_ec_optimizations true
+
 You may not use Erasure Coded pools as CephFS metadata pools, because CephFS metadata is stored using RADOS *OMAP* data structures, which EC pools cannot store.
 

doc/rados/configuration/pool-pg-config-ref.rst

@@ -69,6 +69,7 @@ See :ref:`pg-autoscaler`.
 .. confval:: osd_max_pg_log_entries
 .. confval:: osd_default_data_pool_replay_window
 .. confval:: osd_max_pg_per_osd_hard_ratio
+.. confval:: osd_pool_default_flag_ec_optimizations
 
 .. _pool: ../../operations/pools
 .. _Monitoring OSDs and PGs: ../../operations/monitoring-osd-pg#peering

doc/rados/operations/erasure-code-profile.rst

@@ -80,7 +80,9 @@ Where:
               ``osd_pool_erasure_code_stripe_unit`` when a pool is
               created.  The stripe_width of a pool using this profile
               will be the number of data chunks multiplied by this
-              stripe_unit.
+              stripe_unit. See `Erasure Coding Optimizations`_ for
+              more information.
+
 
 :Type: String
 :Required: No.

doc/rados/operations/erasure-code.rst

@@ -206,6 +206,62 @@ erasure-coded pool as the ``--data-pool`` during image creation:
 For CephFS, an erasure-coded pool can be set as the default data pool during
 file system creation or via :ref:`file-layouts`.
 
+Erasure Coding Optimizations
+----------------------------
+
+Since Tentacle, an erasure-coded pool may have optimizations enabled
+with a per-pool setting. This improves performance for smaller I/Os and
+eliminates padding which can save capacity:
+
+.. prompt:: bash $
+
+    ceph osd pool set ec_pool allow_ec_optimizations true
+
+The optimizations will make an erasure code pool more suitable for use
+with RBD or CephFS. For RGW workloads that have large objects that are read and
+written sequentially there will be little benefit from these optimizations; but
+RGW workloads with lots of very small objects or small random access reads will
+see performance and capacity benefits.
+
+This flag may be enabled for existing pools, and can be configured
+to default for new pools using the central configuration option
+:confval:`osd_pool_default_flag_ec_optimizations`. Once the flag has been
+enabled for a pool it cannot be disabled because it changes how new data is
+stored.
+
+The flag cannot be set unless all the Monitors and OSDs have been
+upgraded to Tentacle or later. Optimizations can be enabled and used without
+upgrading gateways and clients.
+
+Optimizations are currently only supported with the Jerasure and ISA-L plugins
+when using the ``reed_sol_van`` technique (these are the old and current
+defaults and are the most widely used plugins and technique). Attempting to
+set the flag for a pool using an unsupported combination of plugin and
+technique is blocked with an error message.
+
+The default stripe unit is 4K which works well for standard EC pools.
+For the majority of I/O workloads it is recommended to increase the stripe
+unit to at least 16K when using optimizations. Performance testing
+shows that 16K is the best choice for general purpose I/O workloads. Increasing
+this value will significantly improve small read performance but will slightly
+reduce the performance of small sequential writes. For I/O workloads that are
+predominately reads, larger values up to 256KB will further improve read
+performance but will further reduce the performance of small sequential writes.
+Values larger than 256KB are unlikely to have any performance benefit. The
+stripe unit is a pool create-time option that can be set in the erasure code
+profile or by setting the central configuration option
+:confval:`osd_pool_erasure_code_stripe_unit`. The stripe unit cannot be changed
+after the pool has been created, so if enabling optimizations for an existing
+pool you will not get the full benefit of the optimizations.
+
+Without optimizations enabled, the choice of ``k+m`` in the erasure code profile
+affects performance. The higher the values of ``k`` and ``m`` the lower the
+performance will be. With optimizations enabled there is only a very slight
+reduction in performance as ``k`` increases so this makes using a higher value
+of ``k`` more viable. Increasing ``m`` still impacts write performance,
+especially for small writes, so for block and file workloads a value of ``m``
+no larger than 3 is recommended.
+
 Erasure-coded pool overhead
 ---------------------------
 

doc/rados/operations/pools.rst

@@ -451,7 +451,14 @@ You may set values for the following keys:
    :Type: Boolean
 
    .. versionadded:: 12.2.0
+
+.. describe:: allow_ec_optimizations
 
+   :Description: Enables performance and capacity optimizations for an erasure-coded pool. These optimizations were designed for CephFS and RBD workloads; RGW workloads with signficant numbers of small objects or with small random access reads of objects will also benefit. RGW workloads with large sequential read and writes will see little benefit. For more details, see `Erasure Coding Optimizations`_.
+   :Type: Boolean
+
+   .. versionadded:: 20.2.0
+
 .. describe:: hashpspool
 
    :Description: Sets or unsets the ``HASHPSPOOL`` flag on a given pool.
@@ -900,6 +907,7 @@ Here are the break downs of the argument:
 
 .. _Bloom Filter: https://en.wikipedia.org/wiki/Bloom_filter
 .. _Erasure Coding with Overwrites: ../erasure-code#erasure-coding-with-overwrites
+.. _Erasure Coding Optimizations: ../erasure-code#erasure-coding-optimizations
 .. _Block Device Commands: ../../../rbd/rados-rbd-cmds/#create-a-block-device-pool
 .. _pgcalc: ../pgcalc