Default Bucket Replication

The data redundancy for keys within an Apache Ozone bucket is determined by a replication configuration (type and value). This configuration can be set as an explicit default on the bucket itself, or it can be inherited from cluster-wide defaults if not set on the bucket. This document explains this process.

Setting a Default Replication on a Bucket

When a bucket is created, a default replication configuration can be explicitly stored on the bucket. This bucket-level default will apply to all keys created within it, unless a key is written with its own specific replication override.

Ozone uses the following order of precedence to set a default replication property on a bucket during creation:

1. Explicit Flags: Using both the --type and --replication flags in the ozone sh bucket create command. This is the most direct way to set a bucket's default replication.
2. Client-Side Value with Type Flag: Using the --type flag during bucket creation in combination with the ozone.replication property set in the client's ozone-site.xml.

Example: If ozone.replication=1 is set on the client, the following command sets the default replication for bucket2 to RATIS-1:

ozone sh bucket create vol1/bucket2 -t RATIS

Fallback to Cluster-Wide Defaults

If no default replication is set on the bucket during creation (i.e., neither of the methods above is used), the bucket is created without an explicit default replication property.

In this scenario, any key created within that bucket will fall back to using the cluster-wide default replication settings defined in the Ozone Manager's ozone-site.xml.

  <property>
    <name>ozone.server.default.replication.type</name>
    <value>RATIS</value>
    <tag>OZONE</tag>
    <description>
      The cluster-wide default replication type (e.g., RATIS, EC) used for keys
      when their bucket has no default replication set.
    </description>
  </property>
  <property>
    <name>ozone.server.default.replication</name>
    <value>3</value>
    <tag>OZONE</tag>
    <description>
      The cluster-wide default replication value used for keys when their
      bucket has no default replication set.
      <ul>
        <li><b>For RATIS:</b> 1, 3</li>
        <li><b>For EC (Erasure Coding):</b> Formatted as {Codec}-{Data}-{Parity}-{ChunkSize}.
          <ul>
            <li><b>Codec:</b> XOR, RS</li>
            <li><b>Data-Parity Blocks:</b> 3-2, 6-3, 10-4</li>
            <li><b>ChunkSize:</b> e.g., 1024k, 2048k</li>
          </ul>
        </li>
      </ul>
    </description>
  </property>

Example: If you run ozone sh bucket create vol1/bucket1 (with no flags or client properties), the bucket bucket1 is created with no default replication. When a key is later written to bucket1 without a specific replication override, it will use the cluster-wide server defaults (e.g., RATIS-3).