The prefix-based File System Optimization feature supports atomic rename and delete of any directory at any level in the namespace in deterministic/constant time.
This feature can be enabled for each specific bucket that requires it by setting the --layout
flag
to FILE_SYSTEM_OPTIMIZED
at the time of bucket creation.
ozone sh bucket create /<volume-name>/<bucket-name> --layout FILE_SYSTEM_OPTIMIZED
Note: File System Optimization favors Hadoop Compatible File System instead of S3 compatibility. Some irregular S3 key names may be rejected or normalized.
This feature is strongly recommended to be turned ON for Ozone buckets mainly used via Hadoop compatible interfaces, especially with high number of files in deep directory hierarchy.
OzoneManager supports two metadata bucket layout formats - Object Store (OBS) and File System Optimized (FSO).
Object Store (OBS) is the existing OM metadata format, which stores key entry with full path name. In File System
Optimized (FSO) buckets, OM metadata format stores intermediate directories into DirectoryTable
and files
into FileTable
as shown in the below picture. The key to the table is the name of a directory or a file prefixed by
the unique identifier of its parent directory, <parent unique-id>/<filename>
.
Following picture describes the OM metadata changes while performing a delete operation on a directory.
Following picture describes the OM metadata changes while performing a rename operation on a directory.
The following configuration can be configured in ozone-site.xml
to define the default value for bucket layout during bucket creation
if the client has not specified the bucket layout argument.
Supported values are OBJECT_STORE
, FILE_SYSTEM_OPTIMIZED
and LEGACY
.
By default, this config value is empty. Ozone will default to LEGACY
bucket layout if it finds an empty config value.
<property>
<name>ozone.default.bucket.layout</name>
<value/>
</property>