The prefix based FileSystem optimization feature supports atomic rename and delete of any directory at any level in the namespace. Also, it will perform rename and delete of any directory in a deterministic/constant time.
Note: This feature works only when
enabled which means that Hadoop Compatible File System compatibility is
favored instead of S3 compatibility. Some irregular S3 key names may be
rejected or normalized.
This feature is strongly recommended to be turned ON when Ozone buckets are mainly used via Hadoop compatible interfaces, especially with high number of files in deep directory hierarchy.
OzoneManager supports two metadata layout formats - simple and prefix.
Simple is the existing OM metadata format, which stores key entry with full path
name. In Prefix based optimization, OM metadata format stores intermediate
DirectoryTable and files into
FileTable as shown in the
below picture. The key to the table is the name of a directory or a file
prefixed by the unique identifier of its parent directory,
Following picture describes the OM metadata changes while performing a delete operation on a directory.
Following picture describes the OM metadata changes while performing a rename operation on a directory.
By default the feature is disabled. It can be enabled with the following
<property> <name>ozone.om.enable.filesystem.paths</name> <value>true</value> </property> <property> <name>ozone.om.metadata.layout</name> <value>PREFIX</value> </property>