If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
Please untar the ozone-<version> to the directory where you are going to run Ozone from. We need Ozone jars on all machines in the cluster. So you need to do this on all machines in the cluster.
Ozone relies on a configuration file called ozone-site.xml
. To
generate a template that you can replace with proper values, please run the
following command. This will generate a template called ozone-site.xml
at
the specified path (directory).
ozone genconf <path>
Let us look at the settings inside the generated file (ozone-site.xml) and
how they control ozone. Once the right values are defined, this file
needs to be copied to ozone directory/etc/hadoop
.
ozone.metadata.dirs Allows Administrators to specify where the metadata must reside. Usually you pick your fastest disk (SSD if you have them on your nodes). OzoneManager, SCM and datanode will write the metadata to this path. This is a required setting, if this is missing Ozone will fail to come up.
Here is an example,
<property>
<name>ozone.metadata.dirs</name>
<value>/data/disk1/meta</value>
</property>
Here is an example,
<property>
<name>ozone.scm.names</name>
<value>scm.hadoop.apache.org</value>
</property>
Here is an example,
<property>
<name>ozone.scm.datanode.id.dir</name>
<value>/data/disk1/meta/node</value>
</property>
Here is an example,
<property>
<name>ozone.om.address</name>
<value>ozonemanager.hadoop.apache.org</value>
</property>
Setting | Value | Comment |
---|---|---|
ozone.metadata.dirs | file path | The metadata will be stored here. |
ozone.scm.names | SCM server name | Hostname:port or IP:port address of SCM. |
ozone.scm.block.client.address | SCM server name and port | Used by services like OM |
ozone.scm.client.address | SCM server name and port | Used by client-side |
ozone.scm.datanode.address | SCM server name and port | Used by datanode to talk to SCM |
ozone.om.address | OM server name | Used by Ozone handler and Ozone file system. |
hdds.datanode.dir | file path | HDDS datanodes store data in this directory. |
Before we boot up the Ozone cluster, we need to initialize both SCM and Ozone Manager.
ozone scm --init
init
command is similar to Namenode format. Init command is executed only once, that allows SCM to create all the required on-disk structures to work correctly.
ozone --daemon start scm
Once we know SCM is up and running, we can create an Object Store for our use. This is done by running the following command.
ozone om --init
Once Ozone manager is initialized, we are ready to run the name service.
ozone --daemon start om
At this point Ozone’s name services, the Ozone manager, and the block service SCM is both running.
Please note: If SCM is not running
om --init
command will fail. SCM start will fail if on-disk data structures are missing. So please make sure you have done both scm --init
and om --init
commands.
Now we need to start the data nodes. Please run the following command on each datanode.
ozone --daemon start datanode
At this point SCM, Ozone Manager and data nodes are up and running.
Congratulations!, You have set up a functional ozone cluster.
If you want to make your life simpler, you can just run
ozone scm --init
ozone om --init
start-ozone.sh
This assumes that you have set up the workers
file correctly and ssh
configuration that allows ssh-ing to all data nodes. This is the same as the
HDFS configuration, so please refer to HDFS documentation on how to set this
up.