How to add a new HDFS NameNode metadata directory to an existing cluster?

We have a running HDFS cluster. Currently, the NameNode metadata data directory has only one directory configured in hdfs-site.xml:

<property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///home/hadoop/hdfs/</value>
    <description>NameNode directory for namespace and transaction logs storage.</description>
</property>

We would like to add a new directory for dfs.namenode.name.dir to make replicas of the metadata on a separated disk for higher data reliability.

How to safely add the new directory?

Backup first as the NameNode metadata directory contains critical data for the HDFS cluster.

Then following these steps.

First, shutdown the NameNode.

Second, copy the files from the old NameNode metadata directory to the new metadata directory. Such as to /home/hadoop/hdfs2,

cp -r /home/hadoop/hdfs/* /home/hadoop/hdfs2/

Third, add the new metadata directory to hdfs-site.xml:

<property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///home/hadoop/hdfs/, file:///home/hadoop/hdfs2/</value>
    <description>NameNode directory for namespace and transaction logs storage.</description>
</property>

Last, restart the NameNode.

Answered by Eric Z Ma.

Eric Z Ma

Eric is a father and systems guy. Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

Leave a Reply

Your email address will not be published. Required fields are marked *