Making Hadoop Java process heap larger?

In Hadoop 2.5.0, I use ‘ps -aux’ and find the Java process has options:

-Xmx1000m

However, my nodes have 32GB memory.

How to make Hadoop Java process heap larger?

In yarn-env.sh, you can find:

# For setting YARN specific HEAP sizes please use this
# Parameter and set appropriately
# YARN_HEAPSIZE=1000

In hadoop-env.sh, you can find:

# The maximum amount of heap to use, in MB. Default is 1000.
#export HADOOP_HEAPSIZE=
#export HADOOP_NAMENODE_INIT_HEAPSIZE=""

So, uncomment them and change them to the required values you want. Restart Hadoop and it will go.

Eric Ma

Eric is a systems guy. Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

Leave a Reply

Your email address will not be published. Required fields are marked *