However, my nodes have 32GB memory.
How to make Hadoop Java process heap larger?
In yarn-env.sh, you can find:
# For setting YARN specific HEAP sizes please use this # Parameter and set appropriately # YARN_HEAPSIZE=1000
In hadoop-env.sh, you can find:
# The maximum amount of heap to use, in MB. Default is 1000. #export HADOOP_HEAPSIZE= #export HADOOP_NAMENODE_INIT_HEAPSIZE=""
So, uncomment them and change them to the required values you want. Restart Hadoop and it will go.