How to manually kill HDFS DataNodes?

stop-dfs.sh report that there are no datanodes running on some nodes like

hdfs-node-000208: no datanode to stop

However, there are DataNode process running there. How to clean these processes on many (100s) of nodes?

You may use this piece of bash script:

for i in `cat hadoop/etc/hadoop/slaves`; do 
echo $i; 
ssh $i 
  'jps | grep DataNode | cut -d" " -f1 
  | xargs --no-run-if-empty -I@ bash -c "echo  -- killing @; kill @"'; 
done

What it does is, for each slave ndoes, run the command started by jps:

finds DataNode Java processes, gets the process IDs and passes these IDs to kill to kill them.

Answered by Eric Z Ma.

Eric Z Ma

Eric is a father and systems guy. Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

Leave a Reply

Your email address will not be published. Required fields are marked *