Pitfalls and Lessons on Configuing and Tuning Hadoop

This post lists pitfalls and lessons learning when configuring and tuning Hadoop.

Hadoop with IPv6

Hadoo doesn’t support IPv6 currently (up to 0.20.2 and 0.21.0): Hadoop and IPv6. The performance of the cluster may suffer from turning IPv6 on in clusters: mail archive.

One good practice is to disable IPv6 on servers in the Hadoop cluster.

Hostname vs. IP

Hostname is preferred. Using hostname instead of IP address may possible solve some problem magically.

Java Virtual Machine

Take care whether the JVM on all nodes are friendly with Hadoop. In case there are some nodes are configured with wrong JVM, problems may flow up.

  •  
  •  
  •  
  •  
  •  
  •  
Eric is a father and systems guy. Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

One Reply to “Pitfalls and Lessons on Configuing and Tuning Hadoop”

Leave a Reply

Your email address will not be published. Required fields are marked *