Tutorial

Pitfalls and Lessons on Configuing and Tuning Hadoop

ByEric Ma Apr 26, 2011Mar 27, 2018

This post lists pitfalls and lessons learning when configuring and tuning Hadoop.

Hadoop with IPv6

Hadoo doesn’t support IPv6 currently (up to 0.20.2 and 0.21.0): Hadoop and IPv6. The performance of the cluster may suffer from turning IPv6 on in clusters: mail archive.

One good practice is to disable IPv6 on servers in the Hadoop cluster.

Hostname vs. IP

Hostname is preferred. Using hostname instead of IP address may possible solve some problem magically.

Java Virtual Machine

Take care whether the JVM on all nodes are friendly with Hadoop. In case there are some nodes are configured with wrong JVM, problems may flow up.

How to find out the MX record for a domain on Linux?

ByEric Ma Mar 24, 2018Mar 24, 2018

How to find out a domain’s MX record for Email on Linux? For example, to find which domain/IP the email to gmail.com is delivered to. You may make use of the host command. The -t option is used to select the query type. type can be any recognized query type: CNAME, NS, SOA, SIG, KEY,…

How to make CentOS 6.6 power off the console screen automatically?

ByEric Ma Mar 24, 2018Mar 24, 2018

On CentOS 6.6, I find that it can make the screen (console, not X) blank after a while. However, it does not power the screen off. This usually work on Fedora or other releases. I guess this is caused by some configurations specific in CentOS 6.6. How to make CentOS 6.6 power off the console…

how to get the size of python file without using the os namespace

ByEric Ma Mar 24, 2018Mar 24, 2018

i need to create a program which is taking information from printer , so that i cannot use os namespace , so there is any way without using the os namespace You may use a way as follows. open the file seek to the end tell its position which is the file size An example…

How to not use concrete types in lambda function parameters in C++11?

ByEric Ma Mar 24, 2018Nov 21, 2019

C++11 requires that lambda function parameters be declared with concrete types. This is sometimes annoying. auto is really nice, especially when the type is complex like std::vector<std::string>::iterator is quite long to type. I know C++14 allows auto in lambda functions. But how to not use concrete types in lambda function parameters in C++11? In C++11,…

How to view a file at a specific commit in git?

ByEric Ma Mar 24, 2018Mar 24, 2018

How to view a file at a specific commit/revision in git? You can use git show to view a file’s content at a specific commit in git: https://www.systutorials.com/docs/linux/man/1-git-show/ $ git show REVISION:/path/to/file You can also save a copy by $ git show REVISION:/path/to/file >file.copy Read more: How to produce a patch file for a specific…

Programming

4 Features of Python 3.9 That You Can’t Take Your Eyes Off

ByAaronjacobson Aug 23, 2020Sep 9, 2020

Python is one of the most popular programming languages in the world. It is widely popular for a plethora of tasks due to its flexible nature and ease of use. Python has also managed to beat other programming languages such as Java, which were once upon a time, the world’s favorite. In fact, the extent…

One Comment

Zhiqiang Ma says:

Oct 18, 2012 at 8:20 pm

Check the /etc/hosts file whether it contains wrong record. Always ensure it is correct.

Reply

Hadoop with IPv6

Hostname vs. IP

Java Virtual Machine

Similar Posts

One Comment

Leave a Reply Cancel reply