What is the design of Snapshots in HDFS?
This PDF documents the design of snapshot.
Jing Zhao and Tsz-Wo Sze from Hortonworks gave a great talk on the design of HDFS snapshots.
The slides can be downloaded at here.
The development of snapshot is tracked by HDFS-2802.
How to filter RSS items and leave only certain items that I want? Rules may be like: satisfies all or any rules { contains keyword; does not contain keyword } or etc. Two sServices that I find working well for me: Yahoo Piples: https://pipes.yahoo.com/pipes/</s> Pipes is a powerful composition tool to aggregate, manipulate, and mashup…
How to change the hostname on Linux Mint? The hostname (e.g. mypc or mypc.example.com) can be changed by putting the hostname in one line in the file /etc/hostname on Linux Mint. Read more: How to Change Hostname of Fedora Linux How to get hostname in Python on Linux? Getting Hostname in C Programs in Linux…
This post lists pitfalls and lessons learning when configuring and tuning Hadoop. Hadoop with IPv6 Hadoo doesn’t support IPv6 currently (up to 0.20.2 and 0.21.0): Hadoop and IPv6. The performance of the cluster may suffer from turning IPv6 on in clusters: mail archive. One good practice is to disable IPv6 on servers in the Hadoop…
In system, sometimes, we need backstage threads with very very low priority since it cannot preempt any normal process asks for CPU. SCHED_IDLE class satisfies this requirement which has the priority lower than “nice=19” of CFS. Following code does this. 241 void set_idle_priority(void) { 242 struct sched_param param; 243 param.sched_priority = 0; 244 int s…
How to exclude directories with certain names like “cache” from rsync on Linux during backup? The “cache” directory may in many different paths, such as file1/cache/ or file2/cache/, and adding all “cache” directories to rsync command is not a doable way. You can use rsync with –exclude=cache/ like rsync -avxP –exclude=cache/ /path/to/src/directory/ /path/to/dst/dir/ Read more:…
How to iterate all dirs and files in a dir in C++? To open a dir, we can use opendir() to open a directory stream. DIR *opendir(const char *name); Then we can use readdir() to iterate the directory stream. struct dirent *readdir(DIR *dirp); Here is an example C++ program using these 2 library functions. #include…