Hadoop 2 or YARN is the new version of Hadoop. It adds the yarn resource manager in addition to the HDFS and MapReduce components. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce designed and implemented by Google initially for processing and generating large data…
I ever introduced seting up git server using SSH or gitosis. However, gitolite is the way to go for managing git servers if you want an lightweight authentication layer. gitolite provides many very usefull features which can control each user’s right on each branch. I set up one gitolite git server and am very happy…
Error handling and resource management are pervasive in programs. RAII originated in C++ is great. With RAII, it is much easier to write easy-to-read code that allocats/deallocats resources in the constructor/destructors. By representing a resource with a local object, we are sure that local object’s destructor will release the resource and will not forget to…
The LSI MegaRAID is widely used like in IBM x3620M3 servers. This post introduces the process with screenshots of how to configure the new disks and set up RAID0 in the MegaRAID BIOS for reference by other and myself. It is illustrated with one example by adding a new 2TB hard disk and creating a…
The performance of Xen’s Dom0 is important for the overall system. The disk and network drivers are running on Dom0. I/O intensive guests’ workloads may consume lots Dom0′s CPU cycles. The Linux kernel calculates various network related parameters based on the amount of memory at boot time. The kernel also allocate memory for storing memory…
An introduction to the general method of installing Domain-U on Xen is introduced here: Setting Up Stable Xen DomU with Fedora: Unmodified Fedora 12 on top of Xenified Fedora 12 Dom0 with Xen (this is a general introduction, some details are changed, such as ‘xl’ replacing ‘xm’, LVM backing the disk for higher performance. But…
I ever discussed Starting KDE from Command Line by startx for KDE and GNOME. Recently, I installed MATE desktop on Fedora 17 and am very happy with it—dear, Gnome 2 is back. However, I checked the /etc/X11/xinit/Xclients script which is called by ‘startx’ and it only includes ‘startkde’ and ‘gnome-session’ without support to MATE. But…
ns is a discrete-event network simulator targeted primarily for research and educational use. There are two version of ns, ns-2 and ns-3, which are quite different with each other in the implementation and method to use them. In this post, we introduce how to install ns-2 and ns-3 on Fedora Linux 15. Install ns-2.34 on…
Setting up Ubuntu 10.10 DomU on top of Fedora Xen Dom0 is introduced in this post. The process of setting up Ubuntu 10.10 DomU is the same as Setting Up Stable Xen DomU with Fedora: Unmodified Fedora 12 on top of Xenified Fedora 12 Dom0 with Xen 4.0 This post only show the difference which…
This is a introduction to *nix’s process management tools: pkill and pgrep. As this site’s domain name was pkill.info, a introduction to pkill should exist here. NAME pgrep, pkill – look up or signal processes based on name and other attributes SYNOPSIS pgrep [-flvx] [-d delimiter] [-n|-o] [-P ppid,…] [-g pgrp,…] [-s sid,…] [-u euid,…]…
LVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds rather than minutes. We no longer need to copy the entire file system image like backing up file backed Xen DomU. We just need to make a snapshot of the current Xen DomU in seconds. When there are changes to the…
LVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds inside of the same volume group. When we want to duplicate LVM backed Xen DomU from a remote server, we need to make use of LVM’s snapshot function and tools like scp and dd. Backing up the DomU is only part…
Creating file-backed virtual block device (VBD) for Xen virtual machines and installing Fedora 12 in Xen DomU via internet will be introduced. Note that this tutorial is based on a pretty old OS (Fedora 12). But the method here is still valid while some minor details may need to be changed for latest Xen and…
Linux port forwarding is simple to do with iptables which may probably already being used as the firewall or part of the setting up a Linux gateway. In the Linux kernel, port forwarding is achieved by packet filter rules in iptables. Port forwarding Port forwarding also called “port mapping” commonly refers to the network address…
Compiling a latex documents may take several steps when bibtex is used with latex. However, this latex compilation process is the same for most documents. We can make it a template with Makefile so that simply running make will generate the dvi/ps/pdf files for us. A simple Makefile for using latex with bibtex enabled is…
How to set up eCryptFS in Linux will be introduced in this post. We can store encrypted files in one eCryptFS directory, the manual way. The content can be seen only after it is mounted as eCryptFS file system. Otherwise, the users can only see garbled characters in the files. Note that this tutorial will…
A script for backing up file-backed Xen DomU is introduced in this post. This script can be changed to similar platform. In our cluster, virtual machines are stored under /lhome/xen/. Virtual machine with id vmid is stored in directory vmvmid. The raw image disk file name can also be derived from vmid. Some more details…
Under some situation, we want to start some long jobs and then return to the shell to continue our work. We can do this by starting a job in the background. The shell such as bash will keep a list of background jobs and their status. But only one foreground job is allowed. Job control…
As a multimedia enthusiast, you may want to convert your movie files to audio files for various reasons such as creating soundtracks, audio books or listening to dialogues & music without the video. Converting movie files to WAV and MP3 files using MPlayer and LAME is a simple and straightforward process. By following the steps…
Update: If you are new to Hadoop and trying to install one. Please check the newer version: Hadoop Installation Tutorial (Hadoop 2.x). Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed…