We can set up a socks proxy on top of a SSH tunnel. Besides the common proxy functions, such as web browsing, the proxy on top of SSH tunnel also ensures the security between the browser and the proxy server (the SSH server). In this post, we introduce and explain how to set up a…
A script for backing up file-backed Xen DomU is introduced in this post. This script can be changed to similar platform. In our cluster, virtual machines are stored under /lhome/xen/. Virtual machine with id vmid is stored in directory vmvmid. The raw image disk file name can also be derived from vmid. Some more details…
Under some situation, we want to start some long jobs and then return to the shell to continue our work. We can do this by starting a job in the background. The shell such as bash will keep a list of background jobs and their status. But only one foreground job is allowed. Job control…
Sometimes we need to have the GUI of some application on the remote server. ssh’s X11 forwarding with “-X” parameter is a good and fast method. But if we want to have a whole desktop environment, vnc is a good choice. In this post, the method of how to control remote server using vnc through…
MPlayer has a fully configurable, command-driven control layer which allows you to control MPlayer using keyboard. But which are these shortcuts are not immediately know to users of MPlayer. Below is a list of mostly used MPlayer keyboard control shortcuts. They come from the [[man:1|mplayer|mplayer manual]] where you can find a full list of all…
We may need to flush DNS cache of our client when the site’s DNS has been changed. Flush DNS cache of Linux with NetworkManager/dnsmasq You may simply restart NetworkManager by sudo systemctl restart NetworkManager Flush DNS Cache on (Old) Linux with nscd 1) Use su – to get root privilege or sudo with the following…
When editing some files fro Windows in vim, there are some ugly “^M” returns. Here is a method to remove and replace these “^M”s (please note that you should not copy/past to use the follows): :1,$s/{Ctrl+V}{Ctrl+M}//{Enter} {Ctrl+V} means keyboard operation that is push ‘ctrl’ and ‘v’ together. {Enter} is push ‘enter’. : means that it…
Git and SSH are both powerful tools, and git/ssh work well together. We introduce how to set up git server via ssh in this post. Git server through SSH is easy and fast to set up, although every user will have access to all repositories in the git server over SSH and every user is…
DVD disks are still commonly used in some areas while the disks are not fast and convenient enough for frequent usage. For frequent usage or archiving, an image of the DVD on hard drives will be better. ISO images do not use a particular container format. They are just a sector-by-sector copy of the data…
1. Install relevant packages #yum install ccsm emerald-themes compizconfig-backend-gconf fusion-icon-gtk emerald compiz-fusion compiz-fusion-gnome libcompizconfig compiz-gnome compiz-bcop compiz compizconfig-python compiz-fusion-extras compiz-fusion-extras-gnome 2. Start compiz-fusion by using command “fusion-icon”. It can be found in “Applications -> system tools -> fusion-icon”. It will be more convenient if “fusion-icon” is added to autostart list by adding a entry in…
We need more loop devices on Linux than the default number (most commonly 8) some time, such as we are building 12 or more virtual machines on top of xen using loop device backed VBD. In this post, we want to use 32 loops. 1. If loop device is compiled directly in the kernel Pass…
As a multimedia enthusiast, you may want to convert your movie files to audio files for various reasons such as creating soundtracks, audio books or listening to dialogues & music without the video. Converting movie files to WAV and MP3 files using MPlayer and LAME is a simple and straightforward process. By following the steps…
We may want to run some jobs for every two weeks/months/days… under some situation such as backing up for every other week. In addition, we may add more complex rules for running jobs, e.g. run a command when the load of the server is higher than a certain level. With the help of the shell…
Compress/uncompress files are frequent operations. The normal tools for compressing/uncompressing in Linux is gzip, bzip2, 7z, rar and zip. This post introduces how to compress and uncompress file in Linux using these tools. We use best compressing rate with all these tools and mark the options for “best rate” in bold fonts. We can delete…
TeraSort is one of Hadoop’s widely used benchmarks. Hadoop’s distribution contains both the input generator and sorting implementations: the TeraGen generates the input and TeraSort conducts the sorting. Here, we provide a short tutorial for using the Hadoop TeraSort benchmark. TeraGen generates random data that can be used as input data for a subsequent running…
Update: If you are new to Hadoop and trying to install one. Please check the newer version: Hadoop Installation Tutorial (Hadoop 2.x). Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed…
Update: Since gitosis is not maintained and supported, please check out gitolite for setting up a new git server. (see the comment from Sitaram Chamarty, the gitolite author, the author of gitolite.) Gitosis is a piece of software writen by Tommi Virtanen for hosting git repositories. It manages multiple repositories under the same user account….
After [[hadoop-installation-tutorial|installing Hadoop]], we usually run some benchmark programs to test whether the system works well. In the post of the Hadoop install tutorial, we show a very simple to grep strings from a simple sets of files. In this post, we introduce the Sort for testing and benchmarking Hadoop. The Sort program is also…
The mrcc project’s homepage is here: mrcc project. Abstract mrcc is an open source compilation system that uses MapReduce to distribute C code compilation across the servers of the cloud computing platform. mrcc is built to use Hadoop by default, but it is easy to port it to other could computing platforms, such as MRlite,…