How to Set the Static IP Address Using CLI in Fedora/CentOS Linux

How to set the static IP address for CentOS 7/Fedora 22+ is introduced in this post. Here, we assume the Linux is using NetworkManager to manage the network. All steps are done as root. Find the interface you want to set the address ∞ Command ifconfig -a lists all interfaces. Find the interface name to set, such as enp2s0. datacenter-servers.jpg Edit the interface’s configuration file ∞ For interface NAME, the configuration file is stored in /etc/sysconfig/network-scripts/ifcfg-NAME. » Read more

Uploading Large Files to Amazon S3 with AWS CLI

Amazon S3 is a widely used public cloud storage system. S3 allows an object/file to be up to 5TB which is enough for most applications. The AWS Management Console provides a Web-based interface for users to upload and manage files in S3 buckets. However, uploading a large files that is 100s of GB is not easy using the Web interface. From my experience, it fails frequently. » Read more

OpenVPN + Ovpn Spider: Free VPNs for iPhone Users

VPNs are useful for testing viewing webpages from a remote location or visiting blocked websites. In this post, I will introduce a method for finding and using free VPNs on iPhone. Note that the VPNs are free and the providers are not identified. Hence, only use this method for visiting public webpages or keeping visiting https sites. If security or privacy are important concerns, find a reliable VPN service or set up your own. » Read more

MPlayer over SSH to Play Movie from Remote Host

MPlayer is a great movie player. SSH is a great tool to connect and transfer data over the network securely. Combining these 2 great tools together will be greater. You may already has a small server storing some movies or videos or music and playing them on your laptop will be convenient. In this post, we will introduce three methods to play movie using MPlayer over SSH. » Read more

Creating Fedora 20 Domain-U on Fedora 20 Domain-0

In this post, creating a file-backed virtual block device (VBD) and installing Fedora 20 in the Xen DomU via internet will be introduced. This domain is created on a Fedora 20 Dom0 as introduced in Installing Xen on Fedora 20 as Domain-0. For better performance, you may consider using LVM backed VM. Create file-backed VBD The actual space of VBD will be the amount of disk the virtual machine used. » Read more

Installing Xen on Fedora 20 as Domain-0

I ever introduced Installing Xen on Fedora as Domain-0 (Fedora 17) as the first try to use the xen package delivered from Fedora and get away from manually compiling Xen and patching the kernel. In this post, I introduce installing Xen Dom0 on Fedora 20. Installing Xen First, install the xen pacakges: # yum install xen The Linux kernel is already ready to run in Domain-0 with the pv_ops technology enabled. » Read more

Git through SSH Tunnel as Proxy

git is a great tool and it is common to have a git server over SSH possibly managed by gitolite. However, there are situations that we can not directly connect to the git server but be able to SSH to another node that can connect to the git server. The git server may allow only internal connections because of security. With the node that we can SSH to, we can still use the git server “directly” by setting up a SSH tunnel as a proxy for the git connection. » Read more

Directly SSH to Hosts’ Internal IPs Through the Gateway

We have many hosts with internal/LAN IPs like 10.0.3.* behind a gateway and the hosts with LAN IPs can connect to the Internet through the gateway. We used iptables to forward port from the gateway to internal IPs so that users from hosts with Internet connections can SSH to the gateway’s forwarded port to log on the internal hosts. However, there should be rules added for these hosts and the users need to connect to these non-standard (not 22) ports of the gateway that may be blocked by firewalls of their network. » Read more

How to Set Up A Gitolite Git Server – A Ten-Minute Tutorial

I ever introduced seting up git server using SSH or gitosis. However, gitolite is the way to go for managing git servers if you want an lightweight authentication layer. gitolite provides many very usefull features which can control each user’s right on each branch. I set up one gitolite git server and am very happy with it. In this post, let’s look at how to set up one gitolite git server. » Read more

Sending Email Using mailx in Linux Through Internal SMTP

Sending Email from mailx Command in Linux Using Gmail’s SMTP introduces how to send email using mailx or mail command in Linux through Gmail’s SMTP which requires some configuration. On the other hand, there are many environments that do not require SSL/TLS/etc. One example is the SMTP server (smtp.ust.hk) for HKUST requires no authentication for sending email from IPs inside the campus network. » Read more

SSH Port Forwarding on Linux

Port forwarding (or tunnelling) is a method to forward one network traffic to another. We will introduce how to forward ports using SSH tunnel in this post. A simple example Let’s start with a simple and useful example: we want to forward local port 8080 to server:port. We can easily do this by using ssh like this: ssh -L 8080:server:port username@ssh_server ssh_server is the sshd server that we can use. » Read more

Managing Xen Dom0′s CPU and Memory

The performance of Xen’s Dom0 is important for the overall system. The disk and network drivers are running on Dom0. I/O intensive guests’ workloads may consume lots Dom0′s CPU cycles. The Linux kernel calculates various network related parameters based on the amount of memory at boot time. The kernel also allocate memory for storing memory metadata (per page info structures) is also based on the boot time amount of memory. » Read more

Linux UDP Programming Tutorial

UDP has its advantages over TCP, such as being relatively lightweight and receiving one packet per read call (recvmsg), although the programmers need to handle related issues, such as packet lost and out-of-order packets delivery. This post gives information and references on how to write UDP programs in a C/Linux environment. What is UDP ∞ Check the wikipedia article on UDP User Datagram Protocol for a brief overview. » Read more

Software Engineering Advice from Building Large-Scale Distributed Systems by Jeff Dean

Software Engineering Advice from Building Large-Scale Distributed Systems by Jeff Dean. You can download the slides from Software Engineering Advice from Building Large-Scale Distributed Systems by Jeff Dean. These slides contain the “Numbers everyone should know” which everyone working on systems should be familiar with. Numbers Everyone Should Know ∞ L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns Mutex lock/unlock 100 ns Main memory reference 100 ns Compress 1K bytes with Zippy 10,000 ns Send 2K bytes over 1 Gbps network 20,000 ns Read 1 MB sequentially from memory 250,000 ns Round trip within same datacenter 500,000 ns Disk seek 10,000,000 ns Read 1 MB sequentially from network 10,000,000 ns Read 1 MB sequentially from disk 30,000,000 ns Send packet CA->Netherlands->CA 150,000,000 ns        » Read more

Hadoop MapReduce Tutorials

Here is a list of tutorials for learning how to write MapReduce programs on Hadoop, the opensource MapReduce implementation with HDFS. MapReduce Tutorials ∞ The official tutorial on Hadoop MapReduce framework: http://hadoop.apache.org/docs/r1.0.4/mapred_tutorial.html. Yahoo! Hadoop Tutorial ∞ A comprehensive tutorial on Hadoop from Yahoo! Developer Network: http://developer.yahoo.com/hadoop/tutorial/. More about MapReduce ∞ To better understand the design behind MapReduce, it is always good to read Jeff Dean and Sanjay Ghemawat’s MapReduce paper: http://research.google.com/archive/mapreduce.html. » Read more