Directly SSH to Hosts’ Internal IPs Through the Gateway

We have many hosts with internal/LAN IPs like 10.0.3.* behind a gateway and the hosts with LAN IPs can connect to the Internet through the gateway. We used iptables to forward port from the gateway to internal IPs so that users from hosts with Internet connections can SSH to the gateway’s forwarded port to log on the internal hosts. However, there should be rules added for these hosts and the users need to connect to these non-standard (not 22) ports of the gateway that may be blocked by firewalls of their network. » Read more

SSH Port Forwarding on Linux

Port forwarding (or tunnelling) is a method to forward one network traffic to another. We will introduce how to forward ports using SSH tunnel in this post. A simple example Let’s start with a simple and useful example: we want to forward local port 8080 to server:port. We can easily do this by using ssh like this: ssh -L 8080:server:port username@ssh_server ssh_server is the sshd server that we can use. » Read more

Flushing iptables on Fedora

iptables is a mechanism in Linux kernel for port forwarding, NAT, firewalls etc. In Linux distros, such as Fedora, the iptables is configured to be as a “strict” firewall that opens a limited know ports, such as 22 for SSH. However, in some network environment, such as a private cluster, the nodes are trusted and firewalls are usually not needed. We may flush the default iptables to make it accept all connections and add rules as needed. » Read more

Installing Xen on Fedora as Domain-0 (Fedora 17)

The new development of Xen and Linux kernel make it easy to install Xen on Fedora as the Domain-0 now. This post uses Fedora 17 as an example platform to introduce how to set up Domain-0 on Fedora Linux. Compared to our old method (Setting up Stable Xen Dom0 with Fedora: Xen 3.4.3 with Xenified Linux Kernel in Fedora 12) which requires manually compiled Xen and patched kernel, the current packages and support to Xen in Fedora and Linux kernel make the system administrators life much easier. » Read more

Setting Up VPN-like Network Between Several Clusters Using iptables

It is common to connect servers with only internal IPs from several clusters. VPN is a common technique for this. With iptables, we can implement many functions of VPN with possibly higher performance. The slides here give a brief introduction to how to set up a VPN-like network between 2 clusters which connect to each other through Internet. Note that this is not a “real” VPN. » Read more

Linux Cluster Solutions

Solutions to Linux cluster construction and management such as unified account management, NFS home directory, network configurations are summarised in this post. The post is keeping updating while new solutions is added to this site. Account and storage management ∞ Unified Linux Login and Home Directory Using OpenLDAP and NFS/automount Backup Linux Home Directory Using rsync Setting Up eCryptFS in Linux Network related ∞ Setting Up Gateway Using iptables and route on Linux Flush DNS Cache of Linux and Windows Client Finding out Linux Network Configuration Information Changing MAC Address in Linux aka. » Read more

Port Forwarding Using iptables

Port forwarding is simple to do with iptables in a Linux box which may probably already being used as the firewall or part of the gateway operation. In Linux kernels, port forwarding is achieved by packet filter rules in iptables. Port forwarding ∞ Port forwarding also called “port mapping” commonly refers to the network address translator gateway changing the destination address and/or port of the packet to reach a host within a masqueraded, typically private, network. » Read more

Setting Up Gateway Using iptables and route on Linux

Sharing the networking is important. Setting up a gateway is a good solution to it. Building up the gateway on a Linux box is easy and cost efficient, but reliable. With a Linux box, you can share the internet connection or the only cable connected to the network. The Linux box network configuration The Linux box that we use has this configuration: NIC1: eth0 with ip connected to our small local area network. » Read more

Hadoop Installation Tutorial (Hadoop 1.x)

Update: If you are new to Hadoop and trying to install one. Please check the newer version: Hadoop Installation Tutorial (Hadoop 2.x). Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed and implemented by Google for processing and generating large data sets [1]. » Read more