How to put files with spaces in names into HDFS?

Posted on

I got this error when I tried to save a file with a space in its name into HDFS: $ hdfs dfs -put -f “/home/u1/testa/test a” “/u1/testa/test a” put: unexpected URISyntaxException while the HDFS seems allow spaces in its file names: https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/filesystem/model.html . How to achieve the effect of saving the files with spaces in
Read more

How do I force Linux to unmount a filesystem?

Posted on

Some time, Linux fails to unmount a filesystem reporting “device is busy”. I understand that this helps avoid data lost by disallowing unmouting a filesystem when it is being used. But for some situations, I am sure there is something wrong happened or I care not data lost, such as a NFS mounting while the
Read more

Where is the source code for the free command on Linux?

Posted on

Where can I find the source code for the free command on Linux? The source code for the free commands (and many more, like kill, ps, top, sysctl) can be found in procps-ng: https://gitlab.com/procps-ng/procps procps is a set of command line and full-screen utilities that provide information out of the pseudo-filesystem most commonly located at
Read more

mkfs refuses to make filesystem with message “is apparently in use by the system; will not make a filesystem here!”

Posted on

I have a disk from another server installed on a new server. However, when I try to make a filesystem on it, mkfs reports # mkfs -t ext4 /dev/sdd1 mke2fs 1.42.8 (20-Jun-2013) /dev/sdd1 is apparently in use by the system; will not make a filesystem here! This is a new disk to the new server.
Read more

How to handle missing blocks and blocks with corrupt replicas in HDFS?

Posted on

One of HDFS cluster’s hdfs dfsadmin -report reports: Under replicated blocks: 139016 Blocks with corrupt replicas: 9 Missing blocks: 0 The “Under replicated blocks” can be re-replicated automatically after some time. How to handle the missing blocks and blocks with corrupt replicas in HDFS? Understanding these blocks A block is “with corrupt replicas” in HDFS
Read more

How to allow non-root users on Linux to mount and unmount disks?

Posted on

On a Linux box, we may allow non-root users mount and umount certain disks, such as /dev/sde1. How to allow non-root users on Linux to mount and unmount disks? Please check the tutorial at Controlling Filesystem Mounting on Linux by Playing with /etc/fstab: Allow non-root users to mount and unmount filesystems.

How to avoid mounting failures blocking Linux booting?

Posted on

Some entries in /etc/fstab may not not critical for booting Linux or even not available until Linux has booted. How to avoid the failures or unavailability of some mounting entries in /etc/fstab blocking the boot process of Linux? Please check the tutorial at Controlling Filesystem Mounting on Linux by Playing with /etc/fstab: Allow non-root users
Read more

Where are the backup files of my iPhone by iTune and how to get the real files from them?

Posted on

I backup my iPhone with iTune to my local computer. Where can I find out the backup files of my iPhone by iTune? And how to get the real files (like songs, notes, etc.) from the backup files? The location of the backup files from iTune It depends on the computer’s operating system: Mac: ~/Library/Application
Read more

Forcing Linux to Unmount a Filesystem Reporting “device is busy”

Posted on

Linux may report "device is busy" when we try to umount a filesystem. This behavior is reasonable as it can help us avoid data loss by disallowing unmouting a filesystem when it is being used. But for situations when we are sure there is something wrong happened or we care not data lost such as
Read more

How to Clean RAID Signatures on Linux

Posted on

RAID systems such as MegaRAID add signatures to disks to maintain the infomration on these didks. When we simply remove these disks and install them to another server, Linux on the new server may detect these RAID signature infomration and refuses to continue write to the disk. Here is one example that mkfs reports "apparently
Read more

How to Get Available Filesystem Space on Linux: a C Function with a C++ Example

Posted on

It is common for programs to write to files in filesystems on disks. However, what if the disk was almost full when your program writes to the filesystem on a disk? For systems software and mission critical programs, it is a better or must-to-do practice to check the available filesystem space before actually writing to
Read more

Controlling Filesystem Mounting on Linux using /etc/fstab

Posted on

Controlling the mounting of filesystems is a useful technique for managing Linux systems. The mounting configurations are mostly in the /etc/fstab file. In this post, we will discuss 2 common and useful techniques for controlling the filesystem mounting by playing with the /etc/fstab file: allowing non-root users to mount/unmount filesystems and avoiding mounting failures blocking
Read more

SSD Enabled For DreamHost Shared Hosting: Simple Performance Measurement

Posted on

SSD is common for VPS and PaaS virtual machines for higher I/O performance. Now, it is coming to shared hosting too. DreamHost states that "Now with solid state drives (SSDs), our standard web hosting loads pages 200% faster". We ourselves are happy to know this performance improvement with the price kept the same. Good work,
Read more

Shrinking an Ext4 File System on LVM in Linux

Posted on

In Extending a Mounted Ext4 File System on LVM in Linux, we introduced how to extend a mounted ext4 filesystem on LVM Logical volume by adding a new physical volume. It is also common to shrink an ext4 file system as to spare some disk space. In this post, I will discuss how to shrink
Read more

Hadoop Installation Tutorial (Hadoop 2.x)

Posted on

Hadoop 2 or YARN is the new version of Hadoop. It adds the yarn resource manager in addition to the HDFS and MapReduce components. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce designed and implemented by Google initially for processing and generating large data
Read more

How to Extend a Mounted Ext4 File System on LVM in Linux

Posted on

LVM is a great tool to manage hard disks on Linux—you can abstract the hard drives away and manage logical volumes from volume groups, you can dynamically add or remove hard drives while the file systems on the logical volumes need not to backed up and recovered, and you may create many snapshots of the
Read more

HP Compaq BIOS Crisis Recovery from a Failed BIOS Flashing Using a USB Flash Disk

Posted on

I tried to flash my HP Compaq CQ35-240TX’s BIOS to F.15 but unfortunately failed. The computer turned to white screen. When I restarted (powered off then boot again), this is no response any more: the hard disk and caps lock’s LED keep blinding. I thought "Oh my God! I must send it to HP to
Read more

Setting Up Xen DomU on Fedora Linux (Fedora 12)

Posted on

Creating file-backed virtual block device (VBD) for Xen virtual machines and installing Fedora 12 in Xen DomU via internet will be introduced. Note that this tutorial is based on a pretty old OS (Fedora 12). But the method here is still valid while some minor details may need to be changed for latest Xen and
Read more