It is common that the root disk space is not enough when running a Virtual Machine in the cloud such as Amazon Web Service (AWS). The cloud storage usually provides tools or facilities to enlarge a virtual disk size. However, to make the Linux recognize and and use the enlarged disks without rebooting the OS,…
What is the difference between work conserving I/O scheduler and non-work conserving I/O scheduler? In a work-conserving mode, the scheduler must choose one of the pending requests, if any, to dispatch, even if the pending requests are far away from the current disk head position. The rationale for non-work-conserving schedulers, such as the anticipatory scheduler…
As is known, keys may be overlap in SST files of level 0 in LevelDB. I am wondering why it needs to be overlap? An sstable is read-only after being written to the disk from a memtable. Accumulated K/Vs in the log up to a certain size are organized as a memtable and written to…
I keep seeing messages in dmesg as follows with “exception Emask 0x10” -> “SError: { UnrecovData 10B8B BadCRC }” -> “failed command: READ FPDMA QUEUED” -> “hard resetting link”: [ 7395.936692] ata4.00: exception Emask 0x10 SAct 0xe000 SErr 0x280100 action 0x6 frozen [ 7395.936701] ata4.00: irq_stat 0x08000000, interface fatal error [ 7395.936703] ata4: SError: {…
I can only connect to a Linux server by SSH. How to check whether a USB disk is connected to the Linux server? 2 ways I usuaully use: Method One: Check the print out of dmesg After the USB storage is connected and detected by Linux, it may print some messages containing “[sde]” (the new…
On a Linux box, we may allow non-root users mount and umount certain disks, such as /dev/sde1. How to allow non-root users on Linux to mount and unmount disks? Please check the tutorial at Controlling Filesystem Mounting on Linux by Playing with /etc/fstab: Allow non-root users to mount and unmount filesystems.
I prefer Linux Mint (I am using Cinnamon) not automatically mounting a portable storage such as USB disk or iPhone for me and I would like do the mounting manually if I need it. The default behavior of Linux Mint is to automatically mount the disks. How to disable this? The auto mounting on Linux…
As nodes are added and deleted in a Hadoop cluster. Storage usage across DataNodes may be different. Some DataNodes’ disks are almost used up while some others’ are almost empty. How to balance data across DataNodes in HDFS? Hadoop provides the balancer to redistribute the data. Brief introduction to balancer in Hadoop: balancer. The design…
How to erase disk partitions on U disks on Windows? Check the HDD LLF low level format tool: http://hddguru.com/software/HDD-LLF-Low-Level-Format-Tool/ It is free for personal usage. Run it as Adminiatrator and you can choose to erase partitions and MBR on the U disks. Be careful that the operation is is recoverable.
dd is a handy tool on Linux. But is it possible to run it on Windows? I find the dd in cygwin works very well for me. It provides as almost (if not the same) functions as the dd on Linux. The disks are specified in /dev/ as on Linux. This solution means: you need…
VM disks in public cloud may be shared by customers. This is privacy problem. Is there a quick method to sniff the data left on a Linux VM? One possible method: # dd if=/dev/xvda bs=1M | strings -n 100 > strings.txt Using the dd and strings which are pervasive on Linux systems.
This is the scenario: I want to remove a old hard disk which is a LVM PV and contains data. There is free space available on other PVs in the VG. It should move the data from the disk to be removed to other PVs and then remove the disk. The process to remove sdb…
I find one disk failed on my server which have several ones installed. I know the disk’s name in Linux (e.g. sda, sdb). However, the Linux disk name to SATA port mapping does not follow the same order. Now, I want to find out the failed disks. How to quickly find out them and which…
Any tutorials or tips on how to manage LVM? LVM provides a flexible and easy way to management storage disks on Linux. It also provides a set of tools for management. Here are some tutorials and references for managing LVM: LVM man pages:Â https://www.systutorials.com/docs/linux/man/8-lvm/#lbAL Manage Disk Volume in Linux:Â http://www.hongkedavid.com/blog/disk_volume.html LVM with Xen:Â https://www.systutorials.com/b/linux/tag/lvm/ Remove missing phyiscal volumes…
RAID systems such as MegaRAID add signatures to disks to maintain the infomration on these didks. When we simply remove these disks and install them to another server, Linux on the new server may detect these RAID signature infomration and refuses to continue write to the disk. Here is one example that mkfs reports “apparently…
It is common for programs to write to files in filesystems on disks. However, what if the disk was almost full when your program writes to the filesystem on a disk? For systems software and mission critical programs, it is a better or must-to-do practice to check the available filesystem space before actually writing to…
My best favorite disk partition table manipulation tools are cfdisk/fdisk on Linux. However, for large disks, cfdisk/fdisk (of the versions by this post is written) will just give up with a message suggesting GPT partition table format and using GNU parted like WARNING: The size of this disk is 6.0 TB (6001042391040 bytes). DOS partition…
LVM is a great tool to manage hard disks on Linux—you can abstract the hard drives away and manage logical volumes from volume groups, you can dynamically add or remove hard drives while the file systems on the logical volumes need not to backed up and recovered, and you may create many snapshots of the…
Programs may want to catch the kill signals sent by the kill command in C programs on Linux so that it can gracefully shutdown itself before it is killed. For example, I have a daemon progd running, and it may be killed by pkill progd. The progd may want to save some in-memory state to…
The LSI MegaRAID is widely used like in IBM x3620M3 servers. This post introduces the process with screenshots of how to configure the new disks and set up RAID0 in the MegaRAID BIOS for reference by other and myself. It is illustrated with one example by adding a new 2TB hard disk and creating a…