It is common that the root disk space is not enough when running a Virtual Machine in the cloud such as Amazon Web Service (AWS). The cloud storage usually provides tools or facilities to enlarge a virtual disk size. However, to make the Linux recognize and and use the enlarged disks without rebooting the OS,
Read more
Tag: disks
What is the difference between work conserving I/O scheduler and non-work conserving I/O scheduler?
Posted onWhat is the difference between work conserving I/O scheduler and non-work conserving I/O scheduler? In a work-conserving mode, the scheduler must choose one of the pending requests, if any, to dispatch, even if the pending requests are far away from the current disk head position. The rationale for non-work-conserving schedulers, such as the anticipatory scheduler
Read more
Why are keys overlap in SST files at Level 0 in LevelDB system?
Posted onAs is known, keys may be overlap in SST files of level 0 in LevelDB. I am wondering why it needs to be overlap? An sstable is read-only after being written to the disk from a memtable. Accumulated K/Vs in the log up to a certain size are organized as a memtable and written to
Read more
What’s wrong with the messages like “UnrecovData 10B8B BadCRC” and “failed command: READ FPDMA QUEUED” on Linux?
Posted onI keep seeing messages in dmesg as follows with “exception Emask 0x10” -> “SError: { UnrecovData 10B8B BadCRC }” -> “failed command: READ FPDMA QUEUED” -> “hard resetting link”: [ 7395.936692] ata4.00: exception Emask 0x10 SAct 0xe000 SErr 0x280100 action 0x6 frozen [ 7395.936701] ata4.00: irq_stat 0x08000000, interface fatal error [ 7395.936703] ata4: SError: {
Read more
How to allow non-root users on Linux to mount and unmount disks?
Posted onOn a Linux box, we may allow non-root users mount and umount certain disks, such as /dev/sde1. How to allow non-root users on Linux to mount and unmount disks? Please check the tutorial at Controlling Filesystem Mounting on Linux by Playing with /etc/fstab: Allow non-root users to mount and unmount filesystems.
How to check whether a USB disk is connected to a Linux server?
Posted onI can only connect to a Linux server by SSH. How to check whether a USB disk is connected to the Linux server? 2 ways I usuaully use: Method One: Check the print out of dmesg After the USB storage is connected and detected by Linux, it may print some messages containing “[sde]” (the new
Read more
How to disable auto mounting on Linux Mint Cinnamon?
Posted onI prefer Linux Mint (I am using Cinnamon) not automatically mounting a portable storage such as USB disk or iPhone for me and I would like do the mounting manually if I need it. The default behavior of Linux Mint is to automatically mount the disks. How to disable this? The auto mounting on Linux
Read more
How to balance DataNode storage in HDFS?
Posted onAs nodes are added and deleted in a Hadoop cluster. Storage usage across DataNodes may be different. Some DataNodes’ disks are almost used up while some others’ are almost empty. How to balance data across DataNodes in HDFS? Hadoop provides the balancer to redistribute the data. Brief introduction to balancer in Hadoop: balancer. The design
Read more
How to erase disk partitions on U disks on Windows?
Posted onHow to erase disk partitions on U disks on Windows? Check the HDD LLF low level format tool: http://hddguru.com/software/HDD-LLF-Low-Level-Format-Tool/ It is free for personal usage. Run it as Adminiatrator and you can choose to erase partitions and MBR on the U disks. Be careful that the operation is is recoverable.
How to dd on Windows?
Posted ondd is a handy tool on Linux. But is it possible to run it on Windows? I find the dd in cygwin works very well for me. It provides as almost (if not the same) functions as the dd on Linux. The disks are specified in /dev/ as on Linux. This solution means: you need
Read more
Quick method to sniff the data left on a Linux VM in the public cloud
Posted onVM disks in public cloud may be shared by customers. This is privacy problem. Is there a quick method to sniff the data left on a Linux VM? One possible method: # dd if=/dev/xvda bs=1M | strings -n 100 > strings.txt Using the dd and strings which are pervasive on Linux systems.
How to delete a disk from a LVM group while keeping the data
Posted onThis is the scenario: I want to remove a old hard disk which is a LVM PV and contains data. There is free space available on other PVs in the VG. It should move the data from the disk to be removed to other PVs and then remove the disk. The process to remove sdb
Read more
How to quickly find out failed disks’ SATA port in Linux? (how to map Linux disk names to SATA ports)
Posted onI find one disk failed on my server which have several ones installed. I know the disk’s name in Linux (e.g. sda, sdb). However, the Linux disk name to SATA port mapping does not follow the same order. Now, I want to find out the failed disks. How to quickly find out them and which
Read more
Managing LVM
Posted onAny tutorials or tips on how to manage LVM? LVM provides a flexible and easy way to management storage disks on Linux. It also provides a set of tools for management. Here are some tutorials and references for managing LVM: LVM man pages:Â https://www.systutorials.com/docs/linux/man/8-lvm/#lbAL Manage Disk Volume in Linux:Â http://www.hongkedavid.com/blog/disk_volume.html LVM with Xen:Â https://www.systutorials.com/b/linux/tag/lvm/ Remove missing phyiscal volumes
Read more
How to Clean RAID Signatures on Linux
Posted onRAID systems such as MegaRAID add signatures to disks to maintain the infomration on these didks. When we simply remove these disks and install them to another server, Linux on the new server may detect these RAID signature infomration and refuses to continue write to the disk. Here is one example that mkfs reports “apparently
Read more
How to Get Available Filesystem Space on Linux: a C Function with a C++ Example
Posted onIt is common for programs to write to files in filesystems on disks. However, what if the disk was almost full when your program writes to the filesystem on a disk? For systems software and mission critical programs, it is a better or must-to-do practice to check the available filesystem space before actually writing to
Read more
Making GPT Partition Table and Creating Partitions Using parted in Linux
Posted onMy best favorite disk partition table manipulation tools are cfdisk/fdisk on Linux. However, for large disks, cfdisk/fdisk (of the versions by this post is written) will just give up with a message suggesting GPT partition table format and using GNU parted like WARNING: The size of this disk is 6.0 TB (6001042391040 bytes). DOS partition
Read more
Extending Mounted Ext4 File System on LVM in Linux
Posted onLVM is a great tool to manage hard disks on Linux—you can abstract the hard drives away and manage logical volumes from volume groups, you can dynamically add or remove hard drives while the file systems on the logical volumes need not to backed up and recovered, and you may create many snapshots of the
Read more
How to Catch the Signal Sent by Kill in C on Linux
Posted onPrograms may want to catch the kill signals sent by the kill command in C programs on Linux so that it can gracefully shutdown itself before it is killed. For example, I have a daemon progd running, and it may be killed by pkill progd. The progd may want to save some in-memory state to
Read more
How to Set Up RAID0 in MegaRAID BIOS with New Disks
Posted onThe LSI MegaRAID is widely used like in IBM x3620M3 servers. This post introduces the process with screenshots of how to configure the new disks and set up RAID0 in the MegaRAID BIOS for reference by other and myself. It is illustrated with one example by adding a new 2TB hard disk and creating a
Read more