For Dell PowerEdge T630 server, if you install latest Ubuntu desktop version (16.04) or (14.04), you will get a very slow GUI (X-window). $ inxi -G Graphics: Card: Matrox Systems G200eR2 X.org: 1.15.1 driver: vesa tty size: 205×58 Advanced Data: N/A out of X For Ubuntu 14.04.1, this problem can be solved by following steps….
PARSEC is the most important CPU-bound benchmark for systems. It is huge and hard to install because it needs lots of 3-part libs. PARSEC download link for 3.0 version: http://parsec.cs.princeton.edu/download.htm#parsec I remembered I added the answer yesterday night but I could not see the answer currently. Anyway, let me add the answer again after I…
How to configure Linux (am using Fedora 21) managed by systemd to boot to console (init 3) mode? systemd has the concept of targets as a more flexible replacement for runlevels in sysvinit. Runlevel 3 is emulated by multi-user.target. runlevel3.target is a symbolic link to multi-user.target. You can switch to ‘runlevel 3’ by running #…
One of HDFS cluster’s hdfs dfsadmin -report reports: Under replicated blocks: 139016 Blocks with corrupt replicas: 9 Missing blocks: 0 The “Under replicated blocks” can be re-replicated automatically after some time. How to handle the missing blocks and blocks with corrupt replicas in HDFS? Understanding these blocks A block is “with corrupt replicas” in HDFS…
After a node failure and restarting the HDFS, the NameNode reports: “The reported blocks 1968810 needs additional 5071 blocks to reach the threshold 0.9990 of total blocks 1975856. Safe mode will be turned off automatically.” in the log. Why this happens? And how to fix it? About why the NameNode stays in the safe mode:…
What suggested web browser in command line? This will be very useful for browsing the Internet from a command line via SSH. You may use Lynx, a fully-featured World Wide Web (WWW) client for users running cursor-addressable, character-cell display devices, in Linux. Homepage: http://lynx.isc.org/ Lynx is a fully-featured World Wide Web (WWW) client for users…
In hadoop I need to skip mapper function and directly execute the reducer function. We doing this to improve hadoop performance, if the hadoop framework is used to analyze same data sets, then mapper’s output will be same for different kind of jobs. To save the redundant computation for same results, I am planning to…
dd seems slow when I use command like # dd if=/dev/sda2 of=./sda2.bak How to make it faster? You can make dd faster by specifying a good bs like # dd if=/dev/sda2 of=./sda2.bak bs=8192 8192 is a magic number. There are may be other good sizes for bs for different systems. But 8192 works pretty well…
When we design a system, we may want our systems to be consistency, scalability and so on. Currently, there are some famous consistency algorithms. How to understand them easily. 1, Paxos and its extensions 2, Replicated State Machine mechanisms 3, Quorum Welcome to adding other famous consistency algorithms and its understanding ;-) Reading text books…
Differences beween DDL (Data Definition Language) and DML (Data Manipulation Language) Data Definition Language (DDL) statements are used to define the database structure or schema. Data Manipulation Language (DML) statements are used for managing data within schema objects. References:http://www.orafaq.com/faq/what_are_the_difference_between_ddl_dml_and_dcl_commands
Currently, I wanna take Shark’s (Spark SQL) DDL and DML as an reference to design/implement SQLE’s DDL and DML. However, I cannot find its DDL and DML. I can only find several SQLs in Shark paper[1]. [1] shark paper – http://tab.d-thinker.org/showthread.php?tid=2585 Shark’s language is Hive QL. HQL’s DDL and DML can be found at Hive…
As is known, database schema is a collection of database rules like data type and data relationship. However, this is still a little abstract. So what is database schema in one sentence everyone could understand it clearly. The wikipedia page on Database schema is quite good for understanding the concept. A database schema of a…
How to list all files from a .deb package like rpm -ql on RPM based systems? Use this command dpkg -L pkg_name -L lists files: -L, –listfiles package-name… List files installed to your system from package-name. More on the dpkg command, check dpkg manual.
What is the design of Snapshots in HDFS? This PDF documents the design of snapshot. Jing Zhao and Tsz-Wo Sze from Hortonworks gave a great talk on the design of HDFS snapshots. The slides can be downloaded at here. The development of snapshot is tracked by HDFS-2802.
Some important concepts in distributed system like Hadoop distributed file system, Google file system and so on. Answer from http://www.quora.com/Whats-the-difference-between-Reliability-Durability-and-Availability-for-data-storage-system The difference between durability and availability is fairly simple. Durability is about what happens when all power goes out everywhere. Has all data been written to stable storage that doesn’t require power (e.g. disk/flash), in…
How to backup iPhone on Linux? Not using iTunes or Windows. You can backup iPhone using libimobiledevice. By the time I replied here, it supports up to iOS 7. Please check this post for a tutorial. Actually, the methods introduced can be used on Linux systems with the libimobiledevice installed. Backup iPhone: idevicebackup2 backup ~/iPhoneBackups/…
The disk management tools of Windows can adjust it to some level. But there are more space available as far as I can tell. How to further adjust the system partition (C:) size of Windows? You may check these tools: EASEUS Partition Master (free) Includes Partition Manager, Disk & Partition Copy Wizard and Partition Recovery…
How to upgrade Fedora 19 to Fedora 20 through the network? The tool for Fedora to upgrade the distribution is FedUp. Check this page for how to upgrade Fedora 19 to Fedora 20: http://fedoraproject.org/wiki/FedUp#How_Can_I_Upgrade_My_System_with_FedUp.3F Be sure to check the bugs in Fedora 20 before the upgrading: http://fedoraproject.org/wiki/Common_F20_bugs#Upgrade_issues Generally, it contains 3 steps: Preparation # yum…
VM disks in public cloud may be shared by customers. This is privacy problem. Is there a quick method to sniff the data left on a Linux VM? One possible method: # dd if=/dev/xvda bs=1M | strings -n 100 > strings.txt Using the dd and strings which are pervasive on Linux systems.
About caching system at Facebook. According to: https://www.facebook.com/notes/facebook-engineering/monitoring-cache-with-claspin/10151076705703920 Facebook has two major cache systems: Memcache, which is a simple lookaside cache with most of its smarts in the client, and TAO, a caching graph database that does its own queries to MySQL. The NSDI’13 paper introduces more about Memcache: https://www.usenix.org/conference/nsdi13/scaling-memcache-facebook The USENIX ATC’13 paper introduces…