performance

How to choose the number of mappers and reducers in Hadoop

How to choose the number of mappers and reducers in Hadoop to get good job performance? The Hadoop Wiki gives a discussion on this: http://wiki.apache.org/hadoop/HowManyMapsAndReduces Some valuable points: About the number of Maps: The number of maps is usually driven by the number of DFS blocks in the input files. Although that causes people to…

SQL layers on NoSQL databases

What are the SQL layer solution over NoSQL databases such as key/value stores? Phoenix: A SQL layer on HBase: https://github.com/forcedotcom/phoenix They also show some performance results: https://github.com/forcedotcom/phoenix/wiki/Performance F1 – The Fault-Tolerant Distributed RDBMS Supporting Google’s Ad Business: http://research.google.com/pubs/pub38125.html With F1, we have built a novel hybrid system that combines the scalability, fault tolerance, transparent sharding,…

|

I/O Microscopy: Tasks’ Disk I/O Information with High Accuracy

Abstract Most popular task monitor systems (such as top, iotop, proc, etc) can only get tasks’ disk I/O information like tasks’ I/O utilization percentage every seconds due to kernel timer/tick frequency and high time cost of system interfaces. This article presents I/O Microscopy, a new way to get tasks’ disk I/O information with high accuracy….

Make Better Decisions for Your Businesses with Data Visualization

In today’s time, data visualization has become a significant part of the success story of an organization. With the help of right techniques, visualizing data can reveal insights which the management staff can use in their decision-making in order to make sound data-driven decisions. Mapping software is among the robust data visualization tools that you…

| |

x-data-plane feature in QEMU/KVM

Abstract In systems, sometimes, we use one global lock to keep synchronization among different threads. This principle also happens in QEMU/KVM (http://wiki.qemu.org/Main_Page) system. However, this may cause lock contention problem. The performance/scalability of whole system will be decreased. In order to solve this problem in QEMU/KVM, x-data-plane feature is designed/implemented, which the high-level idea is…

|

How migration thread works inside of Linux Kernel

Abstract In computer systems, resources have to be balanced so that the performance will be better based on the same hardware. In Linux Kernel system, we will see some migration kernel threads running as daemons to do this kind of jobs as follows. In this article, we will discuss how Linux Kernel balances its hardware/software…

|

SSD Enabled For DreamHost Shared Hosting: Simple Performance Measurement

SSD is common for VPS and PaaS virtual machines for higher I/O performance. Now, it is coming to shared hosting too. DreamHost states that “Now with solid state drives (SSDs), our standard web hosting loads pages 200% faster”. We ourselves are happy to know this performance improvement with the price kept the same. Good work,…

|

Making GPT Partition Table and Creating Partitions Using parted in Linux

My best favorite disk partition table manipulation tools are cfdisk/fdisk on Linux. However, for large disks, cfdisk/fdisk (of the versions by this post is written) will just give up with a message suggesting GPT partition table format and using GNU parted like WARNING: The size of this disk is 6.0 TB (6001042391040 bytes). DOS partition…

| |

How to Create Fedora 20 Domain-U on Fedora 20 Domain-0

In this post, creating a file-backed virtual block device (VBD) and installing Fedora 20 in the Xen DomU via internet will be introduced. This domain is created on a Fedora 20 Dom0 as introduced in https://www.systutorials.com/installing-xen-on-fedora-20-as-domain-0/. For better performance, you may consider using LVM backed VM. Create file-backed VBD The actual space of VBD will…

|

How to Install, Run and Uninstall VMware Player and VirtualBox on Fedora Linux

VMware Player and VirtualBox are two cool and free full virtualization solutions and both can run on top of a Linux host. In this post, I introduce how to install, run, and uninstall VMware Player and VirtualBox on Fedora Linux. VMware Player Install VMware Player Download the installation bundle from VMware’s website. For example, the…

| | |

Improving ssh/scp Performance by Choosing Suitable Ciphers

Update on Oct. 9, 2014: You should be aware of the possible security problems of blowfish and it is suggested not to be used. Instead, you may consider ChaCha20 as suggested by Tony Arcieri. To use this with OpenSSH, you need to specify the Ciphers in your .ssh/config files as chacha20-poly1305@openssh.com possibly with another default…