How to choose the number of mappers and reducers in Hadoop to get good job performance? The Hadoop Wiki gives a discussion on this: http://wiki.apache.org/hadoop/HowManyMapsAndReduces Some valuable points: About the number of Maps: The number of maps is usually driven by the number of DFS blocks in the input files. Although that causes people to…
The –no-print-directory option of make tells make not to print the message about entering and leaving the working directory. However, how to specify the –no-print-directory inside the Makefile itself? Add this line to the Makefile: MAKEFLAGS += –no-print-directory You can also set MAKEFLAGS in a makefile, to specify additional flags that should also be in…
Facebook uses lots MySQL databases. Any information about how Facebook scales MySQL? Some information on the Web: MySQL at Facebook’s page https://www.facebook.com/MySQLatFacebook?filter=1 A post by Ryan Thiessen, Database Operations at Facebook on Quora: http://www.quora.com/Facebook-Engineering/How-does-Facebook-structure-MySQL-so-that-it-is-robust-and-scalable And more: http://mashable.com/2011/12/15/facebook-timeline-mysql/ http://gigaom.com/2011/12/06/facebook-shares-some-secrets-on-making-mysql-scale/ http://www.wired.com/wiredenterprise/2011/12/facebook-timeline-anatomy “A lot of people are surprised that for this shiny new thing for Facebook, we’re using…
For example, I want to spawn many ssh background processes in one bash script: for i in `cat ./all-hosts` do ssh $i “ifconfig | grep Link” done Simply adding a & to the ssh commands does not work. Here is the script that works: for i in `cat ./all-hosts` do ssh $i “ifconfig | grep…
How to force Linux to reboot when the reboot command does not work. Enable the use of the magic SysRq option: # echo 1 > /proc/sys/kernel/sysrq Reboot the machine: # echo b > /proc/sysrq-trigger Even if you could not log on the system but sshd is working, you can force the Linux to reboot by:…
I find Chrome on my iPhone caches pages/css/js files. That makes browsing faster. However, under some situation, I’d like to force it refresh the pages/css/js and give me the latest one. How to force Chrome on iPhone refresh pages and invalidate the caches? The only method as far as I can find is to clear…
I find there are too many TCP segments retransmited in a Xen virtual machine. sudo netstat -s | grep segments 537559 segments received 558908 segments send out 3533 segments retransmited 2677 bad segments received. There are 4 VMs on a host which has 4 cores. The segment retransmision rate is too high. It finally turns…
About caching system at Facebook. According to: https://www.facebook.com/notes/facebook-engineering/monitoring-cache-with-claspin/10151076705703920 Facebook has two major cache systems: Memcache, which is a simple lookaside cache with most of its smarts in the client, and TAO, a caching graph database that does its own queries to MySQL. The NSDI’13 paper introduces more about Memcache: https://www.usenix.org/conference/nsdi13/scaling-memcache-facebook The USENIX ATC’13 paper introduces…
This tutorial introduces how to install Scala 2.9.2 on 64-bit Fedora Linux 17. There are some changes needed to make scala work on Fedora Linux 17. Please check it out in the answer. Install Java Scala depends on Java, so please install Java first. In this tutorial, we installed JDK to: /usr/java/jdk1.6.0_24/lib/. Install Scala from…
This tutorial introduces how to exclude certain repositories in yum. Excluding certain repositories is needed usually. One example may be when you have several repositories that have the same package and you only want the one from a certain repository. For my situation, I choose to exclude the google-chrome and other repositories from Google when…
How to install Scala from the official Scala distribution? This is needed on a Linux release with older version of Scala in the repository, e.g. Fedora 12. Use the install-scala.sh script: # wget https://raw2.github.com/zma/usefulscripts/master/script/install-scala.sh # sh ./install-scala.sh VER where VER is the scala version that you want to install. First step, install and configure the…
This is a tutorial on “how to delete all topics and posts from a user/spammer in myBB”. Please check out the answer. I use the Goodbye Spammer plugin for MyBB. First, install it after downloading the plugin form its homepage: Upload ./inc/plugins/goodbyespammer.php to ./inc/plugins/ Upload ./inc/languages/english/goodbyespammer.lang.php to ./inc/languages/english/ Go to ACP > Plugins > Activate…
There is a “Include full text in feeds:” option in the “RSS feeds” configuration panel but no options/method to only show excerpt instead of the “full text” or nothing for RSS feeds. This need to hack the question2answer source code. Details are in the answer. The changes based on Question2asnwer 1.5.4: diff –git a/qa-include/qa-index.php b/qa-include/qa-index.php…
How to get the native assembly code (e.g. x86-64 asm) for OCaml code generated by the native ocamlopt compiler? To get the assembly code for an OCaml program, add these parameters to ocamlopt: -S -inline 20 -nodynlink An example is as follows. The OCaml program: $ cat not.ml let not x = ((x – 1)…
After a server crash and restarting, MyBB reports a SQL Error as follows: MyBB SQL Error MyBB has experienced an internal SQL error and cannot continue. SQL Error: 145 – Table ‘./mybb/mybb_sessions’ is marked as crashed and should be repaired Query: SELECT * FROM mybb_sessions WHERE sid=’40021925bd0494ea31…’ AND ip=’x.x.x.x’ LIMIT 1 The dababase is MySQL….
Plain text file pipelined to Linux mailx turns to “Content-Type: application/octet-stream” which is recognized as an attachment by some email client. The command is like this: $ cat log.txt | mail -s “Updated log file” -r “from@example.com” “to@example.com” I expect it to be: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit But it turns out to be: Content-Type:…
I know the routing tables on Linux is in memory after being set. However, where are the routing table entries stored on disk? I mean where are the routing table is persistently stored so that the routing table can be reloaded like the iptables (under /etc/sysconfig/iptables on Fedora/RHEL/CentOS Linuxes). If the system uses the /etc/rc.d/init.d/network…
How to save a Word document to be a PDF in Office 2007? I find this add-on for Office 2007 from Microsoft: 2007 Microsoft Office Add-in: Microsoft Save as PDF or XPS. It works very well for me. After installing this plugin, I can open a Word document and save it as a PDF file.
How to use OCaml as a script language like bash and python? It will be good that I can write a script in OCaml and directly invoke it. For example, I write a script named “script.ml” and then I can invoke it directly by $ ./script.ml Thanks! Three helloworld scripts: script3.ml #! /usr/bin/env ocaml print_string…
I can list all local branches by git branch How to list a git repository’s all branches on the remote server? You need the -r option: git branch -r From the man page of git-branch. -r List or delete (if used with -d) the remote-tracking branches.