Posts tagged Cloud computing

PUMA: A MapReduce Benchmark Suite

By Eric Z Ma,
MapReduce is a well-known programming model designed for generating and processing large data. There are various MapReduce implementations. One widely known and used one may be Hadoop. Benchmarking MapReduce frameworks gets to be important. Faraz Ahmad et al. developed a benchmark suite: PUMA MapReduce Benchmark. During our work on MapReduce, we developed a benchmark suite which represents a broad range of MapReduce applications exhibiting application characteristics with high/low computation and high/low shuffle volumes. » Read more

Hadoop TeraSort Benchmark

By Eric Z Ma,
TeraSort is one of Hadoop’s widely used benchmarks. Hadoop’s distribution contains both the input generator and sorting implementations: the TeraGen generates the input and TeraSort conducts the sorting. Here, we provide a short tutorial for using the Hadoop TeraSort benchmark. TeraGen generates random data that can be used as input data for a subsequent running of TeraSort. Generate input by TeraGen The syntax for TeraGen: $ hadoop jar hadoop-*examples*.jar teragen <number of 100-byte rows> <output dir> To make the TeraGen run on multiple nodes with multiple tasks, you may need to specify the number of map tasks (30 here as an example; for Hadoop 2): $ hadoop -D mapreduce.job.maps 30 jar hadoop-*examples*.jar teragen <number of 100-byte rows> <output dir> The number of mappers depends on the number of rows you will generate and the number of nodes you have. » Read more

Conferences on Cloud Computing 2013

By Eric Z Ma,
This post lists important conferences related to Cloud Computing in year 2013. SOSP 2013 SOSP’13: The 24th ACM Symposium on Operating Systems Principles. November 3-6, 2013, Nemacolin Woodlands Resort, Pennsylvania.The biennial ACM Symposium on Operating Systems Principles is the world’s premier forum for researchers, developers, programmers, and teachers of computer systems technology. Academic and industrial participants present research and experience papers that cover the full range of theory and practice of computer systems software. » Read more

Hadoop Default Ports

By Eric Z Ma,
Hadoop’s namenode and datanodes expose a bunch of TCP ports used by Hadoop’s daemons to communicate to each other or listen directly to users’ requests. These ports information are needed by both the Hadoop users and cluster administrators to write programs or configure firewalls/gateways accordingly. A post written by Philip Zeyliger from Cloudera’s blog summarizes the default ports used by Hadoop: Hadoop Default Ports Quick Reference. » Read more

Conferences on Cloud Computing 2012

By Eric Z Ma,
This post lists important conferences on Cloud Computing in year 2012. OSDI 2012 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’12) October 8–10, 2012, Hollywood, CA “The tenth OSDI seeks to present innovative, exciting research in computer systems. OSDI brings together professionals from academic and industrial backgrounds in what has become a premier forum for discussing the design, implementation, and implications of systems software.” Important Dates Complete paper submissions: Thursday, May 3, 2012, 9:00 p.m. » Read more

Conferences on Cloud Computing 2011

By Eric Z Ma,
This post lists important conferences on Cloud Computing in year 2011. ACM Symposium on Cloud Computing October 27 and 28, 2011, Cascais, Portugal Submission Deadline: April 30, 2011 23rd ACM Symposium on Operating Systems Principles (SOSP) October 23-26, 2011, Cascais, Portugal Submission deadline: March 18, 2011, 11:59 PM GMT EuroSys 2011 April 10-13, 2011. Salzburg, Austria. CLOUD COMPUTING 2011 The Second International Conference on Cloud Computing, GRIDs, and Virtualization September 25-30, 2011 – Rome, Italy Submission (full paper): April 20, 2011 » Read more