MapReduce is a foundational distributed computing model, but it’s no longer the default choice for new projects. Hadoop’s MapReduce (via YARN in Hadoop 2.x+) remains production-stable and useful for specific scenarios, while Apache Spark, Flink, and cloud-native services handle most modern workloads more efficiently. When MapReduce Still Makes Sense Use MapReduce if you’re: Maintaining existing…
ns (Network Simulator) is a discrete-event simulator used primarily for research and educational purposes in networking. The two major versions—ns-2 and ns-3—are substantially different in architecture and usage patterns. This guide covers installation on current Fedora systems. Installing ns-2 on Fedora Note: ns-2 is legacy software and requires significant compatibility workarounds on modern systems. Consider…
This post analyzes I/O performance between loopback and blktap backends for Xen file-backed virtual block devices (VBDs), based on testing from 2013. While Xen remains in production use (AWS EC2, Citrix Hypervisor), most Linux deployments now default to KVM. For modern virtualization needs, consider KVM/libvirt or container solutions like Docker and Kubernetes. This post is…
Andrew Fikes, Principal Engineer at Google, presented on storage architecture and the fundamental challenges that emerge when operating at scale. The presentation from the 2010 Faculty Summit remains relevant for understanding how distributed storage systems must be designed and the trade-offs inherent in large-scale deployments. Core Storage Challenges at Scale Google’s storage systems had to…
Jeff Dean’s “Designs, Lessons and Advice from Building Large Distributed Systems” is essential reading for anyone building or operating distributed infrastructure. This talk, presented at multiple conferences including LADIS and various tech events, distills decades of experience at Google and now at Google Research. If you’re designing systems that scale beyond a single machine, you…
Modern datacenters rely on distributed storage and processing systems designed to handle massive datasets across clusters of commodity hardware. Understanding these systems is essential for anyone working with infrastructure at scale. Here’s an overview of the major architectures and implementations you’ll encounter. Storage Systems Google File System (GFS) and successors GFS established the foundation for…
Historical Archive: This post lists conferences from 2012. The submission deadlines and venues are no longer current. For up-to-date conference information, check the official conference websites or resources like Conference Alert and CCF Deadlines. I am trying to find out the top conferences that have the largest average number of citations in the last 5…
Hadoop 1.x reached end-of-life in 2014 and is no longer maintained. This post documents a deprecated architecture for historical reference only. For new deployments, use Hadoop 3.x+ with YARN, which offers significant improvements in resource management, multi-tenancy, and reliability. See the Apache Hadoop documentation for current versions. For managed services, consider AWS EMR, Google Dataproc,…
Understanding distributed systems and cloud computing starts with the literature. These papers form the foundation for anyone working in the space — they’re not just historical artifacts, they’re still actively referenced and their design principles remain relevant. Internet-Scale Systems and Datacenters The Google cluster architecture paper remains essential reading. It laid out the practical realities…
Historical Archive: This post lists conferences from 2012. The submission deadlines and venues are no longer current. For up-to-date conference information, check the official conference websites or resources like Conference Alert and CCF Deadlines. This post lists important conferences related to Cloud Computing in year 2013. SOSP 2013 SOSP’13: The 24th ACM Symposium on Operating…
Historical Archive: This post lists conferences from 2011. The submission deadlines and venues are no longer current. For up-to-date conference information, check the official conference websites or resources like Conference Alert and CCF Deadlines. This post lists important conferences on Cloud Computing in year 2012. OSDI 2012 10th USENIX Symposium on Operating Systems Design and…
Archived Content (2010): This post describes a research project from 2010. The tools and versions mentioned (Hadoop 0.20, MapReduce Streaming) are historically significant but have been largely superseded by modern distributed build systems like Bazel Remote Execution and cloud-native CI/CD pipelines. mrcc – A Distributed C Compiler System on MapReduce Original Project Date: January 2010…