RPM Fusion provides software that the Fedora Project doesn’t ship—primarily due to licensing restrictions, patent concerns, or proprietary redistributability requirements. The repository splits into two branches: free: Open Source Software that Fedora cannot distribute due to other legal or policy reasons nonfree: Non-open source software, including proprietary drivers and restricted-use applications Enable both repositories on…
Google Apps accounts can connect to Google Talk through XMPP, and Pidgin handles this well. Here’s how to configure it properly. Adding the Account Open Pidgin and create a new account. You’ll configure it across two tabs. Basic Tab Settings: Setting Value Protocol XMPP Username eric (local part of your email) Domain example.com (your domain…
Xen remains a production hypervisor for enterprise deployments (AWS EC2, Citrix Hypervisor), though KVM has become the default for most Linux distributions. If you’re deploying Xen in 2026, you’ll likely be working with recent stable releases rather than the legacy versions covered in older guides. This guide covers building a modern Xen Dom0 environment on…
The mailx (or s-nail on modern distributions) command remains useful for scripted email delivery, particularly when you need to send alerts or notifications automatically. Gmail’s SMTP service works well for this, though it requires TLS authentication and specific configuration. Prerequisites Most Linux distributions include mailx or s-nail by default. On systems using s-nail (Ubuntu 20.04+,…
Configuring proper Chinese character rendering in Fedora requires attention to font selection and system settings. This guide covers configuration for contemporary Fedora/RHEL systems where you run an English locale but need reliable Chinese character display. Font Selection You have several solid open-source options for Chinese fonts. Each has different characteristics suited to different use cases….
Customizing your desktop environment to match macOS aesthetics remains popular on Linux, though the approach differs significantly from GNOME 2 era configurations. Modern desktop environments like GNOME (40+) and KDE Plasma (5/6) use different theming systems, but achieving an Apple-inspired look is still straightforward. Current Desktop Environment Options GNOME 40+ uses CSS-based theming and GTK…
Andrew Fikes, Principal Engineer at Google, presented on storage architecture and the fundamental challenges that emerge when operating at scale. The presentation from the 2010 Faculty Summit remains relevant for understanding how distributed storage systems must be designed and the trade-offs inherent in large-scale deployments. Core Storage Challenges at Scale Google’s storage systems had to…
Jeff Dean’s “Designs, Lessons and Advice from Building Large Distributed Systems” is essential reading for anyone building or operating distributed infrastructure. This talk, presented at multiple conferences including LADIS and various tech events, distills decades of experience at Google and now at Google Research. If you’re designing systems that scale beyond a single machine, you…
TeraSort is Hadoop’s standard benchmark for measuring distributed sorting performance across clusters. It consists of three components: TeraGen generates random input data, TeraSort performs the distributed sort, and TeraValidate confirms the output is globally sorted. Modern Hadoop deployments typically run on Hadoop 3.x or higher, with improvements in YARN resource management, better performance tuning options,…
Modern datacenters rely on distributed storage and processing systems designed to handle massive datasets across clusters of commodity hardware. Understanding these systems is essential for anyone working with infrastructure at scale. Here’s an overview of the major architectures and implementations you’ll encounter. Storage Systems Google File System (GFS) and successors GFS established the foundation for…
Colossus is Google’s successor to the Google File System (GFS), designed to handle the massive scale of data storage across Google’s infrastructure. While GFS was documented in detail in the 2003 SOSP paper, Colossus information remains sparse in academic literature—much of what we know comes from talks and interviews with Google engineers. Colossus serves as…
Hadoop 1.x reached end-of-life in 2014 and is no longer maintained. This post documents a deprecated architecture for historical reference only. For new deployments, use Hadoop 3.x+ with YARN, which offers significant improvements in resource management, multi-tenancy, and reliability. See the Apache Hadoop documentation for current versions. For managed services, consider AWS EMR, Google Dataproc,…
Understanding distributed systems and cloud computing starts with the literature. These papers form the foundation for anyone working in the space — they’re not just historical artifacts, they’re still actively referenced and their design principles remain relevant. Internet-Scale Systems and Datacenters The Google cluster architecture paper remains essential reading. It laid out the practical realities…
Historical Archive: This post lists conferences from 2012. The submission deadlines and venues are no longer current. For up-to-date conference information, check the official conference websites or resources like Conference Alert and CCF Deadlines. This post lists important conferences related to Cloud Computing in year 2013. SOSP 2013 SOSP’13: The 24th ACM Symposium on Operating…
Hadoop daemons communicate over specific TCP ports that you’ll need to know for cluster setup, firewall configuration, and application development. These ports are configurable but come with sensible defaults in Hadoop 3.x. HDFS Ports Service Port Configuration Property Purpose Namenode HTTP UI 9870 dfs.namenode.http-address Web UI and metadata operations Namenode HTTPS UI 9871 dfs.namenode.https-address Secure…
Running benchmarks after a Hadoop installation helps validate that your cluster works correctly and gives you baseline performance metrics. The Sort benchmark is a standard test that exercises the MapReduce execution engine and HDFS, making it useful for stress-testing both the distributed storage layer and computation framework. What the Sort Benchmark Does The Sort benchmark…
Hadoop 3.x remains the current stable version with YARN improvements, HDFS federation, and erasure coding. Hadoop 1.x and 2.x are no longer maintained. For new deployments, evaluate whether Hadoop fits your needs or if managed services like AWS EMR, Google Dataproc, or Azure HDInsight better suit your infrastructure. This post covers common configuration mistakes and…
Hadoop’s default configuration runs in standalone (local) mode as a single JVM process — no HDFS daemons, no YARN, no distributed overhead. This mode is useful for development, testing MapReduce jobs, and debugging before moving to a clustered deployment. For new deployments, evaluate whether you need Hadoop at all. Consider cloud-native alternatives like AWS EMR,…
Archived Content (2010): This post describes a research project from 2010. The tools and versions mentioned (Hadoop 0.20, MapReduce Streaming) are historically significant but have been largely superseded by modern distributed build systems like Bazel Remote Execution and cloud-native CI/CD pipelines. mrcc – A Distributed C Compiler System on MapReduce Original Project Date: January 2010…