Master Linux administration, command-line tools, system configuration, and single-machine systems. Explore our full academy at: Linux & Systems Administration Academy.
Modern datacenters rely on distributed storage and processing systems designed to handle massive datasets across clusters of commodity hardware. Understanding these systems is essential for anyone working with infrastructure at scale. Here’s an overview of the major architectures and implementations you’ll encounter. Storage Systems Google File System (GFS) and successors GFS established the foundation for…
Cosmos is Microsoft’s internal petabyte-scale distributed data storage and query system, built to handle massive analytical workloads across their infrastructure. While Microsoft hasn’t published formal technical papers on Cosmos, information has surfaced through conference talks and engineering blogs over the years. Architecture and Scale Cosmos operates at truly massive scale: Storage capacity: Approximately 62 physical…
Hadoop 1.x reached end-of-life in 2014 and is no longer maintained. This post documents a deprecated architecture for historical reference only. For new deployments, use Hadoop 3.x+ with YARN, which offers significant improvements in resource management, multi-tenancy, and reliability. See the Apache Hadoop documentation for current versions. For managed services, consider AWS EMR, Google Dataproc,…
Gitosis provides a lightweight way to manage Git repositories and user access on a self-hosted server. This post covers user management, repository configuration, and access control. For initial server setup, refer to your Gitosis installation documentation. Adding a New User Each user needs an SSH key pair for authentication. The user generates their key locally:…
Gitosis is unmaintained and shouldn’t be used for new deployments. Use Gitolite or Forgejo instead. This guide covers Gitolite, which provides fine-grained access control, is actively maintained, and handles multiple repositories under a single system user. Prerequisites A server running Linux (Debian, Ubuntu, CentOS, or similar) SSH access to the server with sudo privileges A…
Archived Content (2010): This post describes a research project from 2010. The tools and versions mentioned (Hadoop 0.20, MapReduce Streaming) are historically significant but have been largely superseded by modern distributed build systems like Bazel Remote Execution and cloud-native CI/CD pipelines. mrcc – A Distributed C Compiler System on MapReduce Original Project Date: January 2010…