Shell scripting (Bash, awk, sed), Unix utilities, file handling, string manipulation, and command-line tools.
Excel files remain a common data format you’ll encounter. The readxl package handles both .xls and .xlsx formats efficiently without external dependencies, making it the standard choice for Excel import in R. Installation Install the package once per R environment: install.packages(“readxl”) Load it in your session or script: library(readxl) Basic Import The read_excel() function handles…
Perl one-liners are powerful for text processing, but by default the -p and -n options process input line by line. When you need to match patterns across multiple lines, you’ll hit limitations because \s doesn’t match newlines by default, and the record separator is set to \n. The solution is the -0777 flag, which changes…
OpenSSL’s enc command supports passing passphrases programmatically rather than prompting interactively. This is essential for automation—cron jobs, CI/CD pipelines, backup scripts, and batch processing. Basic encryption with environment variables The safest approach is storing the passphrase in an environment variable, keeping it out of process listings and shell history: export PASS=examplepass openssl enc -aes-256-cbc -in…
You don’t need to fully extract a .tar archive just to read one file inside it. The tar command can stream individual files directly to stdout, which saves time and disk space—especially useful for large archives. Basic syntax tar -xOf archive.tar path/to/file.txt The flags break down as: -x: Extract mode -O: Write to stdout instead…
By default, rsync connects as your standard user. When you need to write to protected directories like /etc, /var/www, or other root-owned locations, use the –rsync-path option to invoke rsync with sudo privileges on the remote side. Basic syntax rsync -avz –rsync-path=”sudo rsync” local-file user@remote:/protected/path This tells the remote server to execute rsync under sudo….
For iterating through directories and files in C++, you have several options depending on your needs and C++ standard version. Using <filesystem> (C++17 and Later — Recommended) The modern approach uses std::filesystem, which is cleaner, safer, and handles cross-platform differences for you: #include <filesystem> #include <iostream> #include <string> namespace fs = std::filesystem; int ListDir(const std::string&…
Stripping newlines from strings is common when processing text input, especially from files or network streams. Here are the practical approaches. Using erase() with std::remove() The most efficient method combines std::remove() and erase() — this is the erase-remove idiom: #include <iostream> #include <algorithm> #include <string> int main() { std::string input{“line 1\n\n\nline 3\nline 4”}; std::cout <<…
Sometimes you have a global gitignore rule—like ignoring all *.log files—but need to track one specific exception. Git lets you negate patterns using the ! character. # Ignore all log files *.log # But DO track this one specific file !important.log Pattern matching is evaluated top-to-bottom, and the last matching pattern wins. Your negation rule…
By default, make executes the first target it encounters in your Makefile. This works fine for simple projects, but as Makefiles grow and you have multiple build configurations, deployment targets, and test suites, you’ll need explicit control over which target runs by default. The .DEFAULT_GOAL Variable The cleanest way to change the default target is…
Determining whether a path points to a directory or regular file is a common task in system programming. You need to stat the path and inspect the file mode. Using lstat() and stat macros The lstat() system call retrieves file metadata into a struct stat. For a given path, you can use the standard macros…
You can extract and display a single file’s content from a tar archive without unpacking the entire archive using tar‘s standard output option. Basic Syntax tar -xOf archive.tar.gz path/to/file The key flags are: -x: Extract files from the archive -O (capital O): Write extracted files to standard output instead of disk -f: Specify the archive…
The diff command on Linux is standard, but Windows doesn’t ship with an equivalent by default. You have several solid options depending on whether you prefer command-line or GUI tools. Command-Line Alternatives If you want a command-line experience similar to Linux diff: Windows PowerShell (built-in, Windows 10+): Compare-Object (Get-Content file1.txt) (Get-Content file2.txt) For more diff-like…
Python’s str.split() method splits a string by a separator and returns a list. Here are the common patterns: “a string separated by space”.split() # [‘a’, ‘string’, ‘separated’, ‘by’, ‘space’] “a,string,separated,by,comma”.split(“,”) # [‘a’, ‘string’, ‘separated’, ‘by’, ‘comma’] Note that calling split() without arguments splits on any whitespace and automatically removes empty strings. Handling consecutive separators When…
When you call os.makedirs(path, mode=0o777) in Python, the resulting directory often won’t have the permissions you expect. The mode argument gets masked by the process’s umask, which removes certain permission bits before applying them to the filesystem. Your umask acts as a filter. If your umask is 0o022 (common on many systems), it removes write…
When you access a non-existent key in a Python dictionary, you’ll hit a KeyError. There are several approaches to handle this gracefully depending on your use case. The .get() Method The simplest and most common approach: value = my_dict.get(‘key’, ‘default_value’) This returns the value if the key exists, or the default value if it doesn’t….
By default, Python buffers stdout when it’s not connected to a terminal — typically when you redirect output to a file, pipe to another command, or run inside a container. This buffering can hide output until the buffer fills or the program exits, making debugging and log monitoring difficult. Using the flush parameter The simplest…
The error ValueError: zero length field name in format occurs when using implicit positional arguments in .format() strings with Python versions that don’t support this syntax. The problematic code print(‘== {} SPF’.format(d)) This raises ValueError: zero length field name in format on older Python interpreters. Why this happens The {} syntax without explicit position numbers…
When truncating strings, you often need to avoid cutting off in the middle of a word. The naive substr() approach will break words, so you need logic to backtrack to the previous word boundary. The problem Given a string like “ab cc dde ffg ff”, if you want a substring of maximum 7 characters starting…
When processing text data, you often need to extract everything from a certain field onward. awk makes this straightforward once you understand the mechanics. Basic Syntax The fundamental approach uses awk’s field variables and looping: awk ‘{for(i=N;i<=NF;i++) printf “%s “, $i; print “”}’ file.txt Replace N with the field number you want to start from….
Files with replication factor 1 are a liability in production HDFS clusters. They have no redundancy, meaning a single node failure results in data loss. Identifying and fixing these files should be part of your regular cluster maintenance. Using the HDFS CLI The most straightforward approach is using hdfs fsck with output parsing: hdfs fsck…