Sparse files are common in Linux/Unix and are also supported by Windows (e.g. NTFS) and macOSes (e.g. HFS+). Sparse files uses storage efficiently when the files have a lot of holes (contiguous ranges of bytes having the value of zero) by storing only metadata for the holes instead of using real disk blocks. They are […]
Posts tagged Programming
337 posts in total.
How to not use concrete types in lambda function parameters in C++11?
C++11 requires that lambda function parameters be declared with concrete types. This is sometimes annoying. auto is really nice, especially when the type is complex like std::vector<std::string>::iterator is quite long to type. I know C++14 allows auto in lambda functions. But how to not use concrete types in lambda function parameters in C++11? In C++11, […]
How to operator[] access element in a const map in C++?
How to operator[] access element in a const map in C++? For example, the compiler will report error on this piece of code: #include <iostream> #include <string> #include <map> std::int64_t count(const std::map<std::string, std::int64_t>& map) { return map[“one”] + map[“two”]; } int main () { std::map<std::string, std::int64_t> map = { {“one”, 1}, {“two”, 2} }; std::cout […]
How to make Vim indent C++11 lambdas correctly?
Vim seems not indent C++11 lambas very well. How to make Vim indent C++11 lambdas correctly? For this following program, Vim indents it as #include <iostream> #include <string> #include <vector> #include <algorithm> int main () { std::vector<std::string> strs({“one”, “two”}); std::vector<std::string> newstrs; std::transform(strs.begin(), strs.end(), std::back_inserter(newstrs), [](const std::string& s) -> std::string { if (s == “one”) { […]
How to remove newline characters from a string in C++?
How to remove newline characters from a string in C++? For example, a string like line 1 line 3 line 4 should be converted to line 1line 3line 4 In short, use this code snippet: input.erase(std::remove(input.begin(), input.end(), ‘\n’), input.end()); std::remove() shifts all elements that are equal to the value n by moving the elements in […]
How to change the default target of `make`?
The default target of make is the first target. But can I change the default target in Makefile and how to change the default target of make? The default goal of make is the first target whose name does not start with ‘.’ if .DEFAULT_GOAL is not set. Ref: make manual. To set the default […]
How to judge whether its STDERR is redirected to a file in a Bash script on Linux?
Within a Bash script, how to judge whether its STDERR is redirected to a file in Bash on Linux? For example, ./script.sh /tmp/log 2>&1 Can script.sh detect that its STDERR is redirected? Knowing the destination file is better. To test whether a script’s STDERR (or STDOUT) is redirected to a file, check by [[ -f […]
How to force a checkpointing of metadata in HDFS?
HDFS SecondaraNameNode log shows 2017-08-06 10:54:14,488 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in doCheckpoint java.io.IOException: Inconsistent checkpoint fields. LV = -63 namespaceID = 1920275013 cTime = 0 ; clusterId = CID-f38880ba-3415-4277-8abf-b5c2848b7a63 ; blockpoolId = BP-578888813-10.6.1.2-1497278556180. Expecting respectively: -63; 263120692; 0; CID-d22222fd-e28a-4b2d-bd2a-f60e1f0ad1b1; BP-622207878-10.6.1.2-1497242227638. at org.apache.hadoop.hdfs.server.namenode.CheckpointSignature.validateStorageInfo(CheckpointSignature.java:134) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:531) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:395) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$1.run(SecondaryNameNode.java:361) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:415) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:357) It seems the checkpoint […]
How to flush STDOUT buffer in Python?
How to flush the STDOUT buffer in Python so that the content wrote to STDOUT is shown immediately? Call the flush library function on sys.stdout which is the STDOUT: import sys sys.stdout.flush() From python doc: flush() Flush the write buffers of the stream if applicable. This does nothing for read-only and non-blocking streams. If you […]
How to tune systems to achieve high performance in virtualization circumstances?
Most time, we need to tune system parameters to achieve better performance but what the general parameters to be tuned in Linux systems. I think you may want to add following parameters to Kernel boot (/etc/default/grub) parameters intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll intel_pstate=disable At the same time, you may also want to shutdown/open Pause Loop Exiting (PLE). […]
How to set one task’s CPU affinity quickly?
How to set one task’s CPU affinity quickly? 1, Get this task’s ID # taskset -pc PID 2, Set this task’s CPU affinity # taskset -c affinity ./task Answered by harryxiyou. There is also another format # taskset -pc affinity pid For example: set process with PID 3783 to CPU 15 # taskset -pc 15 […]
How to convert a piece of HTML code to plain text in PHP?
How to convert a piece of HTML code to plain text without leading and ending spaces in PHP? For example, I would like to convert <div> <b>hello</b> world</div> to a string hello world You may use this piece of code in PHP to strip HTML tags, remove leading and ending spaces and convert special characters […]
How to get the full request URL in PHP?
How to get the full request URL of the page being processed in PHP? If you are sure the request is a http or https one, and the PHP script is executed according to (e.g. by a load balancer or apache reverse proxy) the REQUEST_URI/HTTP_HOST which are set by the client, the PHP script can […]
How to print all fields after a certain field with awk on Linux?
How to print all fields after a certain field with awk on Linux? Say, I want to print out all fields after field $3: a b c d e f a b b b a a c d should be transformed to d e f b d You may use a for loop in awk […]
How to add a prefix string at the beginning of each line in Bash shell script on Linux?
How to add a prefix string at the beginning of each line in Bash shell script on Linux? For example, assume we have a file a.txt: line 1 line 2 I want to have, pre line 1 pre line 2 You may use sed, the stream editor for filtering and transforming text: sed -e ‘s/^/pre […]
How to detect whether a file is being written by any other process in Linux?
How to detect whether a file is being written by any other process in Linux? Before a program open a file to processes it, it wants to ensure no other processes are writing to it. Here, we are sure after the files are written and closed, they will not be written any more. Hence, one-time […]
How to count the number of reads in each chromosome in a bam file?
How to count the number of reads in each chromosome in a bam file? The bam file is already sorted by the chromosome names. If the bam file is indexed, you may quickly get these info from the index: samtools idxstats in.bam | awk ‘{print $1″ “$3}’ If the bam file is not indexed, you […]
How to get the mtime of a file on Linux?
How to get the mtime of a file on Linux from the file’s path? You can use stat to get the file status including the mtime: %y time of last modification, human-readable %Y time of last modification, seconds since Epoch As an example, $ stat -c %y ./file 2017-06-26 13:33:06.764042064 +0800 $ stat -c %Y […]
How to print a line to STDERR and STDOUT in C++?
In C++, how to print a string as a line to STDOUT? That is, the string and the newline character, nicely? And similarly, how to print the line to STDERR? In C++, you may print the string and then ‘n’ or std::endl to STDOUT by operating on the std::cout stream: std::cout << your_string << std::endl; […]
How to print a line to STDERR and STDOUT in OCaml?
In OCaml, how to print a string as a line to STDOUT? That is, the string and the newline character, nicely? And similarly, how to print the line to STDERR? In OCaml, you may print a string with the end line character to STDOUT using the function in the Pervasives module: val print_endline : string […]
