Reading compressed LZMA files on-the-fly using a custom C++ std::streambuf

DRANK

IntroductionHandling big text files can be quite demanding in terms of disk space and bandwidth if these files are frequently shared via network; especially when the total amount of data exceeds several Tera-Bytes.A workaround provides the LZMA compression which yields very good compression rates for text files (in a current case, it reduces the file size to about 25% of the original size). Simultaneously it also provides relatively fast decompression and is thus eligible for on-the-fly decompression which is necessary if the data needs to be accessed frequently.To access LZMA compressed files, I use liblzma (XZ utils, which is usually distributed on linux OS but it is also available for Windows.In this tip I present a customized std::streambuf which can be used in conjuction with a std::istream to read data from a LZMA compressed file on-the-fly. The code is tested on linux with GCC 6.4.0 and liblzma 5.2.2.Using the codeMinimum exampleAs an example, the following snippet opens the L…