Optimize the record-level RLE algorithm for a denser compression of shorter-than-declared strings and sets of subsequent NULLs [CORE4401] #4723
Description
Submitted by: @dyemanov
Votes: 3
The RLE algorithm used by the engine compresses sets of up to 128 equal bytes into two bytes. This means that every empty (or NULL) VARCHAR(32000) is compressed into 500 bytes. This storage overhead is too big. The same issue applies to many shorter fields that are all NULLs -- they along with their padding also represent the long chain of zero bytes that cannot be compressed more efficiently than with a 64x ratio.