ASDF support for various compression algorithms

This packages includes a plugin for the Python library asdf to add support for reading and writing various compression algorithms including:

• Blosc
• LZ4 Frame
• Zstandard

Installation

This plugin is not yet stable and released on PyPi.

To install all compression algorithms supported by asdf-compression, install using the all optional parameters.

$ pip install "asdf-compression[all] @ git+https://github.com/asdf-format/asdf-compression"

Or alternatively by cloning and installing this repository.

$ git clone https://github.com/asdf-format/asdf-compression
$ cd asdf-compression
$ pip install ".[all]"

If only a specific algorithm is needed, just that algorithm can be installed.

$ pip install ".[zstd]"

Or a subset can be selected

$ pip install ".[zstd,blsc]"

Usage

When installed asdf-compression will register any supported and available compression algorithms with asdf using the Compressor interface.

The following example shows saving an array with zstandard compression provided via asdf-compression.

import asdf, numpy as np

af = asdf.AsdfFile({"arr": np.arange(42)})
af.set_array_compression(af["arr"], "zstd")
af.write_to("test.asdf")

Testing

pytest is used for testing. Tests can be run (from the source checkout of this repository):

$ pytest

Contributing

We welcome feedback and contributions to this project.