OGC considering Zarr as community standard; seeks public comment on new Work Item

A new Work Item justification to begin the Community Standard endorsement process is available for public comment. Comments are due by 11th September, 2020.

Zarr is an open-source specification for the storage of multi-dimensional arrays of data (also known as N-dimensional arrays, ND-arrays, or tensors). Such arrays are ubiquitous in scientific research and engineering.

Zarr stores metadata using .json text files and array data as (optionally) compressed binary chunks. Zarr can store data into most storage systems, including databases, standard ‘directory based’ file systems, and cloud object stores, such as Amazon S3. This flexibility allows implementations to experiment with novel storage technologies while maintaining a uniform API for downstream libraries and users.

Zarr arose in genomics research in 2016. It was created by Alistair Miles of Oxford as a library optimized for massively parallel array analytics. It has since grown into a community project with a range of developers and users from fields such as genomics, bioimaging, astronomy, physics, quantitative finance, oceanography, atmospheric science, climate science, and geospatial imaging.

Because it can represent very large array datasets in a simple, scalable way, and is compatible with cloud object storage, Zarr is an ideal format for analysis-ready geospatial data in the cloud. Indeed, Zarr has already been adopted by several OGC communities as a format for cloud-optimized, analysis-ready geospatial data.