EarthServer: 1+ Petabyte Analysis-Ready Datacubes

The intercontinental EarthServer initiative unites Europe, US, and Australia in the quest for scalable datacubes on 3D x/y/t satellite image timeseries and 4D x/y/z/t weather data, based on the European datacube technology, rasdaman. After 6 years project running, a particularly spectacular milestone has been reached: operational real-time data analytics on more than a Petabyte of satellite timeseries through open OGC standard interfaces.

EO Data Service (https://eodataservice.org/) has crossed the 1 PB data providing access and exploitation capabilities to the Copernicus Sentinel-2A (launched on 23 June 2015) and Sentinel-2B (launched on 7 March 2017) satellites. The Sentinel 2 archives grow at a rate of 3TB/day and allow the Earth Observation scientists in monitoring variability in land surface conditions", says Simone Mantovani, CEO of MEEO S.R.L.

With the European partners funded by the European Commission, the EarthServer initiative was launched in 2011 to establish a standards-based, highly scalable framework for providing analysis-ready Earth science data. The underlying distinguishing principle is a geospatial datacube analytics language, the OGC Web Coverage Processing Service (WCPS), and scaling its use into the Petascale.

"It was planned like this, but still it excites me to see the Petabyte frontier finally crossed. Building such a datacube allows for new science and applications using both new and old data”, comments EarthServer coordinator, Prof. Dr. Angelo Pio Rossi from Jacobs University.

"And the next challenge is already being tackled", adds Prof. Dr. Peter Baumann, rasdaman architect and Technical Coordinator of EarthServer. "Now we are working closely with the European Centre for Medium Range Weather Forecast (ECMWF) to unleash their 220 PB archive via rasdaman and OGC datacube standards".

Image: EO Data Service: availability of global Sentinel 2A data on May 19th, 2017 on top of MODIS normalize difference vegetation index (NDVI) visualised in the ESA/NASA Web World Wind virtual globe. Source: MEEO S.R.L.