On 29 October, around 80 people filled the Zurich office of Meta for the cloud-native geo community gathering “CNG Zürich”1, announced here in September. The sentence heard most often: “files in a bucket”. A close second: “we all only work with files”.
From files to infrastructure
The recipe is simple: put data into object storage, in formats that can be read in parts, and let others build on top. Oliver Wipfli showed how far this carries with Mapterhorn, which aggregates open terrain and aerial imagery from government platforms (swisstopo’s swissALTI3D among them) and ships it as PMTiles2. Getting hold of the source data in bulk was the painful part, a point for us data providers to take home. In my talk3, “From Tiles to Trust” (slides), I described that provider side: two years of serverless pipelines that publish Sentinel-2 and aerial imagery as COGs4 in rapid mapping to a national STAC catalogue5 within hours of acquisition.
Nissim Lebovits took the idea a step further with Portolan, an open specification and toolkit for running a spatial data infrastructure as files in a bucket. It is only metadata, the bytes never leave the bucket, and the bucket can be anywhere. Portolan also helps non-experts get their data into STAC, and agents featured prominently (there is a skill for that). And for GeoParquet to PMTiles the new and fastest kid on the block is: tylertoo. One challenge was named plainly: secure your bucket. For a taste of how government data could look this way, see this proof of concept for Dutch geodata (not affiliated with the Dutch authorities).
Industry is already moving: Jan Winkler explained how Swiss Re went from a lengthy desktop GIS update cycle for geocoding in catastrophe modelling to automated cloud-native pipelines. In a corporate environment, he noted, “you don’t look for the best solution but for the best available one”, and cloud-native formats brought transparency and speed.
Scale, embeddings and agents
The formats keep up with the ambition. Federico Oldani presented Terrazarr, an open-source library for lazy, resumable GeoZarr6 pyramids, used to build the global 10 m raster behind the WSF Tracker in about three hours.
Gjore Milevski ran SQL against a 3D Zarr cube in the browser, a really nice way to interact with Zarr, and Pirmin Kalberer showed that cloud-optimised formats are more or less ready for the Equal Earth projection (see also Ben Joffe’s map) – equal-area when zoomed out and Mercator when zoomed in.
Embeddings7 (see also this primer) were the other big thread. Tom a.k.a. Hutch Ingold of Earth Genome showed Earth Index, which finds look-alikes in Sentinel-2 imagery. Its engine, Lancelot (a Rust service on LanceDB), searches about 7 billion embeddings in tens of milliseconds on a single node with plain object storage, with reported cost savings of around a factor of six. The readout of the second Geo-Embeddings Sprint (27 and 28 October at IBM Zürich) pointed to the next steps: standards, evaluation and adoption.
Michael DeWitt of Google asked whether Earth Engine is cloud-native, framed AI as an abstraction layer that tames its complexity, and pointed to DatensEE, a demo for parallel exports into COGs in a bucket. His warning: it is risky to be on a tech island.
Numbers over maps
All of this serves an end. Stephan Heuel of Resilens put it neatly: “data and maps are not the product, decision-grade numbers are” (slides). Giorgio Basile of VIDA made a similar case with the DRE Atlas, which turns building data into settlement-level energy access statistics for 58 countries, published as GeoParquet, PMTiles and STAC on Source Cooperative.
Panel and takeaway
The panel agreed that we should understand user needs8 and invest in the community instead of creating yet another format (of course, xkcd 927 was mentioned). There was some controversy about the word “cloud”: publicly it tends to suggest a lack of sovereignty, whereas files in a bucket can live in any bucket, with any provider, in any jurisdiction. Worth more attention.
I liked the spirit, the crowd and the openness with which thoughts were shared. My bet: Portolan is the next thing, and visualisation gets easier with it. Thanks to Jed Sundwall for bringing the event to Zurich and for reaching out to the #SwissGIS community, and to Meta for hosting. The full programme is online.
Footnotes
“CNG” stands for “cloud-native geospatial”. “Cloud-native” means taking full advantage of the cloud computing model; in the data world typically through enabling partial downloads of file contents through so-called HTTP range requests.↩︎
PMTiles is an open, single-file archive format for map tile pyramids, designed to be read directly from object storage via HTTP range requests, without a tile server.↩︎
Obvious transparency note: I gave a talk at the event and took part in the panel discussion. I work at swisstopo, whose pipelines were the subject of my talk.↩︎
Cloud-optimized GeoTIFFs.↩︎
STAC stands for “SpatioTemporal Asset Catalog”, a modern, open specification for cataloging spatio-temporal data.↩︎
GeoZarr is a specification, developed within the OGC, that adds geospatial conventions such as coordinate reference systems and multiscale pyramids to Zarr.↩︎
A geospatial embedding is a compact, high-dimensional vector representation of a geospatial location, scene, or feature, learned so that similar places or patterns end up close together in vector space. See this primer.↩︎
The GOV UK Design principles – almost 15 years old – are still the best start for user-centred data publishing pipelines.↩︎