Skip to content

Release notes

0.2.2

Changed

  • Each level group now carries its own spatial:transform and spatial:shape (matching its multiscales layout entry), along with proj:code, proj:wkt2, spatial:dimensions and spatial:registration. Readers can place any level without going back to the root.
  • GeoZarr attrs already on the input (proj:*, spatial:*, multiscales, zarr_conventions, zarr-layer), e.g. from a prior GeoZarr ETL, are dropped and rebuilt, so they no longer leak stale values onto every level. The same goes for a level kept from the store with write(mode="a").

0.2.1

Changed

  • Pyramid.write raises when there is nothing to write (levels=[], or a levels="auto" plan that resolves to only level 0 and isn't asked to write it). It used to return quietly after rewriting the root metadata, dropping the store's existing overviews from multiscales.
  • write(mode="a") raises if the store already lists a level the pyramid doesn't have, or one built with a different resampling method or shape. Use matching create_pyramid settings, or mode="w" to rebuild. Adding levels on top of an existing pyramid still works.

0.2.0

Changed

  • Integer mean rounds half to even (numpy's rint) instead of truncating, so a window averaging 5.75 gives 6, not 5. Existing integer stores differ by up to one unit; don't append new levels (mode="a") to a store written by an older version.
  • Scaled integer input (CF packed data: scale_factor / add_offset with a _FillValue) is written in its stored integer dtype, whether you opened it as floats or with mask_and_scale=False. as_datatree returns it as integers too.
  • Shards along non-spatial dimensions (like time) line up with the source chunks on every level. Coarser levels used to divide those sizes by the level factor too, or skip the alignment.

Added

  • create_pyramid(levels="auto", min_dim=256) picks the level count from the raster size.

Fixed

  • The integer _FillValue of scaled data is no longer used as the missing marker on decoded floats, which dropped real values that decoded to it.
  • Levels kept during a partial rewrite (mode="a") now get the proj attrs too.

0.1.11

Fixed

  • Each level group now carries the proj attrs, so level arrays inherit the CRS (#46).

0.1.10

Performance

  • write produces the lower levels from shared level-0 tiles in memory instead of re-reading them from the store.
  • block_reduce has a faster path for C-contiguous input.
  • New zarrs extra: when installed, Pyramid.write uses the zarrs codec pipeline for local stores.

Deprecated

  • write(keep_levels_in_memory=...) is ignored and warns.

0.1.8

Fixed

  • write no longer drops variables over exactly one spatial dim. A variable like profile(time, x) is legal input and appears in every level template, but only variables carrying both spatial dims were computed. Above level 0 the template placeholder was written verbatim: all-NaN data, float-promoted off the source dtype, with level 0 correct — silent wrong data, easy to miss. Such variables are now coarsened along whichever spatial dim they carry, matching as_datatree(), and get a chunk/shard recommendation at every level.

recommend_encoding covers them on the flat path too, so a partly-spatial variable is now sized along its spatial dim instead of falling through to xarray's defaults. Non-numeric variables over a spatial dim (string labels, datetimes), which neither path can reduce, are rejected at plan time.

  • create_pyramid now validates method when the plan is built instead of letting the Rust kernel reject it on the first coarsened level — by which point level 0 was already written to the store. Pyramid.write re-checks, so editing pyramid.method after planning is caught too, and a rejected write leaves the store untouched.

The valid set comes from the installed kernel (topozarr_core.METHODS, new in core 0.1.7), not from a second list on the Python side. That is the check #26 called for: a topozarr advertising a method its linked topozarr-core does not implement now fails at plan time.

  • as_datatree() now matches write() value for value. xarray.coarsen knows nothing of _FillValue, so the Dask path averaged the sentinel in as data — silently wrong for masked rasters, with nothing in the output to hint at it. It also promoted integers to float, which left pyramid.encoding (sized from the source itemsize) mis-sized for the data being written through it. Levels are now coarsened on an f8 promotion with the fill masked to NaN, then refilled, clipped and cast back to the source dtype. The clip matters for an integer sum: the kernel saturates an out-of-range accumulator where a bare numpy cast wraps.

An f8 source is the one remaining divergence — under 1 ULP on mean/sum, from window summation order. A method with no xarray.coarsen equivalent now raises NotImplementedError instead of AttributeError.

  • create_pyramid rejects datasets whose spatial coordinates are 2-D (curvilinear grids, e.g. lat(y, x)). They were never coarsened: the level templates left them at native resolution, which surfaced as an opaque xarray conflicting sizes for dimension error, and as_datatree silently corner-strided them into a mis-registered grid. The error now names the coordinates and points at ds.drop_vars.

  • Chunk/shard recommendations now line up with Dask writes. ds.to_zarr(encoding=recommend_encoding(ds)) — the snippet in the README, the docs and the docstring — raised safe_chunks errors for most dask sources. xarray requires the zarr write unit (the shard, when sharding is on) to divide the dask block, and the candidate shards admitted multiples of the source chunk as well as divisors; every multiple fails, and restricting to divisors alone left nothing, since the 128-element chunk floor rejects them.

chunks_per_shard is now an upper bound, flexed down per spatial dimension to the largest power of 2 whose shard divides the source chunk. A multiple is used only when no divisor works at any of them.

This extends to pyramids: levels above 0 have no source chunking to sniff, so it is derived from the level-0 blocks and the cumulative factor. A DataTree.to_zarr of a dask source now writes unaided while the coarsened blocks stay above roughly half the ideal chunk size.

pyramid.encoding therefore moves for chunked sources at every level. Unchunked sources are byte-identical, encoding and data alike.

0.1.6

Added

  • recommend_encoding(ds, x_dim=..., y_dim=...) returns the chunk/shard encoding for a single-resolution (flat) dataset — the same heuristic create_pyramid applies per level, previously reachable only as pyramid.encoding. Pair it with attach_geozarr_metadata and pass the result to ds.to_zarr(..., encoding=...). No CRS required; it covers variables with both spatial dims and leaves the rest to xarray's defaults. create_pyramid now builds its per-level encoding through the same function, so pyramid output is unchanged.

Dask-backed datasets write unaided as of the safe_chunks fix above.

0.1.5

  • Chunking/sharding heuristics now take into account the non-spatial dimensions (time, band, ...). If there is available 'space', then shards can include more chunks-per-shard for non-spatial dims. You can still specify your own encoding by editing pyramid.encoding before writing.

  • Removed the experimental io="rust" write path (Pyramid.write(..., io="rust")) and the RustWriter class in topozarr-core. It wrote regions through the zarrs crate instead of zarr-python for roughly a 25% gain on S3, which did not justify carrying a second write path, a store-URL translation layer, and four heavy Rust dependencies. All writes now go through zarr-python or Icechunk. topozarr-core continues to provide the block_reduce coarsening kernel, which is unaffected.

0.1.4

Fixed

  • topozarr now pins topozarr-core exactly instead of >=0.1.0,<0.2. The old range let a resolver pair topozarr 0.1.3 with core 0.1.0/0.1.1, which predate the nearest kernel: the Python layer accepted method="nearest" and the Rust kernel then raised ValueError mid-write, after a pyramid had started. Reported in #26.

Changed

  • topozarr and topozarr-core are released in lockstep under the same version number from here on (core skips 0.1.3). A CI check keeps the pin and both core manifests in sync.

0.1.3

Added

  • nearest coarsening method, alongside mean/max/min/sum. Corner-picks the top-left cell of each window instead of aggregating, for categorical data (class codes, masks) where averaging invents values. Composable across levels like the other methods.

Versions prior to this point are not documented here — see the GitHub releases and tags for history.