API Reference

The public API is five objects: create_pyramid builds a write plan, Pyramid holds it, attach_geozarr_metadata adds geozarr convention attrs without building a pyramid, recommend_encoding returns the chunk/shard encoding for a flat dataset, and ZarrLayerVarConfig carries optional visualization hints. CoarseningMethod is the Literal["mean", "max", "min", "sum", "nearest"] alias accepted by create_pyramid(method=...), kept equal to topozarr_core.METHODS (the installed kernel's own list, which validation checks against) by a test; nearest decimates (corner-pick) for categorical data.

topozarr.coarsen.create_pyramid

create_pyramid(ds: Dataset, levels: int | None = None, *, factors: list[int] | None = None, x_dim: str = 'x', y_dim: str = 'y', method: CoarseningMethod = 'mean', target_chunk_bytes: int = DEFAULT_CHUNK_BYTES, chunks_per_shard: ChunksPerShard | None = DEFAULT_CHUNKS_PER_SHARD, layer_hints: dict[str, ZarrLayerVarConfig] | None = None) -> Pyramid

Build a multiscale Zarr pyramid plan from a georeferenced Dataset.

Exactly one of levels / factors must be given.

Parameters:

• ds (Dataset) –

  Source dataset. Must have a CRS assigned via ds.proj.assign_crs.

• levels (int | None, default: None ) –

  Total number of resolution levels, including the original. Level 0 is the original resolution; each subsequent level coarsens by 2× per spatial dimension (cumulative factors [1, 2, 4, ...]).

• factors (list[int] | None, default: None ) –

  Explicit cumulative downsample factors per level, e.g. [1, 4, 16] for a sparse 4×-spaced pyramid. Must start at 1, be strictly increasing, and have each entry integer-divide the next. Mutually exclusive with levels.

• x_dim (str, default: 'x' ) –

  Name of the x (longitude / easting) dimension.

• y_dim (str, default: 'y' ) –

  Name of the y (latitude / northing) dimension.

• method (CoarseningMethod, default: 'mean' ) –

  Spatial aggregation method for coarsening. Applied along whichever spatial dims a variable carries, so a variable over one of them (e.g. a per-column profile(time, x)) is coarsened along that dim alone. Integer variables keep their dtype: mean truncates toward zero (unlike xarray.coarsen, which promotes to float). nearest decimates (keeps the top-left cell of each window) — use it for categorical data such as class codes or masks, where averaging invents values. It ignores fill values, and a class present only away from window corners can vanish at coarse zoom.

• target_chunk_bytes (int, default: DEFAULT_CHUNK_BYTES ) –

  Target uncompressed size per chunk (default ~500 KB).

• chunks_per_shard (ChunksPerShard | None, default: DEFAULT_CHUNKS_PER_SHARD ) –

  Number of chunks per shard along each spatial dimension (e.g. 4 → 4×4 = 16 chunks per shard, ~8 MB). Must be a power of 2 in the range 1–32. Pass None to disable sharding.

  This also sets the shard byte budget. Spatial dims are sized first; when they cannot use the whole budget (a small raster, or a coarse pyramid level), the remainder widens non-spatial dims such as time or band instead of leaving them at a single element. Chunks along those dims stay at 1, so reads still fetch one element at a time. Edit pyramid.encoding before writing to override.

• layer_hints (dict[str, ZarrLayerVarConfig] | None, default: None ) –

  Optional per-variable colormap / color-range hints written into the zarr-layer root metadata key.

Returns:

• Pyramid –

  A Pyramid write plan; call

• Pyramid –

  pyramid.write(store) to compute and write all levels.

Raises:

• ValueError –

  If ds has no CRS, method is not implemented by the installed topozarr-core, chunks_per_shard is not a power of 2 in the range 1–32, x_dim or y_dim is not a dimension of ds, no data variable has both spatial dimensions, a coordinate is 2-D over the spatial dims (curvilinear grids are not supported), a spatial variable has more than 4 dimensions (topozarr-core kernel limit), or a variable over a spatial dim has a non-numeric dtype.

Examples:

import xarray as xr
import xproj  # registers the .proj accessor
from topozarr import create_pyramid

ds = xr.tutorial.open_dataset("air_temperature").drop_encoding()
ds = ds.proj.assign_crs(spatial_ref="EPSG:4326")

pyramid = create_pyramid(ds, levels=2, x_dim="lon", y_dim="lat")
pyramid.write("pyramid.zarr")

# sparse pyramid: native, 4x, 16x (skips the costly 2x level)
sparse = create_pyramid(ds, factors=[1, 4, 16], x_dim="lon", y_dim="lat")
sparse.write("sparse.zarr")

topozarr.pyramid.Pyramid dataclass

Pyramid(source: Dataset, level_templates: dict[int, Dataset], encoding: dict[str, Any], attrs: dict[str, Any], x_dim: str, y_dim: str, method: CoarseningMethod, factors: list[int] = list(), fill_values: dict[str, float | int | None] = dict())

A write plan for a multiscale Zarr pyramid, returned by create_pyramid.

Attributes:

• source (Dataset) –

  The original (level 0) dataset.

• level_templates (dict[int, Dataset]) –

  Per-level datasets carrying real coordinates and attrs; spatial data variables are zero-cost placeholders with the correct shape/dtype (their data is computed during write).

• encoding (dict[str, Any]) –

  Nested dict {path: {var: {"chunks": ..., "shards": ...}}}.

• attrs (dict[str, Any]) –

  Root group metadata (multiscales / proj: / spatial: / zarr-layer).

as_datatree

as_datatree() -> DataTree

Return a lazy DataTree with all pyramid levels coarsened via xarray.

Each level is produced by chaining xarray.coarsen operations on the source dataset. If the source is Dask-backed, the returned tree is fully lazy — suitable for writing on a Dask distributed cluster or with icechunk. Use self.encoding (already shaped for DataTree.to_zarr) to apply the recommended chunks and shards:

dt = pyramid.as_datatree()
dt.to_zarr(store, zarr_format=3, consolidated=False,
           encoding=pyramid.encoding)

Deep levels of a Dask-backed source can outrun the chunk band that recommend_encoding flexes to (see its Note); if to_zarr raises on safe_chunks, pass safe_chunks=False.

Values match write exactly, source dtype and _FillValue included, at the cost of an f8 intermediate through each coarsen. The exception is an f8 source, where the two differ by under 1 ULP on mean/sum (window summation order).

Raises:

• NotImplementedError –

  If method has no xarray.coarsen equivalent. Use write for those.

write

write(store: Any, *, mode: Literal['w', 'w-', 'a'] = 'w', max_workers: int | None = None, levels: list[int] | None = None, max_region_bytes: int = DEFAULT_MAX_REGION_BYTES, progress: bool = False, stats: bool = False, keep_levels_in_memory: bool | None = None) -> dict[str, Any] | None

Compute and write pyramid levels to a Zarr store.

Level 0 is streamed region by region from the source dataset; each subsequent level is block-reduced from the previously written level, streaming shard-sized regions through the Rust kernel on a thread pool. Levels are written sequentially (each reads the previous one); variables within a level are processed in parallel on a shared pool. For bounded memory on large stores, open the source lazily (e.g. xr.open_zarr(store, chunks=None)).

Parameters:

• store (Any) –

  Anything zarr-python accepts — a local path, ObjectStore, or icechunk session store.

• mode (Literal['w', 'w-', 'a'], default: 'w' ) –

  Zarr open mode for the root group. Use "a" when writing a subset of levels so the root group and any pre-existing levels are preserved; "w" with a levels subset raises if the store already holds data (truncation would delete the levels not being rewritten).

• max_workers (int | None, default: None ) –

  Thread pool size for region processing. None derives a default from the CPU count and available memory (peak memory is roughly max_workers * 5 * region_bytes).

• levels (list[int] | None, default: None ) –

  Subset of levels to write (e.g. [1, 2]). Defaults to all levels. Each coarsened level reads its predecessor, so level N > 0 must have level N - 1 either in the subset or already present in the store.

• max_region_bytes (int, default: DEFAULT_MAX_REGION_BYTES ) –

  Memory budget per level-0 copy region. Regions are widened to cover whole source chunks when that fits the budget, so each source chunk is read once.

• progress (bool, default: False ) –

  Show a tqdm progress bar over written regions (requires tqdm).

• stats (bool, default: False ) –

  Collect and return per-level timing stats: region shapes, worker count, wall time, and cumulative per-region read/reduce/write seconds (summed across threads).

  With level pipelining active (keep_levels_in_memory=True or auto-enabled), level N's reduce_s captures fused-reduce time (reducing level-N blocks into the level-N+1 buffer) rather than the reduce of level N itself (which is zero when reading from memory). read_s = block_s - reduce_s remains the pure source-read time at every level.

• keep_levels_in_memory (bool | None, default: None ) –

  Control level pipelining. None (default) auto-enables fusion when the higher levels fit in half the available RAM after accounting for the worker region budget. True forces fusion and raises MemoryError if the budget is exceeded. False disables fusion and always re-reads from the store.

Examples: Write all levels to a local store:

```python
pyramid.write("pyramid.zarr")
```

Rewrite the coarsened levels, preserving level 0:

```python
pyramid.write("pyramid.zarr", mode="a", levels=[1, 2])
```

topozarr.geozarr.attach_geozarr_metadata

attach_geozarr_metadata(ds: Dataset, *, x_dim: str = 'x', y_dim: str = 'y', crs: str | None = None, layer_hints: dict[str, ZarrLayerVarConfig] | None = None) -> Dataset

Return a copy of ds with geozarr convention attrs (proj + spatial).

No multiscale pyramid is built; the dataset stays a flat group. Write it with ds.to_zarr(...). crs defaults to the dataset CRS (xproj).

topozarr.metadata.recommend_encoding

recommend_encoding(ds: Dataset, *, x_dim: str = 'x', y_dim: str = 'y', target_chunk_bytes: int = DEFAULT_CHUNK_BYTES, chunks_per_shard: ChunksPerShard | None = DEFAULT_CHUNKS_PER_SHARD) -> dict[str, dict[str, tuple[int, ...]]]

Recommended chunks / shards for writing ds as a flat group.

Same heuristic create_pyramid applies per level, exposed for single-resolution datasets. Pass the result straight to ds.to_zarr(..., encoding=...). No CRS is required — the encoding depends only on shape and dtype.

Parameters:

• ds (Dataset) –

  Source dataset. Chunking of a chunked source (dask or a zarr backend) is sniffed and the recommendation snapped to nest with it.

• x_dim (str, default: 'x' ) –

  Name of the x (longitude / easting) dimension.

• y_dim (str, default: 'y' ) –

  Name of the y (latitude / northing) dimension.

• target_chunk_bytes (int, default: DEFAULT_CHUNK_BYTES ) –

  Target uncompressed size per chunk (default ~500 KB).

• chunks_per_shard (ChunksPerShard | None, default: DEFAULT_CHUNKS_PER_SHARD ) –

  Number of chunks per shard along each spatial dimension (e.g. 4 → 4×4 = 16 chunks per shard, ~8 MB). Must be a power of 2 in the range 1–32. Pass None to disable sharding.

  This also sets the shard byte budget: whatever the spatial dims leave unused widens non-spatial dims such as time or band. Chunks along those dims stay at 1.

  Treated as an upper bound on a chunked source: it is flexed down per spatial dim where that is what makes the shard divide the source chunk (see the note below).

Returns:

• dict[str, dict[str, tuple[int, ...]]] –

  {var_name: {"chunks": (...), "shards": (...)}} for variables over at

• dict[str, dict[str, tuple[int, ...]]] –

  least one spatial dim (a variable with only one is sized along that dim

• dict[str, dict[str, tuple[int, ...]]] –

  alone); "shards" is omitted when chunks_per_shard is None.

• dict[str, dict[str, tuple[int, ...]]] –

  A 1-D coordinate along a spatial dim gets "chunks" only, sized as a

• dict[str, dict[str, tuple[int, ...]]] –

  1-D array and never sharded, and is absent when the whole coordinate

• dict[str, dict[str, tuple[int, ...]]] –

  already fits in one chunk. Variables over neither spatial dim are

• dict[str, dict[str, tuple[int, ...]]] –

  absent and fall through to xarray's defaults.

Raises:

• ValueError –

  If chunks_per_shard is not a power of 2 in the range 1–32, x_dim or y_dim is not a dimension of ds, or no data variable has both spatial dimensions.

Examples:

from topozarr import attach_geozarr_metadata, recommend_encoding

ds = attach_geozarr_metadata(ds, x_dim="lon", y_dim="lat")
ds.to_zarr(
    "flat.zarr",
    zarr_format=3,
    consolidated=False,
    encoding=recommend_encoding(ds, x_dim="lon", y_dim="lat"),
)

Note

For a dask-backed ds, xarray's safe_chunks check requires the zarr write unit -- the shard, when sharding is on -- to divide the dask block. The recommendation flexes chunks_per_shard down (never up) until a shard does, so the snippet above writes a dask source as is. A source chunk too small to divide into a chunk of usable size is the exception: the recommendation keeps a read-aligned shard there, and such a write still needs a rechunk or safe_chunks=False.

topozarr.metadata.ZarrLayerVarConfig dataclass

ZarrLayerVarConfig(clim: list[float] | None = None, colormap: str | None = None)

Per-variable visualization hints for zarr-layer.

Attributes:

• clim (list[float] | None) –

  Color range as [min, max].

• colormap (str | None) –

  Colormap name (e.g. "blues").

Examples:

create_pyramid(
    ds,
    levels=2,
    layer_hints={"air": ZarrLayerVarConfig(colormap="blues", clim=[230, 310])},
)