Interfaces¶
aereo.interfaces ¶
Abstract classes that should be used as contract. Includes SearchProvider, Reader, etc.
It is important to note that these are abstract classes and should not be instantiated directly. They serve as a blueprint for the actual implementations of readers, search providers, processors, reprojectors, and writers that will be used in the AEREO framework. By defining these interfaces, we ensure that all implementations adhere to a consistent structure and can be easily integrated into the system.
ExtractionTask ¶
A serializable unit of extraction work.
| ATTRIBUTE | DESCRIPTION |
|---|---|
id | Stable identifier generated by the task builder. TYPE: |
assets | GeoDataFrame of source assets to read. TYPE: |
job | Parent TYPE: |
aoi | Optional task-specific AOI. When provided, the orchestrator uses this geometry instead of TYPE: |
grid_cells | Optional explicit list of MajorTOM grid cells this task is responsible for. When provided, the executor uses these cells directly instead of rediscovering them from the AOI. This is the normal case for tasks produced by TYPE: |
task_context | Optional metadata (e.g. TYPE: |
bbox property ¶
bbox
WGS84 bounding box the reader should crop to, if any.
When the task has explicit grid cells and job.grid_cells_margin is set, the returned bounds cover the expanded GeoBox of every cell so readers fetch enough source data to avoid cutting edge cells.
Precedence
- Expanded grid-cell GeoBoxes when
grid_cellsandgrid_cells_marginare set. task.aoi.boundswhentask.aoiis set.job.target_aoi.boundswhenjob.target_aoiis set.None.
datetime_range property ¶
datetime_range
Minimum start time and maximum end time across task assets.
stac_items property ¶
stac_items
Unique pystac.Item objects reconstructed from assets["stac_item"].
Returns an empty list when the column is missing or contains only nulls.
grid_dist property ¶
grid_dist
Grid cell size in metres shared by all tasks in this run (delegated to job).
Processor ¶
Bases: Protocol
Pure xarray.Dataset -> xarray.Dataset transform.
Reader ¶
Bases: Protocol
Reads source data for an extraction task and returns an xr.Dataset.
Reprojector ¶
Bases: Protocol
Reprojects/resamples an xarray.Dataset to a target definition.
SearchProvider ¶
Bases: Protocol
Interface for search providers.
Search providers are callables that receive collection, spatial, and temporal constraints and return a GeoDataFrame of matched assets.
TaskBuilder ¶
Bases: Protocol
Builds a sequence of extraction tasks from search results.
Task builders are job-level plugins: they run once per job, grouping and chunking search-result assets into ExtractionTask objects that the per-task extraction pipeline can execute.
Writer ¶
Bases: Protocol
Serialises an xarray.Dataset to a single file.
build_collection_asset_filters ¶
build_collection_asset_filters(collections_config)
Derive collection list and per-collection asset filters from configuration mapping or sequence.
| PARAMETER | DESCRIPTION |
|---|---|
collections_config | Mapping of collection -> list of asset/channel keys, or sequence of collection names. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
tuple[list[str], dict[str, set[str] | None]] | A |
Source code in components/aereo/interfaces/core.py
def build_collection_asset_filters(
collections_config: Mapping[str, Sequence[str]] | Sequence[str] | None,
) -> tuple[list[str], dict[str, set[str] | None]]:
"""Derive collection list and per-collection asset filters from configuration mapping or sequence.
Args:
collections_config: Mapping of collection -> list of asset/channel keys,
or sequence of collection names.
Returns:
A ``(collections, asset_filters)`` tuple.
"""
if collections_config is None:
return [], {}
if isinstance(collections_config, Mapping):
asset_filters: dict[str, set[str] | None] = {}
for coll, vars_list in collections_config.items():
if vars_list and "*" not in vars_list:
asset_filters[coll] = set(str(v) for v in vars_list)
else:
asset_filters[coll] = None
return list(collections_config.keys()), asset_filters
collections = list(dict.fromkeys(collections_config))
return collections, {coll: None for coll in collections}
empty_asset_result ¶
empty_asset_result()
Return an empty GeoDataFrame with AssetSchema columns.
Source code in components/aereo/interfaces/core.py
def empty_asset_result() -> GeoDataFrame[AssetSchema]:
"""Return an empty GeoDataFrame with AssetSchema columns."""
import geopandas as gpd
columns = list(AssetSchema.to_schema().columns.keys())
if "geometry" not in columns:
columns.append("geometry")
gdf = gpd.GeoDataFrame(columns=columns, geometry="geometry")
return cast(GeoDataFrame[AssetSchema], AssetSchema.validate(gdf))
infer_dataset_time_bounds ¶
infer_dataset_time_bounds(ds)
Infer and set the start and end time bounds in the dataset's attributes.
If a time coordinate is present, uses its minimum and maximum values. Otherwise, leaves the dataset attributes unchanged.
| PARAMETER | DESCRIPTION |
|---|---|
ds | The xarray.Dataset. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
Dataset | The dataset with inferred time bounds set in its attributes (if possible). |
Source code in components/aereo/interfaces/utils.py
def infer_dataset_time_bounds(ds: xr.Dataset) -> xr.Dataset:
"""Infer and set the start and end time bounds in the dataset's attributes.
If a ``time`` coordinate is present, uses its minimum and maximum values.
Otherwise, leaves the dataset attributes unchanged.
Args:
ds: The xarray.Dataset.
Returns:
The dataset with inferred time bounds set in its attributes (if possible).
"""
import pandas as pd
if "time" in ds.coords:
times = ds.coords["time"].values
if len(times) > 0:
ds.attrs["start_time"] = pd.Timestamp(times.min()).to_pydatetime()
ds.attrs["end_time"] = pd.Timestamp(times.max()).to_pydatetime()
return ds
normalize_geometry_input ¶
normalize_geometry_input(value)
Normalize a geometry input into a Shapely BaseGeometry.
Supports:
BaseGeometryinstances (returned unchanged)- GeoJSON-like
dict - A
strorPathpointing to a GeoJSON file (.geojsonor.json)
| PARAMETER | DESCRIPTION |
|---|---|
value | Geometry input to normalize. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
BaseGeometry | None | A Shapely geometry, or |
| RAISES | DESCRIPTION |
|---|---|
ValueError | If the input cannot be parsed into a geometry. |
Source code in components/aereo/interfaces/utils.py
def normalize_geometry_input(
value: BaseGeometry | dict[str, Any] | str | Path | None,
) -> BaseGeometry | None:
"""Normalize a geometry input into a Shapely ``BaseGeometry``.
Supports:
* ``BaseGeometry`` instances (returned unchanged)
* GeoJSON-like ``dict``
* A ``str`` or ``Path`` pointing to a GeoJSON file (``.geojson`` or ``.json``)
Args:
value: Geometry input to normalize.
Returns:
A Shapely geometry, or ``None`` if the input was ``None``.
Raises:
ValueError: If the input cannot be parsed into a geometry.
"""
if value is None:
return None
if isinstance(value, BaseGeometry):
return value
if isinstance(value, Path):
return _geometry_from_geojson_path(value)
if isinstance(value, str):
if _looks_like_geojson_path(value):
return _geometry_from_geojson_path(Path(value))
raise ValueError(
"Invalid geometry input type: str. "
"Expected a path to a GeoJSON file (ending in .geojson or .json) "
"or a file-system path containing a separator."
)
if isinstance(value, dict):
return shape(value)
raise ValueError(
f"Invalid geometry input type: {type(value).__name__}. "
"Expected BaseGeometry, GeoJSON dict, or path to a GeoJSON file."
)
resolve_callable ¶
resolve_callable(val)
Resolve a callable from a string, dict target, or direct callable.
| PARAMETER | DESCRIPTION |
|---|---|
val | The value to resolve. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
Any | The resolved callable. |
Source code in components/aereo/interfaces/utils.py
def resolve_callable(val: Any) -> Any:
"""Resolve a callable from a string, dict target, or direct callable.
Args:
val: The value to resolve.
Returns:
The resolved callable.
"""
import importlib
if callable(val):
return val
if isinstance(val, str):
if ":" in val:
module_name, func_name = val.split(":", 1)
else:
module_name, func_name = val.rsplit(".", 1)
module = importlib.import_module(module_name)
func = getattr(module, func_name)
return func
if isinstance(val, dict) or (hasattr(val, "keys") and hasattr(val, "get")):
d = dict(val)
target = d.pop("_target_", None)
if not target:
raise ValueError(
"Dictionary-based configuration must include a '_target_' key."
)
func = resolve_callable(target)
kwargs = {k: v for k, v in d.items() if not k.startswith("_")}
if kwargs:
return partial(func, **kwargs)
return func
raise ValueError(f"Cannot resolve callable from type: {type(val).__name__}")
set_dataset_time_bounds ¶
set_dataset_time_bounds(ds, start_time, end_time)
Set the start and end time bounds in the dataset's attributes.
| PARAMETER | DESCRIPTION |
|---|---|
ds | The xarray.Dataset. TYPE: |
start_time | The start time. TYPE: |
end_time | The end time. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
Dataset | The dataset with time bounds set in its attributes. |
Source code in components/aereo/interfaces/utils.py
def set_dataset_time_bounds(
ds: xr.Dataset, start_time: datetime, end_time: datetime
) -> xr.Dataset:
"""Set the start and end time bounds in the dataset's attributes.
Args:
ds: The xarray.Dataset.
start_time: The start time.
end_time: The end time.
Returns:
The dataset with time bounds set in its attributes.
"""
ds.attrs["start_time"] = start_time
ds.attrs["end_time"] = end_time
return ds
update_callable ¶
update_callable(callable_obj, **kwargs)
Update a callable (standard function or partial) with new keywords.
| PARAMETER | DESCRIPTION |
|---|---|
callable_obj | The callable to update. TYPE: |
**kwargs | The keyword arguments to update it with. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
Any | A new callable with updated keyword arguments. |
Source code in components/aereo/interfaces/utils.py
def update_callable(callable_obj: Any, **kwargs: Any) -> Any:
"""Update a callable (standard function or partial) with new keywords.
Args:
callable_obj: The callable to update.
**kwargs: The keyword arguments to update it with.
Returns:
A new callable with updated keyword arguments.
"""
if isinstance(callable_obj, partial):
new_keywords = {**callable_obj.keywords, **kwargs}
return partial(callable_obj.func, *callable_obj.args, **new_keywords)
if callable(callable_obj):
return partial(callable_obj, **kwargs)
return partial(resolve_callable(callable_obj), **kwargs)
validate_aereo_dataset ¶
validate_aereo_dataset(
ds, *, require_crs=True, require_dims=("band", "y", "x")
)
Validate that ds conforms to the AEREO xarray conventions.
| PARAMETER | DESCRIPTION |
|---|---|
ds | The dataset to validate. TYPE: |
require_crs | If True, ensure TYPE: |
require_dims | If given, ensure all listed dimensions exist. TYPE: |
| RAISES | DESCRIPTION |
|---|---|
ValueError | If any convention is violated. |
ImportError | If |
Source code in components/aereo/interfaces/utils.py
def validate_aereo_dataset(
ds: Any,
*,
require_crs: bool = True,
require_dims: Sequence[str] | None = ("band", "y", "x"),
) -> None:
"""Validate that *ds* conforms to the AEREO xarray conventions.
Args:
ds: The dataset to validate.
require_crs: If True, ensure ``ds.rio.crs`` is set.
require_dims: If given, ensure all listed dimensions exist.
Raises:
ValueError: If any convention is violated.
ImportError: If ``rioxarray`` is not installed and *require_crs* is True.
"""
if not isinstance(ds, xr.Dataset):
raise ValueError(f"Expected xarray.Dataset, got {type(ds).__name__}")
if require_crs:
_import_rioxarray()
# Access the rio accessor to trigger its import side-effects
if ds.rio.crs is None:
raise ValueError(
"xarray.Dataset must have a CRS set via rioxarray (ds.rio.crs)"
)
if require_dims:
missing = [d for d in require_dims if d not in ds.dims]
if missing:
raise ValueError(f"xarray.Dataset missing required dimensions: {missing}")