backend.summary Namespace Reference

Functions
int|None	_size_bytes (str run_id)
tuple[int, float|None]	_components_and_span (str run_id)
dict	summarize (str run_id)

Detailed Description

What can be said about a run without exporting it.

Opening a run used to trigger the parquet export — tens of seconds and a few hundred MB
of cache for a big one, spent whether or not you meant to plot anything. So the export is
now an explicit action, and this module answers "what am I about to open?" from the run
directory alone, in well under a second.

Everything here comes from filesystem metadata, deliberately. elodin-db's on-disk layout
(a directory per component, each with sparse `data`/`index` files and a `metadata` blob)
is undocumented and version-specific: an early attempt to read component names and row
counts straight out of it decoded a plausible-looking row count that was wrong for every
component — 2536 where the parquet had 314. Sizes, counts and mtimes cannot lie in that
way, so that is all this reads.

The one number that is an estimate is the duration, and it is marked as one.

Function Documentation

_components_and_span()

tuple[int, float | None] backend.summary._components_and_span ( str run_id )

protected

Component count, and the wall time between the first and last write.

One stat per component. The span is when data was *written*, which trails the sample
times the plot is drawn on by the write latency at each end — measured 0.3-0.7 s low
on 20-35 s runs, i.e. within a percent. Good enough to answer "how long was this run"
before committing to an export; the exact figure replaces it once indexed.

Definition at line 40 of file summary.py.

References max, and min.

Referenced by summarize().

_size_bytes()

int | None backend.summary._size_bytes ( str run_id )

protected

On-disk size. `du` block counts, not apparent size — every component's data and
index file is a sparse 8 GiB, so `st_size` would report the run as ~8 TB.

Definition at line 25 of file summary.py.

Referenced by summarize().

summarize()

dict backend.summary.summarize

(

str

run_id

)

Cheap facts about a run, whether or not it has ever been indexed.

Definition at line 70 of file summary.py.

References _components_and_span(), and _size_bytes().