backend.export_cache Namespace Reference

Functions
float	_dt_to_epoch_s (dt)
threading.Lock	_lock_for (str run_id)
Path	cache_dir (str run_id)
bool	is_cached (str run_id)
tuple[float, float]|None	_time_bounds (Path parquet_path)
tuple[float, float]|None	_value_bounds (Path parquet_path)
tuple[dict[str, str], float|None, float|None]	_entity_clocks (Path cdir)
int|None	_db_size_bytes (str run_id)
dict	build_index (str run_id)
Path	ensure_exported (str run_id, bool refresh=False)
dict	get_index (str run_id)

Variables
dict	_locks = {}
	_locks_guard = threading.Lock()
str	_DONE = "_done"
str	_INDEX = "_index.json"
int	INDEX_VERSION = 3

Detailed Description

Lazy, cached parquet export of a past Elodin run + a component index.

Key mechanic (verified against real runs): the parquet exporter only resolves
human component names when given a `--pattern`; `--pattern '*'` names every
named component and keeps the ~few genuinely-unnamed ones as hashes. Past-run
dirs are immutable, so we export once and reuse the cache forever.

Function Documentation

_db_size_bytes()

int | None backend.export_cache._db_size_bytes ( str run_id )

protected

Real on-disk size of the source DB dir (its data/index files are sparse,
so `du -sk` block counts are used). One-time cost at export.

Definition at line 147 of file export_cache.py.

Referenced by build_index().

_dt_to_epoch_s()

float backend.export_cache._dt_to_epoch_s ( dt )

protected

Naive datetime/np.datetime64 → epoch seconds, treating the wall value as
UTC (matches series.load_series so index bounds and series times align).

Definition at line 23 of file export_cache.py.

Referenced by _time_bounds().

_entity_clocks()

tuple[dict[str, str], float | None, float | None] backend.export_cache._entity_clocks ( Path cdir )

protected

Per-entity clock kind, and the run's extent on the clocks actually drawn.

The extent has to be a union across every entity, each measured on its OWN clock,
because they do not cover the same interval: the heartbeat and command channels start
at boot, ~3.7 s before the first sensor packet on a real run. Taking only the
sensor-clocked entities' extent clipped the start of every one of those traces.

Classification mirrors series.sensor_clock and shares its tolerance so the two always
agree, but works off row-group statistics — no data read. min-vs-min stands in for
that function's median: the two differ by milliseconds, which decides nothing here
and shifts a window bound by nothing that matters.

Definition at line 107 of file export_cache.py.

References _time_bounds(), _value_bounds(), max, and min.

Referenced by build_index().

_lock_for()

threading.Lock backend.export_cache._lock_for ( str run_id )

protected

Definition at line 43 of file export_cache.py.

Referenced by ensure_exported().

_time_bounds()

tuple[float, float] | None backend.export_cache._time_bounds ( Path parquet_path )

protected

Cheap [t_min, t_max] in epoch seconds from row-group stats (no full read).

Definition at line 56 of file export_cache.py.

References _dt_to_epoch_s(), max, and min.

Referenced by _entity_clocks(), and build_index().

_value_bounds()

tuple[float, float] | None backend.export_cache._value_bounds ( Path parquet_path )

protected

[min, max] of the (single) value column from row-group stats — no data read.

Used on a `*.timestamp_ns` parquet to get the sensor clock's extent, which is what
the x-axis actually spans when that clock is in use.

Definition at line 84 of file export_cache.py.

References max, and min.

Referenced by _entity_clocks().

build_index()

dict backend.export_cache.build_index

(

str

run_id

)

Scan cached parquet → component index with families and time bounds.

Definition at line 162 of file export_cache.py.

References _db_size_bytes(), _entity_clocks(), _time_bounds(), cache_dir(), max, and min.

Referenced by ensure_exported(), and get_index().

cache_dir()

Path backend.export_cache.cache_dir

(

str

run_id

)

Definition at line 48 of file export_cache.py.

Referenced by build_index(), ensure_exported(), get_index(), and is_cached().

ensure_exported()

Path backend.export_cache.ensure_exported	(	str	run_id,
		bool	refresh = False )

Export the run to parquet if not already cached. Returns the cache dir.

Definition at line 211 of file export_cache.py.

References _lock_for(), build_index(), and cache_dir().

Referenced by get_index().

get_index()

dict backend.export_cache.get_index

(

str

run_id

)

Return the component index, exporting/rebuilding lazily if needed.

Config-derived names (run_config.annotate) are layered on at read time rather than
cached with the index — the snapshot is a separate file from the parquet, so one
added or corrected later takes effect without discarding the export.

Definition at line 246 of file export_cache.py.

References build_index(), cache_dir(), and ensure_exported().

is_cached()

bool backend.export_cache.is_cached

(

str

run_id

)

Definition at line 52 of file export_cache.py.

References cache_dir().

Variable Documentation

_DONE

str backend.export_cache._DONE = "_done"

protected

Definition at line 32 of file export_cache.py.

_INDEX

str backend.export_cache._INDEX = "_index.json"

protected

Definition at line 33 of file export_cache.py.

_locks

dict backend.export_cache._locks = {}

protected

Definition at line 29 of file export_cache.py.

_locks_guard

backend.export_cache._locks_guard = threading.Lock()

protected

Definition at line 30 of file export_cache.py.

INDEX_VERSION

int backend.export_cache.INDEX_VERSION = 3

Definition at line 40 of file export_cache.py.