nnpd.core.storage

Atomic, relocatable artifacts: JSON metadata, numeric NPY arrays, state dicts.

Classes

Artifact
Writer	Passed to producers. Large products can allocate and fill arrays in chunks.
Store

Functions

safe_name(→ str)
utc_now(→ str)
file_hash(→ str)
write_json(→ None)	Atomic replacement, also for small run-status files outside artifacts.

Module Contents

nnpd.core.storage.safe_name(name: str) → str

nnpd.core.storage.utc_now() → str

nnpd.core.storage.file_hash(path: pathlib.Path) → str

nnpd.core.storage.write_json(path: pathlib.Path, value: object) → None

Atomic replacement, also for small run-status files outside artifacts.

class nnpd.core.storage.Artifact(path: pathlib.Path)

path

manifest

key

metadata

array(name: str, mmap: bool = True) → numpy.ndarray

json(name: str) → object

state_dict() → dict

verify(deep: bool = False) → None

class nnpd.core.storage.Writer(path: pathlib.Path)

Passed to producers. Large products can allocate and fill arrays in chunks.

path

array(name: str, values: numpy.ndarray) → None

allocate(name: str, shape: tuple[int, ...], dtype='float32') → numpy.memmap

json(name: str, value: object) → None

weights(model: torch.nn.Module) → None

finish(name: str, key: str, metadata: dict, recipe: dict) → None

class nnpd.core.storage.Store(root: str | pathlib.Path)

root

path(name: str, key: str) → pathlib.Path

lock(name: str, key: str) → filelock.FileLock

existing(name: str, key: str) → Artifact | None

transaction(name: str, key: str, recipe: dict) → Iterator[Writer]

Caller holds the lock. A failed build never becomes a cache hit.

get(name: str, key: str, recipe: dict, build: Callable[[Writer], dict]) → Artifact