nnpd.core.storage
=================

.. py:module:: nnpd.core.storage

.. autoapi-nested-parse::

   Atomic, relocatable artifacts: JSON metadata, numeric NPY arrays, state dicts.



Classes
-------

.. autoapisummary::

   nnpd.core.storage.Artifact
   nnpd.core.storage.Writer
   nnpd.core.storage.Store


Functions
---------

.. autoapisummary::

   nnpd.core.storage.safe_name
   nnpd.core.storage.utc_now
   nnpd.core.storage.file_hash
   nnpd.core.storage.write_json


Module Contents
---------------

.. py:function:: safe_name(name: str) -> str

.. py:function:: utc_now() -> str

.. py:function:: file_hash(path: pathlib.Path) -> str

.. py:function:: write_json(path: pathlib.Path, value: object) -> None

   Atomic replacement, also for small run-status files outside artifacts.


.. py:class:: Artifact(path: pathlib.Path)

   .. py:attribute:: path


   .. py:attribute:: manifest


   .. py:attribute:: key


   .. py:attribute:: metadata


   .. py:method:: array(name: str, mmap: bool = True) -> numpy.ndarray


   .. py:method:: json(name: str) -> object


   .. py:method:: state_dict() -> dict


   .. py:method:: verify(deep: bool = False) -> None


.. py:class:: Writer(path: pathlib.Path)

   Passed to producers. Large products can allocate and fill arrays in chunks.


   .. py:attribute:: path


   .. py:method:: array(name: str, values: numpy.ndarray) -> None


   .. py:method:: allocate(name: str, shape: tuple[int, ...], dtype='float32') -> numpy.memmap


   .. py:method:: json(name: str, value: object) -> None


   .. py:method:: weights(model: torch.nn.Module) -> None


   .. py:method:: finish(name: str, key: str, metadata: dict, recipe: dict) -> None


.. py:class:: Store(root: str | pathlib.Path)

   .. py:attribute:: root


   .. py:method:: path(name: str, key: str) -> pathlib.Path


   .. py:method:: lock(name: str, key: str) -> filelock.FileLock


   .. py:method:: existing(name: str, key: str) -> Artifact | None


   .. py:method:: transaction(name: str, key: str, recipe: dict) -> Iterator[Writer]

      Caller holds the lock. A failed build never becomes a cache hit.



   .. py:method:: get(name: str, key: str, recipe: dict, build: Callable[[Writer], dict]) -> Artifact