NNPD

Train once. Analyze as often as you like.

NNPD separates experiment execution from scientific choices, and training from analysis. The reusable core handles planning, training, shared artifacts, and results. The Gaussian application implements the study of the paper Prior Dependence in Neural Ratio Estimation (PAI26). What is trained is set in settings_training.py; what is computed from the trained models is set in settings_analysis.py, and can be changed and recomputed at any time without retraining.

Start with your first run, then learn how configurations and sweeps work. To add a simulator, prior, training procedure, metric, or plot, read the extension guide.

uv sync
source .venv/bin/activate
python run.py plan --profile smoke
python run.py run --profile smoke

The smoke profile checks the complete workflow with small datasets. It is not a scientific convergence study. The default command only prints a plan.

Five small concepts

Concept	What it owns
Experiment	What is trained: the problem, prior, training data, model, and training procedure.
Analysis	What is computed from the trained models: products, metrics, and plots.
Product	A reusable dataset with declared dependencies and a persistent cache.
Metric	A measurement from shared products, the model, or both.
Plot	Ordinary plotting code with access to the full experiment context.

There is no fixed number of metrics or plots. Several metrics can share a product without resampling its data or retraining a model. Sweep alternatives vary one knob at a time, not every possible combination.

Important

A working smoke run is not a reproduction of the paper’s full study. Read the scientific notes and testing guide for the implemented definitions, numerical-convergence limits, and what the tests cover.

Get started

• Your first run
  • 1. Install
  • 2. Choose the settings
  • 3. Check the plan
  • 4. Train
  • 5. Analyze
  • 6. Change the analysis later
  • 7. Inspect the results
  • Change settings and run again
  • Where everything is saved
• How the pieces fit together
  • Framework versus application
  • Training versus analysis
  • Configuration, cohort, and job
  • Lifecycle
  • Context and dependencies
• Configuration and one-factor sweeps
  • Two settings files
  • Baseline plus one changed knob
  • Literal lists and atomic choices
  • Change the model or the inferred parameters
  • Priors and three different kinds of repetition
  • Fixed runtime and compatible analysis choices
  • Truth design
  • Analysis must be feasible before anything is computed
• Notebooks and Python scripts
  • Equivalent script

Use and extend

• Working with saved results
  • List runs and restore a model
  • Read a saved artifact without recomputing it
  • Call a metric or plot directly
  • Export a comparison table
• Products, metrics, and plots
  • Shared inference
  • Built-in metrics
  • Plot hooks
  • Add analysis after training
  • Complete registration and extension examples
• Extending the framework
  • The smallest useful abstraction
  • A new simulator or prior
  • A shared post-training dataset
  • Cache correctness is part of the hook contract
  • Metrics that read the model directly
  • Plot anything
  • Adding a model or replacing training
  • Reload and compare results
• CPU, CUDA, and multiple GPUs
  • Choose a device
  • Independent jobs on two GPUs
  • Collective DDP training
  • Precision, reproducibility, and restoration
  • Distributed limits
• Storage, caching and reproducibility
  • Layout
  • Latest settings win
  • Artifact recipes
  • Numeric products and shape conventions
  • Atomicity and interruption
  • Integrity, movement and security

Reference

• Settings reference
  • Training settings: what is trained
  • Analysis settings: what is computed from the trained models
  • Presets
  • Complete settings and preset definitions
• Command-line reference
  • Actions
  • Options
  • Python equivalents
• Python API
  • Start with the extension surface
  • Source API
• The paper
  • Abstract
  • Cite
  • From the paper to the code
  • Run the study
• Scientific notes
  • Basis and scope
  • What the paper specifies
  • Settings for the study’s quantities
  • Interpretation and implementation choices
  • Mathematical clarification: ratio, likelihood shape and posterior
  • HLD, coverage, bias and finite numerical resolution
  • Reweighting and normalization diagnostics
• Testing and validation
  • Run the test suite
  • What the tests cover
  • Full-size reference run
  • Documentation checks
  • Reproducibility scope
• Troubleshooting
  • The default command did not train anything
  • “Analysis is not feasible for these configurations”
  • Analysis says there are no trained models
  • A Choice in the analysis settings is rejected
  • A list did not produce several configurations
  • A selected parameter does not exist
  • An extra metric unexpectedly rebuilt a dataset
  • Results of earlier settings are gone
  • A result was moved and no longer loads
  • CUDA or distributed execution fails immediately
  • Coverage or reweighting looks poor
  • The documentation has not appeared on GitHub

Documentation website

• Publishing the documentation website
  • Publish on GitHub
  • What the workflow does
  • Build and preview locally
  • Edit documentation without changing the application
  • Official documentation for the website tooling

The repository overview is also available as a Markdown file.