If you can't reproduce a result, you can't trust it, debug it or build on it. Reproducibility is a habit, not an afterthought.
Control Randomness
Set random seeds for data splits, model initialisation and sampling (random_state in scikit-learn; seeds in NumPy and PyTorch). Some GPU operations remain non-deterministic, so report averages over several runs when differences are small.
Version Your Data
Record exactly which data a model was trained on: a file hash, a snapshot date, or a dataset version. Keep the train, validation and test splits fixed and saved.
Track Every Run
For each experiment, log:
- code version (a git commit);
- data version;
- hyperparameters;
- metrics on validation and test sets;
- the environment (library versions).
Tools such as MLflow or Weights & Biases automate this, but even a spreadsheet beats nothing.
Pin the Environment
Save dependencies with exact versions (requirements.txt with pinned versions, a lock file, or a container image). Models saved with pickle may not load in other library versions.
Automate the Pipeline
Put data preparation, training and evaluation in scripts rather than manual notebook steps, so the full process can be rerun from scratch.
Document Decisions
Note why choices were made — features dropped, thresholds chosen — so the next person understands the model, not just the code.