Skip to content
RepoReady
Sign up
Menu
Docs menu

Experiment logging and code-to-paper traceability

A metrics file without the configuration, seed and commit behind it cannot be tied to a result. Record every run and map your scripts to the paper.

Last updated

In your report

Area
Execution
Check
Research Traceability

Why it matters

Months after submission, a reviewer asks why a figure changed. Answering takes minutes if every run left a record of its configuration and commit, and days if the figure came from a notebook state nobody saved.

Three of the Ten Simple Rules for Reproducible Computational Research apply here: keep track of how every result was produced, avoid manual data manipulation steps, and store the raw data behind plots.

What good looks like

  • Every run leaves a record next to its outputs: configuration, seed, metrics, git commit and package versions.
  • Parameters that live in config files or command-line options, so a run can be repeated from its record alone.
  • A table in the README that maps each figure and table to its command.
  • Runs that went into the paper kept apart from exploratory runs, for example in results/paper/.
  • Scripts instead of spreadsheet edits or manual filtering; a step that has to stay manual is written down.

How to fix it

Add a paper map to the README, one row per figure or table:

README.md
| Paper item | Command                                                | Output              |
|------------|--------------------------------------------------------|---------------------|
| Figure 2   | Rscript analysis/figure2.R                             | results/figure2.pdf |
| Table 1    | python -m analysis.table1 --config configs/table1.yaml | results/table1.csv  |

Python. Write the configuration, seed and commit next to the outputs of every run:

python
import json, subprocess, sys
from datetime import datetime, timezone
from pathlib import Path

def record_run(cfg: dict, out_dir: Path, metrics: dict) -> None:
    commit = subprocess.run(["git", "rev-parse", "HEAD"],
                            capture_output=True, text=True).stdout.strip()
    out_dir.mkdir(parents=True, exist_ok=True)
    (out_dir / "run.json").write_text(json.dumps({
        "config": cfg,               # includes cfg["seed"]
        "metrics": metrics,
        "git_commit": commit,
        "python": sys.version,
        "finished_at": datetime.now(timezone.utc).isoformat(),
    }, indent=2))

R. Read parameters from YAML and save them with sessionInfo() next to the figure:

R
cfg <- yaml::read_yaml("configs/figure2.yaml")
set.seed(cfg$seed)
# ... analysis using cfg$resolution, cfg$min_cells ...
out <- "results/figure2"
dir.create(out, recursive = TRUE, showWarnings = FALSE)
yaml::write_yaml(cfg, file.path(out, "config_used.yaml"))
writeLines(capture.output(sessionInfo()), file.path(out, "sessionInfo.txt"))

See this check on your repository

Every analysis reports findings for this check, with file references and suggested fixes.

Start free