Skip to content
RepoReady
Sign up
Menu
Docs menu

Manuscript consistency: paper and code agree

RepoReady compares what the paper states with what the code and its saved outputs do. A confirmed contradiction on a key result blocks L2.

Last updated

The code filters at 300 genes, the paper says 200: a contradiction that blocks L2.
Level relationshipA contradiction blocks L2

Paper and code are revised separately, and details drift: a reviewer asks for a one-sided test, the text changes, the script does not. A Materials and workflow check compares them for every key result; a mismatch on a supporting result is a Warning.

Outputs correspond to the reported results

What counts
The code behind each key result implements the method, parameters and statistics the paper states: model and equations, thresholds, test, sidedness, correction, intervals and n.
Common gaps
The paper says Louvain, the code runs Leiden; five seeds stated, three run; a one-sided test reported, a two-sided one computed.
How to fix
Revise text and code together, so every change to the Methods comes with the commit that implements it, and regenerate the results it affects.

Data processing and splits

What counts
Every exclusion, replaced value, transformation and alignment the code applies is in the Methods, and the train/test split or sample selection is the one stated.
Common gaps
Samples dropped without a word, missing values filled with zeros, misaligned rows, or a held-out set that overlaps the training data.
How to fix
Describe each processing step in the Methods. Create splits once, save the held-out identifiers and let every model read the same file:
R
set.seed(2026)
folds <- rsample::vfold_cv(cohort, v = 5, strata = outcome)
saveRDS(folds, "data/processed/cv_folds.rds")

# every model script, your method and baselines alike:
folds <- readRDS("data/processed/cv_folds.rds")

Reported values

What counts
Each reported number follows from the shared code or its saved outputs, and every quantity is computed as the paper defines it.
Common gaps
Rounded values typed in from a notebook output that no longer exists, or a rate computed over a different window than the text defines.
How to fix
Save every reported value with the script that computes it, and add a test that compares it with the paper:
tests/test_paper_values.py
import json

def test_table2_auroc_matches_paper():
    with open("results/claims/table2_auroc.json") as fh:
        result = json.load(fh)
    assert 0.90 <= result["mean"] <= 0.92  # the paper reports 0.91

Paper, data and docs agree

What counts
The paper states each value consistently, each sample, dataset and version has one name, references point to the right target, and docs match code and paper.
Common gaps
A legend and the Methods with different thresholds, a strain with two names, a malformed ontology ID, or a README describing an older pipeline.
How to fix
These findings are Warnings and never change the level. Correct the passage each one cites, and use one name per item across paper, deposits and code.

See where your project stands

Run the analysis on your paper or repository. Every finding names its area and links back here.

Get started