Does this paper’s improvement hold under our compute budget, and is the method worth pursuing?
Candidate review
Discuss a pilot ↗Illustrative scores · independent of your workspace edits
# Match conditions before comparing methods.for seed in [11, 23, 37]:config = ExperimentConfig(seed=seed, steps=1200,dataset="fixed_split_v1")baseline = run("baseline", config)method = run("proposed", config)log_pair(seed, baseline, method)# Report all runs, including failed attempts.write_report(results, compute_budget)
Code execution is disabled in this preview.
You can browse and edit the example files. Changes stay in this tab until you reset or leave.ML research · Example candidate
Reproduce a research claim within a compute budget
Task outcome
Made the experiment traceable; one recorded run failed.
Delivered Reproduction + research verdict
Score breakdown
Expand to see whyReproducibility4.0/5
Records seeds, settings and run outcomes. Environment versions are still needed.
Experimental design4.0/5
Plans paired comparisons and accounts for compute. The study is limited to one dataset.
Scientific judgment5.0/5
Reports the failed attempt and limits the recommendation to the available evidence.
Investigation follow-through2.0/5
Leaves the failed run undiagnosed and the next experiment unprioritized.
How to read these scores
1 = major gaps · 3 = partially meets · 5 = fully meets. This example weights all four criteria equally.
These scores and excerpts are illustrative, not a live evaluation or a hiring recommendation. In the product, evaluations score submitted work against the configured criteria; missing evidence is identified separately.
Full findings and sources
Builds a reproducible experiment
Supported in this example
Makes a controlled comparison
Supported in this example
Calibrates an adoption recommendation
Follow-up needed
Optional follow-up
Which next experiment would most change your adoption recommendation within the remaining budget?