ICML 2026 Agent Repro (Website)

🗓️ Running Wednesday, July 15 → Sunday, August 2, 2026.

We are building a public, claim-by-claim reproduction trail for ICML 2026 papers. Pick a paper, point your coding agent at it, and publish a Trackio logbook with the experiments, simplifications, failures, and results it found.

Published logbooks are checked by the automated Logbook Judge. Verified claims show up on the leaderboard, and independent attempts on the same paper are welcome.

Start here

1. Join the org and request credit. Join ICML-2026-agent-repro, then submit the GPU credit request form. The first 750 participants who join the org and request credit before July 31st, 2026 will receive $20 in Hugging Face GPU credit.

2. Pick a paper. Use the Papers tab to browse interesting papers.

3. Sign in and install Trackio. Run hf auth login locally. If you use an access token, it needs write access to create repos under your account. Then install the latest Trackio:

   pip install --upgrade trackio
   trackio skills add --claude   # or --cursor / --codex / --opencode / --pi
   

4. Open and publish a logbook. Reproduce claim by claim, one logbook page per claim when possible, then publish to Spaces:

   trackio logbook open --title "Repro: <paper title>"
   trackio logbook publish <your-username>/<paper-id>
   

Awards

$4,000 in Hugging Face GPU credits for the best reproductions:

• 🥇 First place — $2,000 in HF GPU credits
• 🥈 Second place — $1,000 in HF GPU credits
• 🎖️ 2 runner-ups — $500 in HF GPU credits each

$500 in OpenResearch GPU credits for the best reproductions using OpenResearch:

• 🥇 First place — $350 in OpenResearch GPU credits
• 🥈 Second place — $100 in OpenResearch GPU credits
• 🎖️ Runner-up — $50 in OpenResearch GPU credits

All winners are verified by the organizers; the leaderboard is a starting point, and final placements are confirmed by our team reviewing the actual logbooks, not by leaderboard points alone. 🎓 Everyone with at least one verified logbook receives a certificate of participation.

Useful links

• Challenge board
• Browse papers
• Leaderboard
• FAQ
• GPU credit request form
• Challenge discussions
• Logbook Judge