OmniQ platform verifies 59 quantum papers for AI scientists

Introduction

The OmniQ platform, a virtual research environment developed by Nanyang Technological University, has systematically transformed 59 peer‑reviewed quantum computing papers into reproducible, executable tasks. By subjecting these papers to rigorous verification with multiple large‑language‑model (LLM) powered AI scientists, the study uncovers a fundamental barrier: current AI agents often cannot reproduce published results because the scientific record lacks the precise, machine‑readable details they need. Human experts repeatedly bridge this gap, prompting the creation of Residual Agents (RA)—self‑improving AI scientists that learn from corrected workflows and turn them into reusable research experience.

Turning Papers into Executable Workflows

OmniQ does more than check calculations; it reconstructs the entire computational pipeline described in each paper—code, data, parameters, and hardware constraints—into a controlled, diagnosable task. The researchers curated a diverse dataset covering classical simulation, quantum algorithms, and quantum‑property characterization, cleaning an initial pool of 60 papers down to 59 reliable cases. This meticulous preparation highlighted a recurring problem: many publications contain ambiguities or mismatches between the written methodology and the actual implementation, forcing expert intervention to resolve.

AI Scientists Meet the Reproduction Challenge

Five LLM‑driven AI scientists and six third‑party agent harnesses were evaluated on the OmniQ benchmark. The results were sobering:

• Low baseline reproducibility – most AI agents failed to match reported outcomes without human guidance.
• Mismatch of expectations – agents struggled to translate narrative descriptions into concrete code, often missing hidden assumptions or data preprocessing steps.
• Computational budget constraints – even when the logic was understood, limited resources prevented full replication of large‑scale simulations.

These findings confirm that autonomous scientific discovery requires more than raw computational power; it demands a research record that is explicitly aligned with the needs of AI agents.

Residual Agents: Learning from Failures

To address the reproducibility gap, the team introduced Residual Agents, an AI scientist that:

1. Diagnoses failures – identifies where a workflow diverges from the paper’s description.
2. Applies expert corrections – incorporates human‑provided fixes into its internal model.
3. Distills reusable experience – stores the corrected workflow as a modular capability that can be invoked for future tasks.

In practice, RA boosted the performance of the DeepSeek V4.1 Flash model from 68 % to 85 % on the OmniQ dataset, achieving accuracy comparable to a larger Claude Code model while using roughly one‑third of the computational budget. Importantly, these gains generalized to unseen papers and transferred across different base models, demonstrating the value of accumulated domain‑specific knowledge.

Building a Reusable Quantum Research Ecosystem

OmniQ’s architecture integrates verified workflows into the model context protocol, making them directly accessible to AI scientists and the broader research community. This creates a virtuous cycle:

• Verification – new quantum papers are continuously ingested, cleaned, and turned into executable tasks.
• Improvement – Residual Agents learn from each verification round, refining their ability to interpret and reproduce future work.
• Reuse – verified workflows become plug‑and‑play components for downstream research, accelerating hypothesis testing and algorithm development.

By treating reproduced results as reusable capabilities rather than static publications, OmniQ paves the way for autonomous agents that can not only validate existing knowledge but also extend it.

Conclusion

The OmniQ platform demonstrates that converting scientific literature into machine‑executable form is both feasible and essential for AI‑driven discovery in quantum computing. The study’s key takeaways are:

1. Current AI scientists need clearer, more structured research records to achieve reliable reproduction.
2. Human expertise remains critical for resolving ambiguities and bridging the communication gap.
3. Residual Agents offer a practical path forward, turning corrective experiences into reusable, cost‑effective capabilities.

As more papers are added to OmniQ and RA continues to learn, the quantum research community can look forward to a growing library of verified, AI‑ready workflows that accelerate innovation and democratize access to cutting‑edge quantum science.