Multiverse Computing tests quantum neural networks in a 30B LLM

Quantum Neural Networks Meet a 30‑Billion‑Parameter LLM

Multiverse Computing, together with researchers at the University of Toronto, has shown that a multi‑head quantum neural network (QNN) can be inserted into a massive 30‑billion‑parameter language model (LLM) and replace a sizable fraction of its transformer blocks. The experiment used a 156‑qubit IBM Heron processor – a current‑generation NISQ device – to run the quantum modules, while the rest of the model continued on conventional GPUs.

How the Quantum Module Works

The QNN contains 145,281 trainable parameters and is designed to replace eight contiguous transformer blocks, which together account for 16.33 % of the model’s parameters. Instead of the usual amplitude‑encoding approach (which would require a number of qubits proportional to the logarithm of the input dimension), the authors partition the hidden space into k disjoint heads. Each head is encoded into its own n‑qubit register and processed in parallel. Classical “mixers” then couple the heads, preserving the residual connections that are essential for transformer architectures.

Experimental Results

• Hardware: The quantum heads were compiled onto a 156‑qubit IBM Heron chip, using 112 qubits to run 14 heads simultaneously with a two‑qubit depth of 13 and no routing swaps.
• Performance: When the QNN replaced a single block, perplexity on WikiText‑2 and C4 stayed within ~1 % of the original model. For larger fused spans, the classical mixers alone achieved within 1‑3 % of the full quantum module’s perplexity.
• Parameter Efficiency: A quantum circuit on n qubits stores O(n) parameters, offering a compact representation compared with the dense weight matrices of classical transformer layers.

Implications and Limits

The work does not claim quantum advantage; rather, it demonstrates that NISQ‑scale circuits can perform a functional slice of a production‑scale LLM. The authors note that as circuit depth grows, trainability may degrade, and the approach may remain limited by the simulability of shallow circuits. Nonetheless, the successful compilation and execution on a real 156‑qubit device suggest a viable path toward hybrid quantum‑classical AI systems.

Looking Ahead

The authors provide full data and protocols in the paper’s appendices, inviting independent verification. Future research will need to address scalability beyond the current eight‑block substitution, explore error‑mitigation techniques, and assess whether deeper quantum modules can deliver measurable speed‑ups or cost reductions compared with purely classical compression methods.

The study is available as a pre‑print on arXiv (https://arxiv.org/html/2611.15250v1).

---

Practical Takeaway: While still early‑stage, this hybrid approach shows that quantum processors can already be woven into the workflow of massive language models, offering a new lever for model compression and potentially opening doors to quantum‑enhanced AI in the near term.