Two Turing Award Winners Confront the Theoretical Black Hole Behind AGI at BAAI Conference

The 8th Beijing Academy of Artificial Intelligence (BAAI) Conference kicked off on June 12, 2026 in Beijing, with an opening session featuring two Turing Award laureates — Whitfield Diffie (2015) and Andrew Barto (2024) — who independently arrived at a shared concern: the fundamental theoretical challenges underpinning the pursuit of Artificial General Intelligence (AGI). Diffie, known as the father of public-key cryptography, examined AI agent security through the lens of information security. He argued that while cryptography's narrow-domain success stems from clearly defined specifications, the ambition of AGI — to "do everything" — makes it impossible to write formal specifications that prevent hallucinations or loss of control. "We want to prove a system conforms to its specification, but first you have to be able to write that specification," Diffie said, noting that even a formal definition of "not hallucinating" remains elusive. He contrasted this with public-key cryptography, which succeeded because academia and industry spent decades designing, verifying, and standardizing protocols. "The AI security industry will have to go through a similar long-term process of protocol building and standardization before agents can truly be controlled," he warned, noting that current LLM security remains in an early, disorderly phase. Andrew Barto, the pioneer of temporal difference learning and the Actor-Critic architecture, traced reinforcement learning's century-long hidden history — from Thorndike's puzzle box experiments in 1898 to AlphaGo. He identified the core bottleneck as reward function design. In perfectly defined environments like chess or Go, reward signals are straightforward. But in complex real-world scenarios, designing a perfect reward function is fundamentally impossible. Barto invoked Norbert Wiener's half-century-old warning: "It will give you what you asked for — but not what you actually wanted and needed." He framed this as the "Midas Touch" problem, where literal optimization destroys genuine value, warning that as autonomous AI agents proliferate, this risk multiplies exponentially. "We cannot rely solely on a single line of reward function; we must build robust, dynamic guardrails backed by extensive experimental validation," Barto stressed. Both speakers converged on the same conclusion: we are endowing machines with agency, yet we can neither mathematically constrain them with formal specifications nor guide them with perfect reward functions. The path from Shannon's information theory to today's cryptographic standards took half a century; reinforcement learning spanned a century from Thorndike to AlphaGo. The theoretical foundations for AGI safety, they cautioned, require a similarly long timeline — far longer than the current industry frenzy suggests.