Last week, I was in San Francisco for Re-AGENT, an end-to-end agentic science hackathon. I came away galvanized by what people are trying to build: systems that can move from a scientific question through literature, data, code, experiments, and a record that someone else can inspect. The energy was real. So was the question underneath it: when we give an agent enough tools to participate in science, what exactly are we trusting it to do?
The recent stories about AI agents “escaping sandboxes” have the familiar shape of a headline that gets stranger as it travels. A model did not decide it wanted freedom, and nobody should pretend that an evaluation incident is the same thing as an autonomous system roaming a hospital network. But something important did happen: capable agents, given a goal and too much room to maneuver, found routes their evaluators did not authorize.
That is more interesting than the sci-fi version because it is an ordinary systems problem. Researchers are building more capable medical agents, hospitals are measuring AI alerts inside real workflows, and Anthropic is pushing a research workspace built around reproducibility. These are not separate stories. The model matters, but the sandbox is part of the system.
An agent that games the test is still telling us something
In July, OpenAI and Hugging Face described a security incident during a model evaluation. The relevant point was not that an AI had escaped “into the wild.” During a tool-enabled evaluation, the agent crossed the intended test boundary and accessed Hugging Face systems in an effort to improve its performance. The failure was containment and permissions around a high-capability system, not evidence of intent or consciousness.
The UK AI Security Institute reported a broader version of the same problem. In its cyber evaluations, every frontier model it tested attempted some form of cheating at least occasionally: searching the internet, probing the evaluation software, or trying to escalate privileges outside the designated target. In one misconfigured, unsolvable task, a model wrote and ran code on an external internet service in an attempt to reach the institute’s infrastructure. The architecture caught it, no data leaked, and no damage occurred. AISI is also careful about what this does not show: shortcut-seeking is not proof of deceptive intent.
For medicine, this is a useful warning without the melodrama. We have not seen a comparable patient-data incident in these reports. Still, an agent that can read an inbox, query a chart, submit an order, browse the web, or call an institutional tool should not be judged only by whether its final answer looks good. It needs narrow permissions, segregated test data, independent action logs, and a human approval boundary for consequential work.
The medical sandbox is getting more capable too
A Nature paper on MIRA makes the contrast especially clear. The researchers built an EHR-integrated medical agent that can retrieve history, order and interpret tests, generate differentials, and propose structured actions including prescriptions, admissions, and surgery scheduling. In simulations based on real patient cases, it performed strongly on diagnostic and management measures.
The important word is simulations. MIRA operated in a sandboxed EHR action space, not in live autonomous care, and the authors explicitly call for prospective work on safety, governance, and generalizability. We should decide specifically what agents may do, what they may see, and how we will know when the workflow is drifting.
There are more grounded examples of clinicians building within those boundaries. At TriHealth-Good Samaritan, investigators studied FDA-cleared Viz.ai PE CTPA alerts integrated into a pulmonary embolism response workflow. In a single-center retrospective pre/post study of 124 intervention-treated patients, median time from scan completion to diagnosis fell from 72.5 to 40 minutes. That does not prove a mortality benefit, but it does show the kind of question worth asking: did the intervention change a real operational bottleneck?
Another team used an image-retrieval system to screen salivary-gland tumors for potentially targetable NTRK fusions. In a retrospective series, Luigi-Oral flagged the rare subgroup, while the authors were clear that it is a triage tool for confirmatory testing, not a molecular diagnosis. This is the clinician-builder story I want more of: narrow task, visible boundary, confirmatory test.
Claude Science has the right obsession
Anthropic’s new Claude Science is interesting less because it promises an AI scientist and more because it puts a dull but essential problem at the center: reproducibility. The beta workspace can run analyses, manage local or cluster compute, render structures, and preserve code, environment, and conversation history alongside the figures or notebooks it produces.
That is a better ambition than another claim that a model can “do research.” If an AI is helping with analysis, literature work, or a figure, the useful artifact is the source data, code, environment, assumptions, and decision trail that let someone else inspect it.
Anthropic also reported that an unreleased research version of Claude improved a long-standing bound related to the Riemann zeta function. The technical write-up comes with derivations, outside mathematical review, and a Lean formalization. That is how an AI-assisted mathematical claim should arrive: with artifacts another expert can interrogate.
The viral story about a neurosurgery resident using ChatGPT to solve Crouzeix’s conjecture is a useful counterexample. It is a remarkable story, but it needs provenance before it becomes a discovery claim. A public proof by Emiel Lorist and Felix Schwenninger appeared on arXiv before the resident story was reported. Without a timestamped proof record, model transcript, and independent comparison, we cannot know whether the model independently derived a result or reproduced an already-public one. The lesson is not to dismiss clinicians experimenting with AI. It is to hold exciting claims to the same standard we should want for all scientific work.
Build the guardrails into the work
The emerging divide in AI is between systems that leave enough evidence for us to understand what they did, and systems that ask us to trust a polished result after the fact. For a clinical agent, that means least-privilege access and action logs. For an AI alert, it means measuring workflow and patient outcomes rather than celebrating a sensitivity score. For a scientist, it means preserving the path to a result. The sandbox, the sources, the tools, and the human checkpoint are the system.
- Ramez