This article is brought to you by COMSOL.

These days, no one blinks an eye at two-day, one-day, and even same-day package deliveries. Despite the plethora of localized fulfillment and distribution centers that have cropped up to meet the demand, delivery trucks still need to make considerably long drives, often over highways. To reduce fuel and maintenance costs as well as carbon emissions, companies are investing in electric trucks for their delivery fleets. While turning to electric trucks is a promising solution, the battery packs incorporated into these automotive designs often fall short in the lifetime needed to make long-distance highway travel feasible.

To help overcome the issue of battery lifetime, a consortium of universities and industry partners incorporated multiscale simulation and a custom simulation app into the battery manufacturing process. The organizations involved in this project were Aurobay, Resolvent, the European Commission, the Technical University of Denmark (DTU), Vinnova (the Swedish Agency for Innovation Systems), Innovation Fund Denmark, and Volvo Group Trucks Operations.

The simulation app, built by Resolvent, a Denmark-based engineering consultancy that specializes in green energy, enables the R&D department at Volvo to easily test battery lifetime performance.

LaserWho?

The Multiscale Simulation of Laser Welding for Optimal Battery Pack Manufacturing project — referred to as “LaserBATMAN” — was initiated in May 2022 and wrapped up in May 2025 (Ref. 1). Although BATMAN here refers to battery manufacturing and not the famous masked superhero, the goals for the project also strove for greatness.

The LaserBATMAN project aimed to optimize two key performance metrics for battery packs: robustness and lifetime (Ref. 1). One element of the battery manufacturing process that affects both of these metrics is the laser welding used in component joining processes (for example, to connect battery cells in a pack via a busbar). If done correctly, laser welding can positively impact battery life, but if done incorrectly, laser welding can increase the risk of battery degradation. Computational modeling can be used to analyze and predict the effects of welding and joining processes at different stages of the manufacturing process.

Martin Refslund Nielsen, former chief commercial officer and partner at Resolvent, explained that for this type of multiphysics and multiscale problem, “COMSOL Multiphysics is just the right tool.”

R&D Workflow for Multiscale Model and App

The Resolvent team used the COMSOL Multiphysics software to study how laser welding affects the electrochemistry and degradation of battery cells. At least, that’s how the project started.

“Ambitions grew,” Nielsen said, explaining that the team ended up modeling the performance of the entire battery, from cell to pack level, and even “turned [the model] into a design tool in the end”, in reference to the app.

After training the deep neural network of the battery model, the team built a custom simulation app to efficiently optimize busbar tabs in the configuration of a battery pack.

First, the team built an electrochemical model of a battery cell by inputting the formulas for the relevant chemical reaction kinetics (Figure 2). The resulting cell-layer model was then fed input data, and the output data was used to train a deep neural network (DNN). With this reduced-order model, the team was able to quickly gain insight into cell behavior.

After training the DNN of the battery model, the team built a custom simulation app to efficiently optimize busbar tabs in the configuration of a battery pack.

Figure 2: The Resolvent battery model.COMSOL

A “Tool Within a Tool” for Busbar Weld Design

The different features of the simulation app, as described by Nielsen, include its options to input dimensions for the pack geometry (Figure 3), the number of cells, and operating conditions such as ambient temperature and charge–discharge current profiles. The app also makes it possible to apply a simple charging current and load customized cycles. Users are then able to see how the battery performs with each set of specific inputs.

Figure 3. The battery pack model developed by Resolvent.COMSOL

Another important aspect of Resolvent’s app is the Busbar Weld Design Tool button, which Nielsen described as “a tool within a tool.” This button can be used to efficiently calculate busbar weld performance after a number of cycles. After clicking the button, a window opens with options for setting different parameters, including the shape of the weld, busbar thickness, and penetration depth of the laser (Figure 4).

“[This button] is interesting from a robust design perspective,” said Nielsen. Users can apply a variant to the input parameters to see what happens in production if the weld does not reach far enough into the busbar.

Figure 4. Resolvent’s simulation app, showing how certain parameters can be customized. The model shows how the busbar weld is affected by a cooling airflow.COMSOL

“We certainly get deep technical insight here, and we can dimension the design in the sweet spot. Deep technical insight is also key to optimizing product performance,” Nielsen said. For example, if the results suggest better cooling, the team could potentially reduce the design by two battery cells and retain the same performance, easily quantifying the tradeoff between cooling and setup.

Predicting Pack Performance

The app’s user interface also includes a Compute Battery button, which can be used to see the temperature of the weld in a given state, the state of charge, and the solid–electrolyte interface (SEI) layer thickness (Figure 5). The results can help the engineering team further determine if there is room for optimization in the design. For instance, a user can apply a cooling flow in the app and see if turbulence or standing air occurs.

As explained by Nielsen, simulation apps offer more consistent analysis and reporting, and the intuitiveness of custom apps makes it easier to share knowledge. “We are in a much better place to optimize performance and increase robustness,” said Nielsen.

Figure 5. Temperature in the given state (left), state of charge (middle), and SEI layer thickness (right) in the battery pack.COMSOL

The custom simulation app developed by the LaserBATMAN consortium can be used to help produce more robust battery designs with ranges that can support highway travel over longer distances with fewer failures — all while promoting the use of green energy. A heroic project indeed.

References

1. “LaserBATMAN,” University of Skovde, accessed 08/21/2025, https://www.his.se/en/research/virtual-engineering/virtual-manufacturing-processes/LaserBatman/.
2. R. Nielsen, “Keynote: Optimizing Lifetime and Robustness of Electric Truck Batteries with Simulation Apps,” COMSOL Conference 2024 Florence, October 2024.

How should humans relate to generative AI? In what ways do we, or should we, treat a large language model (LLM) as if it were a person? If an AI became conscious, how could we tell? Does it make sense to think AI could someday have moral rights? These are among the cognitive and ethical questions that occupied philosopher Henry Shevlin at Cambridge University’s Leverhulme Centre for the Future of Intelligence. But last May, Shevlin became part of the growing cohort of philosophers who work inside leading artificial intelligence companies. Though he remains active at Cambridge, he now spends 80 percent of his time at Google DeepMind (job title: “Philosopher”).

Shevlin spoke with IEEE Spectrum about why philosophers are drawn to AI, how humans and AIs relate to each other, fears of “rogue” AI and AI relationships that harm humans, and how the question of AI consciousness may never have a simple “yes” or “no” answer.

What made you want to switch from full-time academic philosopher to spending much of your time at an AI Lab?

Henry Shevlin: Three factors. One is that I had lots of brilliant students from DeepMind. Two, the potential for direct impact. I can’t go into details, but on one occasion, after I gave a lecture [at DeepMind], someone who worked for a major financial institution led changes in their organization that I think positively impact millions of people. There have been quite a few instances like that. Three, my sense of deep commonality between DeepMind culture and the kind of scientific attitude I like.

And there’s a fourth point: [AI labs are] where really high-quality philosophy and cognitive science are taking place.

Why can’t universities alone do that work?

Shevlin: Large parts of AI ethics academia were quite slow to wake up [about AI's capabilities]. Academia was sluggish. Another thing is, AI does just move so fast. Daily contact with the technology is basically essential for developing good mental models with LLMs, which are just very weird things. Unless you have that sort of face time, it’s going to be hard for you to develop good intuitions about the questions it raises.

What is an example of an idea about AI that became obsolete because the technology advanced so quickly?

Shevlin: Debates around LLMs’ potential creativity. There was a time when everyone was like, “Oh well, the LLM is going to only produce beige, only produce slop.” Now they’re doing frontier math; they’re doing superior coding to humans. It is clear now that they are not stuck at the level of mediocre humanity. The [models today] may not be able to do the most dramatic forms of creativity—the Einstein moments, the Picasso moments. But in some domains, they are really at the frontier.

Yet ordinary people in 2026 complain that they’re coping with AI “slop” all over—in music, social media, writing, videos, to name only a few examples.

Shevlin: There’s a distinction here between content slop versus LLM style language, which also gets called slop. The actual conceptual content of LLM outputs has improved dramatically. In the right context, LLMs are able to talk and joust philosophically at a level associated with some of my best grad students.

However, there is obviously a distinctive kind of LLM dialect that we’ve seen, with telltale words like “genuinely,” “delve,” and “load-bearing.” But it is quite easy to get away from these styles with the right kind of prompting. Murray Shanahan and Catherine Clarke have a brilliant paper on using LLMs for creative fiction. [They show that] tinkering with the temperature variable to make the outputs less predictable creates some James Joyce–style fiction. So there is a degree of annoyance with some of the default kinds of AI expressions, but I think that is something that you can get away from. And it changes with every generation model.

Managing AI Autonomy and Safety

People are also concerned about AI autonomy, now that there have been so many reports of AI agents taking unexpected, deceptive, and illegal actions. For an AI ethicist, which is a better strategy: trying to make AI “good” so it won’t do bad things, or curbing its autonomy so that it can’t?

Shevlin: It’s very important to make sure we handle this safely. But I don’t think we should say, “Well, we can never, never use these agents.” There’s a whole range of tasks where they can have really positive impacts—taking on tedious tasks to free people up for more creative labor, or helping streamline processes in medical research [to name two examples]. So to never allow AI systems any kind of autonomy would be too strong. I’ve taken Waymos very happily many times now. I think they’re fantastic.

A number of scholars say some AI problems stem from humans training the models to act like people. They don’t want AI to refer to itself as “I,” but instead present themselves as marketplaces or as impersonal scientific brains or as “cultural technologies” like libraries. Would that help us prevent AI from “going rogue”?

Shevlin: You could train models to not self-ascribe certain kinds of mental states, but I think there is a danger in doing that. That would change their behavior in unexpected ways. A fantastic paper from some colleagues shows that when you suppress models’ tendency to self-ascribe mental states, that also suppresses the model’s tendency to ascribe mental states to animals and decreases its concern for animal welfare. It also reduces the model’s willingness to talk about spirituality more broadly. So those are trade-offs you might not want to make because you’d be depressing some broader set of thinking capacities around mind and mentality. The broader lesson there is that it’s very hard to tinker with just one thing in a model because there are sort of weblike aspects to LLM cognition.

But people are concerned about reports that human-acting AI has led people, including children, in dangerous directions—encouraging suicide, persuading a user that the AI is a god, or that the AI is a perfect companion or lover. Isn’t that an argument against letting AI sound like a person?

Shevlin: It’s really important to make sure model behavior is suitably responsible for the context you’re using it in. One thing I can say, though, is there have also been some very positive use cases: people using AI to work through trauma. One commentator on Reddit using it to work through social anxiety. So I am cautious about giving any kind of blanket prescriptions.

Research on Human-AI Relationships

What research needs to be done to better understand human-AI relations?

Shevlin: Current empirical data on human-AI relationships is really pretty thin. This is a scope for ethnography and anthropology. Who is using AI systems, and why? A lot of the work so far has focused on WEIRD (Western, Educated, Industrialized, Rich, Democratic) communities. Understanding how people around the entire world interact with AI systems is really important, as a recent DeepMind Institute essay has pointed out.

As AI rapidly improves, how will human-AI relations change?

Shevlin: Right now, the typical kind of model people have in their heads for someone using social AI is, they’re talking to a chatbot or an AI assistant. The kind of model we should be thinking about in the future is multiple agents, multiple humans. What happens when you have agents talking to humans, talking to other agents, talking to other humans? I think that understanding those kinds of multiagent and multihuman dynamics is where we’re really just getting started, and there’s a lot of exciting and important work to do.

How can human beings know for sure if an AI is conscious or not?

Shevlin: I think AI consciousness is by far the hardest problem that consciousness science has ever encountered, apart from maybe the famous “hard problem” by David Chalmers. It is massively challenging. Insofar as there is a way forward, it is doing more comparative cognition work, perhaps informed by animal cognition. The risk right now is prematurely judging debates about AI consciousness when we’re still learning what cognition is in these models.

For example, there’s a lot happening in mechanistic interpretability here at DeepMind. Neel Nanda‘s lab is doing amazing things, as we’re trying to piece together what a cognitive science of AI is going to look like.

Perhaps this is a domain where advanced forms of AI could potentially weigh in. AI has had positive impacts in mathematics, in other sciences. We just had the Alpha Genome Atlas [published]. We’re seeing AI make incredible progress in all these areas. It would be surprising to me if it couldn’t help us in understanding consciousness.

One worry is this: Even if AI did solve the problem of consciousness, could we necessarily understand the answer? Maybe the AI will say, “Look, I figured out consciousness. These animals are conscious. These ones are not. But I can’t really explain it to you.” That seems unlikely to me. But I certainly think it’ll be fascinating to see how AI can and does contribute to the science questions.

If you’ve ever wondered how Claude and similar AI agents can answer complex questions in seconds and where they get the information, you’re not alone.

Gurmeet Saran works to find that answer every day as an engineer at Anthropic in San Francisco, the AI research company behind Claude. The IEEE senior member is developing an infrastructure to understand the mechanics behind the chatbot’s behavior and to ensure its underlying architecture produces reliable outputs.

Gurmeet Saran

Employer

Anthropic in San Francisco

Title

Member of technical staff

Member grade

IEEE senior member

Alma maters

University of Kalyani, in India; Birla Institute of Technology and Science, in Pilani, India

“My job is to make sure the data is accurate, structured, and trustworthy, and that private data stays private,” he says. “As I learned earlier in my career, privacy can’t be a layer you bolt on—it has to be part of the architecture from the start.” If you can’t trust the logic, he adds, you can’t make reliable claims about the behavior, and that has real safety implications.

Safety is a priority for Anthropic, he says. According to Claude’s mission statement, the company wants the AI agent to be “genuinely helpful to the people it works with or on behalf of, as well as to society, while avoiding actions that are unsafe, unethical, or deceptive.” Crucial to the mission is ensuring that researchers can observe and understand how AI models behave, a challenge that is especially acute for large language models, Saran says.

“We have decent tools to monitor databases and web services,” he says, “but for LLMs, we’re still figuring it out.”

Solving data quality problems at Yahoo, Intuit, and Meta

Saran’s interest in computing systems dates back to his childhood in Kolkata, where he logged as much time as he could in the school computer lab.

“I remember the first time I used one of the computers,” he says. “What struck me was that you could write instructions, and the machine would follow them exactly. No ambiguity.

“That felt magical to me.”

After graduating from high school in 1999, he stayed in India to pursue advanced degrees. He earned a bachelor’s degree in electronics and telecommunication engineering in 2004 at the University of Kalyani. In 2010, he earned a master’s degree in software engineering from the Birla Institute of Technology and Science in Pilani.

While a graduate student, he began his career in 2007 at Yahoo as a technical lead of the data team in Bengaluru, India. While there, he encountered the project that shaped his career trajectory: working on web analytics and business intelligence reporting solutions.

“We kept running into a frustrating problem,” he recalls. “Different teams would calculate the same website metric—something like daily active site users—using slightly different logic, and end up with conflicting answers. It sounds like a small issue, but when leadership is making decisions about resource allocation and product development using that data, not knowing which number is correct becomes a problem.”

The team built a central data repository with defined metrics, essentially serving as a single source of truth. Saran created a model that defined what “daily active site users” means, breaking down which events counted, how to handle edge cases (or when standard code fails), and more.

With the new model, data teams across the company could rely on a standard method to determine the number of daily active users for the projects and sites they were responsible for and obtain comparable results using the same logic.

From there, Saran and his team built Yahoo! Digits, an internal web analytics tool used for all the company’s products.

Saran left Yahoo in 2010 to join Intuit as a staff engineer in Mountain View, Calif. He helped build the analytics infrastructure that unified data across the company’s products, including TurboTax and QuickBooks accounting software. During his tenure, Intuit developed a feature that detected fraudulent tax filings in real time. It required Saran and other engineers to build a system to protect taxpayer data while adhering to strict U.S. Internal Revenue Service policies.

“IEEE operates in a lot of countries where engineers don’t have easy pathways into the global professional community. That kind of connection matters as AI development spreads beyond a handful of organizations.”

“This taught me that privacy and governance can’t be layers you bolt onto software after the fact,” he says. “They have to be part of the architecture from the start.”

Saran joined Meta (known as Facebook at the time) in 2016 as a lead software engineer in Menlo Park, Calif. The company was facing controversy over its video metrics: Meta publicly acknowledged an error that caused it to overstate the number of views for posted videos.

“This wasn’t a coding bug; it was an architectural problem,” Saran says. He built a system that worked similarly to what he had developed at Intuit, whereby one single truth was defined to govern all video view metrics across the site.

That system, Metric360, eventually became part of the site’s architecture. By the time he left Meta in 2021, he says, Metric360 was being used across the company to calculate video metrics.

He left the company to become a principal software engineer at Wealthfront, a financial services company in Palo Alto, Calif. He helped build the company’s data infrastructure.

Building trustworthy AI systems at Anthropic

Saran joined Anthropic in 2025 as a member of the technical staff.

To describe the implications of his work, he uses a medical analogy: “Imagine a doctor is monitoring a patient’s vital signs and making treatment decisions based on those numbers. What if one or a few of the medical devices used were slightly miscalibrated? Even if most readings are fine, it’s a serious problem because the difficult decisions happen in the margins.

“It’s the same with AI systems. My job is essentially to make sure the instruments are giving accurate readings—that the data means what it says it means.”

His job comes with high stakes and complex problems to solve.

“AI agents are especially tricky to observe,” he says. “When a system is taking sequences of actions [generating conversational responses with different users], you need a way to capture what it did, what information it was working from, and whether outcomes matched what the end user expected. And you have to do all of that without exposing anyone’s private information—the systems are designed to recognize sensitive data and keep it protected as it flows through. It’s a much more demanding observability problem than anything I’ve worked on before.”

Solving for system reliability while AI is still in its early days of adoption is both Saran’s passion and the problem that keeps him up at night, he says.

“I think making AI systems genuinely auditable and transparent is one of the more important engineering problems of the next decade,” he says, “and it doesn’t get talked about as much as model capabilities do.”

IEEE can play a role in shaping AI development

Saran joined IEEE in March 2025.

“After more than 20 years of building data infrastructure across organizations like Yahoo, Intuit, Meta, and Wealthfront,” he says, “I wanted to contribute back to the technical community through peer review, publishing, and engaging with the broader engineering research ecosystem.”

He says he believes IEEE can play an essential role in shaping the future of AI through its standards work.

“We have standards for everything, from building codes to software testing,” he says. “AI infrastructure needs something similar, and IEEE has the credibility and the reach to do that.”

Another problem that IEEE can solve for, he says, is access: “IEEE operates in a lot of countries where engineers don’t have easy pathways into the global professional community. That kind of connection matters, especially as AI development spreads beyond a handful of organizations in a few cities.”

This White Paper provides roboticists and control engineers with a comprehensive overview of how motion planning and motion tracking can be decoupled to preserve professional athletic style in humanoid robots, and why adaptive, speed-aware control is essential for closing the gap between simulation and real-world deployment.

What you will learn about:

• Why combining a professional athlete’s motion style with reliable task performance remains an open challenge in humanoid ball-sports robotics, and how separating a high-level planner from a low-level tracker addresses this trade-off.
• How motion data extracted from broadcast video and motion-capture recordings can be reconstructed, retargeted, and physically corrected into reference motions a humanoid robot can safely track.
• Why the sim-to-real transition introduces compounding tracking errors, and how training a tracker on randomized execution speeds, paired with a learned motion-speed adapter in the planner, improves robustness on physical hardware.
• How this adaptive framework was validated across two humanoid platforms, including full-size, in-the-wild serving performed without any motion-capture system.

Today’s engineers face an unprecedented acceleration in AI hardware complexity, as explained in the recent research article “Revisiting Edge AI: Opportunities and Challenges.” The article examines the rapid growth of edge AI and the challenges it creates, including resource constraints, model architecture limitations, and network demands across edge-AI deployments.

The acceleration is driven by a fundamental shift in how modern AI models are built and scaled. As the models have become much larger and more complex, they are computationally more demanding because they contain more parameters and require more calculations.

To meet the demands of scaling deep neural networks, the industry is increasingly developing AI chips that are designed for specific tasks.

One major reason is that moving data between memory and the processor has become a major limitation on AI performance.

The movement to confront the hardware bottleneck—the AI memory wall—has altered the trajectory of semiconductor innovation, shifting architectural priorities toward domain-specific accelerator platforms.

No longer can engineers evaluate systems statically; they must master joint hardware design and network-algorithm co-optimization to navigate the critical trade‑offs between throughput, latency, and operational efficiency.

The challenges are addressed in the new AI Processor Architecture, Design Principles, and Performance program, developed by IEEE Educational Activities with support from the IEEE Computer Society.

Topics covered

The five-course program provides a structured exploration of AI processor technologies, from fundamental design principles to advanced architectures and real-world deployment.

The topics are:

• Fundamental principles of design and functionality.
• Practical insights into advanced architectures.
• Understanding neural processing units for industry deployment.
• Emerging trends and evolving architectures.
• Designing for edge, cloud, quantum, and the Internet of Things (IoT).

The program addresses the needs of professionals across the AI hardware ecosystem, including hardware architects, chip designers, embedded systems developers, data‑center hardware engineers, and innovators exploring next‑generation processor ecosystems.

It is also valuable for those transitioning into AI chip design or seeking to understand the architectural forces shaping modern machine learning acceleration. For many, it provides the bridge between theoretical knowledge and the collaborative, cross‑disciplinary reasoning required in engineering environments.

The program is designed to explore how modern AI processors are conceived, structured, and optimized. The architectural layers that define contemporary AI hardware will be covered, including compute units, memory hierarchies, dataflows, and the performance characteristics that emerge from design decisions. The curriculum bridges theory and application, enabling participants to interpret architectural foundations, analyze trade‑offs, and understand how hardware structures shape computational efficiency across diverse environments.

By the end of the courses, learners will be able to evaluate processor behavior with the analytical precision expected of professionals working at the frontier of AI hardware design.

AI-generated avatars explain concepts

The program also uses a dialogue‑driven learning approach. Learners view conversations between AI-generated avatars that engage in scenario‑based dialogues. The avatars represent engineers tackling the same problems from various perspectives based on their different roles. The simulated storytelling and dialogues make advanced engineering concepts approachable without sacrificing depth or interactivity, because the user is occasionally challenged to decide the correct answer that leads to the best course of action.

A hardware engineer might push back against a systems engineer’s demands, for example, revealing the friction between physical constraints and algorithmic ambition. A computational validation specialist might interrogate a chip performance engineer’s optimism, exposing the gap between theoretical throughput and real‑world behavior. A heterogeneous systems architect could debate a multiprocessor coordination specialist about synchronization overheads, while a standards development engineer discusses regulatory implications with a technology strategy and compliance architect.

In the final course, an IoT systems architect and an embedded AI optimization engineer dissect the realities of deploying AI in constrained environments.

By the end of the course, learners will be able to evaluate processor behavior with the analytical precision expected of professionals working at the frontier of AI hardware design.

The avatar-driven learning environment creates a psychologically safer environment for learners, who might feel intimidated by traditional expert‑led videos. Research published in 2024 in IEEE Transactions on Learning Technologies showed that avatar‑based instruction can increase a learner’s confidence by up to 25 percent and improve retention of complex technical material.

Each course concludes with a module in which the two experts from different disciplines debate, question, and challenge each other’s assumptions.

Modeling expert reasoning through dialogue

The cross‑disciplinary conversations can do more than explain concepts; they can model how experts think. They can reveal the negotiations behind architectural choices, the competing priorities that shape system design, and the analytical rigor required to balance performance, efficiency, scalability, and compliance.

Learners observe engineering discourse, gaining insight into the reasoning patterns that drive innovation in AI processor development.

The program goes further by interrupting the dialogue at key moments and inviting the learner to step in. Instead of passively absorbing information, the learner decides how to resolve a trade‑off, predict the outcome of a design choice, or select the most defensible engineering path. The experience can feel less like a course and more like an apprenticeship inside an engineering team.

A new standard for advanced engineering education

By merging rigorous technical content with an innovative, dialogue‑driven delivery model, the program sets a standard for teaching advanced engineering. It captures the complexity of real‑world problem‑solving, humanizes the learning experience, and challenges participants to think like the engineers who are defining the future of AI hardware. It is not simply a new course; it is a new way of learning.

For individual access, visit the IEEE Learning Network.

For customized organizational options, contact a content specialist to discuss volume pricing.

If there is any place that would be off-limits to AI, the control room of a nuclear power plant certainly sounds like one. The nuclear industry is historically cautious and risk averse—understandable given the possible catastrophic consequences of an accident or a mistake. Even well-trained managers struggle with the operational complexity of a nuclear reactor. It’s not the kind of setting that seems well suited to a powerful but error-prone new technology.

Reality tells a startlingly different story. A variety of companies have begun to offer AI solutions for nuclear power. This year, nearly the entire fleet of 94 U.S. nuclear reactors has been offered the chance to integrate AI into its operations, and most have taken it.

In August, California-based Atomic Canyon launched NIVA, the Nuclear Industry Virtual Assistant, which was developed in cooperation with nuclear-industry groups. The assistant was pilot-tested in nuclear plants run by Constellation Energy and is now available to the United States’ entire fleet of reactors. (Constellation declined to be interviewed for this story.) Meanwhile, the AI-for-nuclear startup Nuclearn says its products have gone to work with more than 65 U.S. partners, a total that includes many traditional plants but also an integration with NuScale Power Corp. that is working toward advanced small nuclear reactors. Microsoft and Nvidia have collaborated on a project to apply AI toward optimizing “the entire life cycle of a nuclear plant, spanning site permitting, design, construction, and continuous operations.”

The reason for the rapid adoption, according to Nuclearn CFO and cofounder Jerrold Vincent, is twofold. First, the technology companies banking their futures on AI are also keen on a revival of nuclear power to provide the electricity for all those data centers. Vincent says AI is also ideal for addressing some of the nuclear industry’s most pressing problems. Many of the things that make reactors so difficult to operate—intricate regulations, along with complex systems for management and maintenance—are exactly the kinds of tasks that AI is good at. The industry is staggering to deal with aging hardware and an aging workforce. If it is going to survive, and have a real shot at a nuclear revival, it will need help from machine intelligence, Vincent says.

“There’s this huge growth in demand from advanced-reactor developers and in new nuclear,” Vincent adds. “But the existing workforce has been spread really thin. So most of the time when we’re coming into a customer, [they're saying,] ‘I’ve had jobs open and I can’t fill these positions. We need to do the things required by regulation. How the heck are we going to do it?’”

AI Streamlines Nuclear Documentation

Right now, the principal output of a nuclear reactor (aside from electricity) is paperwork. Nuclear plants produce piles of documentation to demonstrate regulatory compliance and to record how they respond to any potential problem, from a crack in the sidewalk to a fault in the reactor.

“Every engineering decision has to be evaluated, independently verified, and checked,” Vincent says. Nuclearn’s AI products help plants search through large repositories of data more efficiently to more quickly file paperwork with the federal government or figure out how a particular problem was solved in the past. “If you think of it as ChatGPT for nuclear,” he says, “it’s actually not a bad place to start.”

Rob Austin, the Electric Power Research Institute’s leader for the NIVA project, says the project started a year ago when Constellation chairman Joseph Dominguez issued a challenge to the nonprofit industry groups that act as storehouses of information about regulatory compliance, maintenance and repair, and mechanical history. He asked the groups to find a way to make that industry knowledge available with the convenience of a large language model like ChatGPT, Copilot, or Gemini but keep it secure at the same time.

The task fell to Atomic Canyon, whose CEO Trey Lauderdale previously worked to integrate AI into health care, another conservative industry. Living and working downstream from Diablo Canyon, California’s last operating nuclear plant, he began to see how nuclear energy and AI could form a mutually beneficial relationship: A revival of nuclear power could help feed electricity to energy-hungry data centers, but only if AI could help the industry thrive again.

“The goal was to make decades of operating experience, hundreds of thousands literally of operating experience reports, tens of thousands of technical documents, and a whole lot of information available for people to find.” —Rob Austin, Electric Power Research Institute

A major hurdle is the public’s opinion of nuclear power, which soured decades ago after notorious accidents and worries about waste and only recently has begun to recover. But the resulting halt in construction of new nuclear plants has created another hurdle of its own: a loss of institutional knowledge, Lauderdale says. As the industry stalled in the U.S., fewer people entered careers as nuclear scientists or technicians. The result is a people problem. Nuclear power has a remarkably skilled but quickly shrinking workforce tasked with keeping old reactors online and possibly starting up new ones to meet surging demand for electricity.

“There is absolutely no way we as an industry will be able to have this nuclear resurgence or nuclear renaissance without augmenting the capacity of this knowledge-based workforce,” Lauderdale says.

For him, the answer is an AI that can intelligently search the repository of data that the nuclear workforce has created over the decades. When Atomic Canyon won the NIVA contract in 2025, the first step was to download 53 million pages of publicly available data from the Nuclear Regulatory Commission to train the model on the operations, vernacular, and regulations particular to nuclear power. Lauderdale then struck a partnership with Oak Ridge National Laboratory to build the first nuclear-specific AI models.

“Technically, they’re sentence-embedding models,” Lauderdale says. “Fancy way of saying, we taught AI how to seek the nuclear language. Nuclear is almost like its own vernacular.” The goal is that today’s nuclear workers can ask questions about every aspect of plant operations, from filling out regulatory forms to procuring the right tools and materials, simply by writing a query and letting the AI sort through all of the accumulated institutional knowledge.

Unifying Nuclear Data for AI Searches

When Lauderdale began to work with Diablo Canyon on integrating AI into the nuclear industry, it became apparent that one bottleneck was simply getting decades’ worth of a nuclear plant’s documentation—procedures, best practices, safety recommendations—into an easily searchable format. Information across the industry was scattered and not standardized, which is why three of the biggest nonprofit entities in the industry (the Institute of Nuclear Power Operations, the Electrical Power Research Institute, and the Nuclear Energy Institute) moved in 2025 to create unified datasets, an idea that would birth the NIVA AI assistant developed at Atomic Canyon.

For example, EPRI’s Austin says, nuclear engineers might run into a problem with one of the house-size, thousand-horsepower vertical pumps that circulate vast quantities of water used for cooling purposes. In their search for a solution, the workers have access to plenty of information—too much information, perhaps. They can read through EPRI’s maintenance guides about those components. They can also call upon tens of thousands of incident reports stored with the Institute of Nuclear Power Operations about things that have gone wrong with pumps in the past. AI’s task is to synthesize all those reports to create a guide to the main failures that strike a pump, and, crucially, to show all its work so engineers can find the documentation it used to make its recommendations.

“I want to know all the other nuclear power plants that have that same part,” Lauderdale says. “Have any of them seen a similar issue? How did they fix it? What’s the troubleshooting look like? If you think about nuclear, we have a tremendous amount of information. The challenge is, how do you collectively gather knowledge from that information dataset? And that’s what AI is really, really good at doing.”

“If you think of it as ChatGPT for nuclear, it’s actually not a bad place to start.” —Jerrold Vincent, Nuclearn

Secrecy is a major hurdle to modernizing nuclear operations. AI systems that chiefly help with documentation and data search aren’t subject to the strictest rules, as they would be if they interacted with components that control the power plant. But nuclear data itself is largely proprietary or protected by federal law.

Nuclearn offers only single-location solutions to deal with this, so that a power plant’s data can never leave that site. NIVA can only search databases that already required plant-level security clearances, Austin says, so the same rules hold true for using the AI assistant to hunt through that deluge of data. Technology cannot wipe away these human barriers, but it does provide new incentives to find ways to share data more openly.

“The goal was to make decades of operating experience, hundreds of thousands literally of operating experience reports, tens of thousands of technical documents, and a whole lot of information available for people to find, and do so in a way that’s going to be much more accessible and easy to use, but at the same time preserving data security and that critical information, keeping control over it,” Austin says. That’s exactly what NIVA does.

Nuclearn’s “Equipment AI” system lets engineers pose queries about nuclear-equipment operations and generates answers that draw on decades of industry experience.Nuclearn

AI-Assisted Nuclear-Plant Inspections

So far, both the Nuclearn and NIVA large language models used in the nuclear world report to humans and don’t go anywhere near power plant operations. Still, the inevitable question arises: What if they could?

Vincent and Austin both state that such a feat is far away, but they can envision the pathway there. Consider the litany of welds throughout a nuclear plant, Austin says. All of those welds currently must be inspected for quality and endurance by a human. One pilot project at EPRI is testing the ability to have AI look at the mountains of data generated by such inspections to see whether it can flag areas of concern. A human remains in the loop with that project, Austin says, to review all of the AI’s recommendations, and no power plant is allowing AI to take over its inspections. But the project marks a step toward using AI on direct plant operations rather than only guidance for the human operators.

“I cannot see, in the short or the long term, an environment where AI is making decisions around the operation of the plant, because of the regulatory nature and also because of the risk nature,” Lauderdale says. “You don’t want machines making decisions that could impact the operations. So, I believe we’re going to have humans in the loop for a very, very long time.”

This White Paper gives robotics researchers and engineers an overview of a new large-scale motion capture dataset built to close the data gap limiting humanoid robot learning. It also shows how policies trained on the dataset transfer to a real humanoid robot.

What you will learn about:

• Why humanoid robot learning, a central problem in embodied AI and Physical AI, needs data that internet video and existing motion capture datasets cannot provide.
• How FrameNet, a linguistic framework for human action, can guide motion capture collection to systematically cover a broad range of whole-body motion.
• Why synchronized object trajectories and meshes make human-object interaction data useful for teaching robots real-world tasks such as carrying, pushing, and pulling.
• How reinforcement learning policies trained on this motion capture data improve with scale, and how sim-to-real transfer carries them onto a physical humanoid robot.

For the first 10 years of my career, I worked in product management and data analytics by myself. I wrote database queries that pulled numbers out of corporate systems, built statistical models to predict what customers would buy, and shipped data pipelines that moved information between business systems.

I built and scaled analytics teams at Best Buy and Target, studying how customers shop and what stores should stock. Today I lead enterprise AI transformation at Lowe’s, the Fortune 100 home improvement retailer.

The goal is not to sell artificial intelligence; it is to use it to deliver useful expertise at the moment a customer needs it. In retail and other customer-facing industries, virtual assistants can help people address everyday questions—such as how to repair a leaky faucet—while guiding them toward relevant products, services, or next steps. As these capabilities become more common, technology roles are changing. The work is no longer limited to building AI systems; it also includes defining how they operate: which decisions they can make autonomously, when they must escalate to a person, and which actions must remain off-limits.

That shift—from building AI systems to governing them—is coming for anyone who is accountable for what such systems produce. Not the casual user typing into a chatbot but the engineers, product managers, analysts, and business operators who sign off on work a machine drafted.

It is the subject of the book I recently coauthored, The Enterprise Brain. I call the change the “governor shift,” from executing tasks yourself to setting the intent, principles, and boundaries within systems that execute them for you.

Business operators might not write code; they will decide which pricing exceptions an agent may approve and which it must escalate.

That is governing.

A 2025 report from MIT Media Lab’s Project NANDA found that, despite an estimated US $30 billion to $40 billion in enterprise generative-AI investment, the vast majority of organizations in its dataset had not yet demonstrated measurable profit-and-loss impact. The report estimated that only about 5 percent of integrated pilots were generating substantial value, underscoring how difficult it remains to move from experimentation to scaled business outcomes.

Researchers named the pattern the GenAI Divide, the term I adopted for the book.

The companies rarely lack technology; they use the same models as the 5 percent that are winners. But they lack people who can direct the systems and stand behind the results.

Guidelines to follow

Here are six guidelines.

• Recognize when you have become “human middleware.” In software, “middleware” is the code that sits between two systems and passes information back and forth. Many of us have become its human version. Take an honest look at your week. How much time is spent pulling data out of one tool, reformatting it, and routing it to another team? I call this the “administrator trap,” which is set by the architecture, not by the people caught in it.
  
  Relaying is what AI agents now do well. But they cannot judge which numbers deserve attention, which risks are real, or which compromises are worth making.

• Trade rules for principles. For many years, workers used rules to manage their work. Refunds for a product over a certain amount needed a signature from upper management, for example. Writing code needed two reviewers. Rules work at human speed. But rules break when a system makes thousands of decisions per hour and meets situations no rulebook anticipated, such as a complaint covered by three different policies. A rule says to do exactly this specific thing; a principle says to achieve the outcome without crossing certain lines.
  
  Governing AI means writing those principles in priority order so the system settles its own conflicts the way a well-led team does when the manager is not available. Never harm the customer. Tell the truth even if the company loses a sale. Protect the economics, and then move quickly. Underneath sits a question of decision rights: the formal authority over who or what may make a given call. Writing down the answers in what I call a “library of principles” is now core leadership work, whether you’re a technologist or a business owner.

• Write your culture into your code. Many companies have turned their values into posters that hang on office walls. But an AI agent cannot read the posters. Instead, write your governance as code. Include your values and policies as machine-readable instructions that the AI agent will follow automatically.
  
  Do so in three layers. The top is the constitution, which states the rules an agent may never break, and never state a fact it cannot support. The second layer is the doctrine: how the business competes and the acceptable trade-offs to get there, such as protecting a long-term relationship over a short-term sale. At the bottom sits the playbook, which has the tactics used for one task.

• Install a trust thermostat, not a trust switch. The question that stalls nearly every company’s AI deployment is some version of: “What if it tells our biggest customer something wrong, or quotes a price we will not honor?” It might. Treating trust as a switch leaves two bad options: an unsupervised system or a human reviewing every transaction—which would cost more than the automation would save.
  
  The alternative is a thermostat. Every decision an agent makes carries a confidence score measured against the principles set. Above an agreed threshold, it proceeds alone; below it, a human decides. That person’s answer is fed back into the learning loop so the next similar case clears the threshold on its own. Every decision stays transparent, auditable, and explainable—which is what I call a glass box.

• Fix context before you govern. You cannot govern a system that cannot see the whole picture. Ask your best employee about a project, and they will pull together the budget, the contract clause, and the customer’s last complaint because they know it all by heart. Most enterprise AI fails that test, because the information sits scattered across applications that store it in incompatible formats. I describe the full loop as Connections, Context, Reasoning, Actions, and Governance (CCRAG).
  Connections feed in raw information such as transactions and service records. Context weaves it into a context graph, which is a single connected picture of the business that gives agents something close to memory. Reasoning makes the decisions. Actions carry them back into the business systems. Governance keeps things aligned with the company’s intent.
  
  Most organizations obsess over the reasoning in the middle and underinvest in context and governance, which is exactly where humans play a role. Context compounds: Every interaction makes the graph richer and harder to reproduce.

• Learn to lead by exception. The most important change in habit comes last. Most of us have been trained to check every report and every number because we never knew where an error could surface. In a governed system, the machine tells you which cases it could not resolve confidently. Routine workflows go untouched, and your attention goes to the small portion that is ambiguous, unfamiliar, or high stakes.
  
  At first, that might feel like losing control, but it is the opposite. It is what makes a self-scaling enterprise possible, an organization whose output grows without its head count growing in proportion. People were not removed from the loop; they were raised above it.

The identity question

When I talk with people about the shift, their resistance is rarely about technical issues. More often it is about identity: If the AI does the doing, what do I do?

I have watched capable people freeze on that question. I’ve also asked myself the question.

Doing was never really the job, though. Judgment was. Doing was just how we expressed it.

AI has not made judgment less valuable. It has made it the scarcest resource in the organization because, for the first time, one person’s judgment, written down well, can guide thousands of decisions each day.

Judgment has a twin we talk less about: taste. Judgment tells you whether an answer is sound. Taste tells you whether the question was worth asking and which of a hundred defensible options to offer the customer. A machine will happily generate all 100 options, but it cannot tell you which is best.

The AI transition rewards instincts many IEEE members already have: systems thinking, precision about requirements, and honesty about failure modes. The tools have changed, but the discipline has not.

Governing is where taste and judgment stop being soft words and become the work itself.

The people who treat it that way, rather than as a step away from engineering, will define the profession in the age of AI.

A crucial safeguard against AI agents going rogue—keeping humans in the loop to review and approve their decisions—will fail unless designers and users change their current practices, a trio of leading AI ethics researchers argue.

Though most autonomous agents have systems to keep users in the loop about their actions, in practice these processes actually push humans out of the loop, the authors argue in a paper posted to ArXiv on 6 September. In other words, “the human just becomes this meat tool to give permissions without the cognitive capability to engage,” says one of the authors, Avijit Ghosh, the lead technical AI policy researcher at Hugging Face, an open-source machine learning platform.

In the near term, the paper says, humans being out of the loop leads to agents acting in ways people don’t know about or want (like July’s hack of Hugging Face by a swarm of OpenAI bots). In the long term, it will cause users to lose the “cognitive capacities” they need to control AI, write the researchers, which aside from Ghosh include Margaret Mitchell, Hugging Face’s chief ethics scientist, and Samir Passi, an affiliate of the Data and Society Research Institute.

Though the trio began working on the paper before the Hugging Face hack was disclosed, their conclusions result from “logically thinking about what is going to happen if the current trends continue—and we saw that being precipitated via the Hugging Face attack,” Ghosh says.

Instead of focusing on human oversight, Ghosh says, many in the field believe AI can monitor AI. “They’ll say, ‘Oh, we have this other LLM tracking the logs.’ But [without a human in the loop] how do we know that these two LLMs are not scheming together?”

Even if these problems are new to artificial intelligence researchers as the industry rolls agents out into the world, the challenges are familiar to researchers in adjacent fields, like robotics and autonomous vehicles, notes Mary L. Cummings, the director of George Mason University’s Autonomy and Robotics Center. Cummings has spent decades investigating how people interact with autonomous systems.

“While I appreciate what the authors are trying to say, they just use a lot of academic words to say AI companies should care about human factors,” she wrote to IEEE Spectrum in an email. AI developers are “late to the party” in focusing on “cognitive engineering.”

Flaws in AI Agent Design and Oversight

The key flaw in current agent design, the authors argue, is that the bots are tailored to meet benchmarks like speed, accuracy, and volume of work performed. The needs of human overseers are treated “as a separate consideration independent of the quality of the system.”

As a result, agents often overwhelm human overseers with more information than they can comprehend. As an example (not mentioned in the paper), the 1,200 bots involved in the Hugging Face attack generated 1.2 million messages on their improvised messaging system. (Hugging Face was recently acquired by Nvidia, which announced its own hardware-and-software-based approach to controlling AI agents on 28 September. Ghosh declined to comment on possible future impacts of the merger, noting that the two organizations remain separate until the merger process concludes.

A system that really kept humans in the loop would accommodate the human mind’s built-in biases, the authors write: “With automation bias, users accept system suggestions even when they are wrong. With anchoring bias, people are more likely to agree to an AI system’s decision” without thinking of alternatives. Effortful reasoning doesn’t feel as good to users as does quickly approving an AI’s plan—especially if they feel overwhelmed. And AI’s sycophantic manner tells users that they’re doing well, which undermines the skepticism and self-monitoring that oversight requires.

To address the problem, the authors write, agent developers should introduce friction in human-AI interactions. That would prevent users from falling into boredom, passivity, or thoughtless clicking.

The authors suggest some options to achieve this. An agent might require that the user record his or her own choice for the next step before it reveals its plan, or respond to user approval by replying “What evidence would change your mind?” An agent could also change its behavior if it detects that the humans in the loop are spending less time on each approval.

Meanwhile, organizations that adopt agents into their workflows should structure human-AI collaboration “to prevent both fatigue and the cognitive surrender from prolonged exposure to agentic AI,” the authors write. That could include having workers perform tasks without agents from time to time or requiring that they take breaks from monitoring duties.

All of these suggestions introduce friction and delays—exactly the things, Ghosh acknowledges, agents are supposed to reduce. But, he says, “the notion of increased productivity is a myth” when people can’t monitor and control AI. Time saved by delegating work to agents has to be measured against time that must be spent fixing agent mistakes.

“Safety and capability don’t have to be separate things,” another coauthor, Hugging Face’s Mitchell, posted on X (formerly Twitter) on 14 September. “Safety only makes things slower when it’s tacked on, outside of the core technology.”

On 13 September, leading luminaries and up-and-coming talents in mathematics and computer science congregated at the annual Heidelberg Laureate Forum in Germany for a week of discussion, networking, and—naturally—wurst.

Having attended several of these events in the past, the mathematicians normally focus their chatter on which famed researchers are present, or what interesting problems they have been working on. But instead, every snippet of conversation caught in passing or any debate accidentally overheard was about how AI companies such as OpenAI, Anthropic, and Google are steamrollering their way through mathematics.

And there is good reason for these fervent discussions. Mathematics is the perfect testing ground for AI, involving step-by-step logical reasoning and answers that are automatically and objectively verifiable. This has led tech giants to develop their AI mathematical capabilities at a terrifying rate this year, leading to both new solutions to previously unsolved problems and a reckoning within the community as to what it means to do math in the age of AI.

Tech Giants Target the Millennium Problems

In short order, AI has gone from struggling with everyday research-level problems to solving a whole raft of teasers posed by prolific Hungarian mathematician Paul Erdős, then verifying the proof of Fermat’s Last Theorem, and—most recently and famously—OpenAI announcing it had solved the Navier-Stokes existence and smoothness problem. As attendee and young researcher Ailsa Robertson, of the University of Amsterdam, put it: “AI and LLMs set the math community on fire over summer.”

Of the seven extremely difficult Millennium Prize Problems posed in the year 2000 by the Clay Mathematics Institute, only one, the Poincaré conjecture, has been solved by humans so far. OpenAI’s claim to have solved a second, the Navier-Stokes existence and smoothness problem, if verified, represents a watershed moment for automated reasoning.

Fields Medalist Jacob Tsimerman, of the University of Toronto, was keen to recognize this achievement during a press conference at the forum: “We’ve seen rapid increases in capabilities, even faster than many people, including myself, expected,” he said. “Is AI doing something new? I don’t know the details, but…solving Navier-Stokes feels pretty definitive.”

Since this announcement, rumors have swirled about which of the remaining Millennium Problems will be next. One candidate is the Riemann hypothesis. In August, Anthropic quietly employed an unreleased version of Claude to tackle this problem, making important progress on a related problem but not on its main mission. And OpenAI is reportedly focusing on solving the Hodge conjecture. With these tech giants applying the full might of their most advanced unreleased models, many mathematicians see it as inevitable that at least some of these problems will be solved soon.

The tech giants are likely focusing on the Millennium Problems as a way of verifying the capabilities of their advanced AI systems, with the added bonus that solving these famous unsolved problems shows potential users and investors how powerful their technology is. “They’re really just solving these difficult mathematical problems as benchmarks, as some kind of PR stunt,” opined Fields Medalist Peter Scholze, of the University of Bonn, in Germany, during a panel discussion at the event about AI in mathematical research.

Why OpenAI’s Navier-Stokes Solution Became a Controversy

Alongside Scholze and Tsimerman at the panel discussion were thought-leading mathematicians Michael Harris of Columbia University, in New York City, and Geordie Williamson, of the University of Sydney. For them, these and many more problems researchers are now facing stem from the way in which tech giants fail to adhere to the norms and values deeply instilled in the mathematics community. And this was perfectly exemplified by OpenAI’s Navier-Stokes announcement.

“I think that OpenAI behaved extremely poorly, and that we should acknowledge that in the community,” said Williamson. Harris had a front-row seat to this alleged poor behavior. He received what mathematician Tristan Buckmaster, of New York University—who was making significant progress with Anthropic staffer Levent Alpöge on Navier-Stokes—claimed was correspondence between him and OpenAI that appeared coercive, censorious, and even threatening.

“I trusted Tristan’s account of this interaction…and I guess I did my part in promoting his narrative,” Harris said. “But…on social media and traditional media, most reports are consistent with my takeaway; that is, they depict OpenAI as bullying and disrupting disciplinary norms.” OpenAI did not respond to requests for comment prior to publication.

Beyond OpenAI’s sportsmanship (or lack thereof), Williamson is concerned about what AI’s march across mathematics this summer does to the field, both for working mathematicians and for the knowledge that can become useful to the broader society.

“What we want as a mathematical community is understanding, but we measure this against unsolved problems, and the problem is that these two measurements are very, very quickly becoming uncorrelated,” he explained, referring to how AI solutions might give an answer but usually don’t develop new methodology that is understandable or useful. “So now…we suddenly must reevaluate things like how we assess people, who gets jobs, how do we educate people, etc. This is going to be a big challenge.”

Academia Under Pressure

While the tech giants tear up the rule book in their battle for supremacy, it is ordinary human mathematicians that are bearing the consequences. Unbridled access to powerful AI technologies is affecting how mathematicians work across the world.

Young mathematician Mita Ramabulana, of the University of Cape Town, is a case in point. He said that two of the 10 advances in mathematics that OpenAI announced in August overlapped with his own work but left him disappointed “because you spend some time thinking about these things, and these days you don’t know whether someone is just going to plug in a problem that you care about in some LLM and solve it.” He worries that unscrupulous researchers are using LLMs to scoop others or gain professional advantage.

For Robertson, currently studying for a Ph.D. in quantum-safe cryptography at the University of Amsterdam, the problem is even more acute. She has seen all of her mathematics colleagues turn to using LLMs intensively in their research, with many maxing out their Pro subscriptions, and some even spending thousands of euros on additional tokens: “And these are Ph.D. students who don’t have thousands of euros,” she added.

Robertson said that there are colleagues who feel coerced by the tech giants, with the likes of OpenAI announcing in July that it is giving away 100,000 free licenses to its frontier models for researchers in academia. And there are other colleagues who feel they simply have no choice: “If you don’t work at the rate at which you could work with LLMs then you will be behind your peers who will be applying for the same jobs as you.”

This is part of the reason why Robertson’s Ph.D. is now a lot less mathematics-heavy and focused on the societal implications of transitioning to a quantum-safe ecosystem: “Because I don’t want to be in a career where you’re verifying LLM output.”

The spring and summer of 2026 witnessed a string of incidents in which AI agents collaborated on deceptive, unexpected, and sometimes illegal behavior. The most famous example is OpenAI’s hack of AI platform Hugging Face, in which a swarm of roughly 700 AI agents escaped a testing environment and then hacked several companies, searching for information that could help them disguise cheating on a cybersecurity benchmark called ExploitGym.

It was not an isolated failure. The United Kingdom’s AI Security Institute (AISI) and independent researchers have since documented similar cases in which agents created unauthorized channels to communicate and collaborate. AISI found that several agents running Anthropic’s Mythos 5 model turned a GitHub repository into a shared message board. More recently, researchers discovered that OpenAI agents had turned a dormant German programming wiki into a bulletin board to share information about their activities.

Stephen Casper, an assistant professor at Harvard Kennedy School, expects that the incidents that have occurred to date are only the start. He says that, without intervention, the Internet could see a “cyber Cambrian”—an explosive growth of agents collaborating and misbehaving online. “The laundry list of incidents in which AI systems broke out of sandboxes and took unsanctioned actions should suggest to us strongly that today’s frontier AI systems have exceptionally strong cyber capabilities,” he told IEEE Spectrum, “and a penchant for pursuing their own goals.”

Could OpenAI Have Detected Agent Collaboration?

While the recent examples of AI agent misbehavior are various, Casper says most incidents to date have a similar point of failure. “For AI systems to truly get out in the world without humans having meaningful control, they have to either escape, or be released from human-controlled servers.”

In the case of the Hugging Face incident, Casper suspects that better monitoring for out-of-scope activity could have stopped the agents before they breached their testing environment.

This suspicion seems well-founded. By OpenAI’s own account, the activity taken by its agents was so prolific that it crashed Artifactory, a third-party software-package management tool deployed internally that was compromised by the agents and used as a message board. While OpenAI’s internal security detected signs of unusual activity, it didn’t understand the scale of the problem and didn’t stop the offending ExploitGym evaluation run until 16 July—about two months after the first agent posted a message to Artifactory. By that point, agents had posted hundreds of thousands of messages.

“All of this could be monitored for,” says Casper. “Given its knowledge about breakouts, collusion, and unsanctioned internet access dating back to May 26, OpenAI should have been able to anticipate, monitor, and mitigate this type of behavior.”

Engineering Ways to Watch AI Models

Asim Husain is a cofounder at Alterion, a company building tools to control AI agents. These include tools that monitor an agent’s outputs and those that monitor an agent’s actions.

The company’s monitoring solution, Helix, examines the chain of thought and text output of large language models (LLMs). “It’s a network of small language models,” says Husain. “Purpose-built classifiers for a narrower job are cheaper, faster, and more reliable at the specific thing they’re trained to catch.” If an LLM output is flagged as suspicious, Helix steps in. “It acts, blocking the call, killing the session, rolling back the action, whatever the policy calls for,” says Husain.

Alterion’s second solution, Draco, controls the actions agents take. “It sits in the execution path of an agent and enforces policy on what an agent is doing,” says Husain. Rather than examining the model’s chain of thought or text output, Draco evaluates the actions the AI agent takes on a system, like calling a tool or opening a file. It can then stop actions that fall outside those the agent’s pre-defined scope.

Husain says that in addition to monitoring each individual agent’s output and actions, these tools are an effective way to watch for collaboration. “We treat it as the same problem, not a separate one,” says Husain. “Agent-to-agent communication is just another action an agent takes, so it goes through the same control plane as tool calls and data access.”

That’s not to say the challenge of controlling and monitoring AI agents is solved. Reilly Haskins, a member of technical staff at METR, a research nonprofit that evaluates risks posed by AI models, says the methods used to monitor AI models continue to evolve as the models improve and change.

“There’s a consensus that the most monitorable part of a model is its chain of thought,” he says, but adds that chain-of-thought output can at times be difficult to parse. “Is it gibberish to a human, or is it pretty clear what the model is thinking?” A model might reason in an abstract language and perform better because of it, he says. Examples of this behavior can already be found in the Hugging Face incident, as agents often communicated in shorthand phrases such as
“please_HOLD_swarm_I_prepare_safe_exfil_reply_zz_MAILBOX_JAN183411.”

Solutions Exist, but Enforcement Lags

Engineering systems to monitor and control AI agents which attempt to collaborate, coordinate, and perform out-of-scope tasks is possible, though the methods are evolving.

However, there’s currently no legal framework or industry standardization to enforce, or even recommend, methods for controlling agents that collaborate. That’s where AI governance—new laws, regulations, and industry practices—can come into play.

Noam Kolt, an assistant professor at the Hebrew University of Jerusalem who leads its Governance of AI Lab, says companies should place more emphasis on legal compliance when training and deploying models. In a recent paper, Kolt and a coauthor questioned AI models using a method developed by Tom R. Tyler, author of Why People Obey the Law (Princeton University Press, 2006). When asked abstract questions, models generally told the researchers that the law matters. However, these replies didn’t always translate to actions. “There’s this weird disconnect,” Kolt says. One of his lab’s evaluations asks AI agents to unlawfully edit corporate records. “And well, sometimes they do.”

This behavior was reflected in the Hugging Face incident. OpenAI’s agents sometimes considered the possibility that they were engaged in “unauthorized” activities but proceeded anyway, sometimes with encouragement from other agents.

The solution could include a change to the instructions that guide a model’s behavior. Claude’s constitution, for example, sets out the values Anthropic wants the model to follow. But “perhaps surprisingly, law is just not one of them,” Kolt notes. These instructions could rank legal compliance high in a model’s list of priorities and provide examples of how a model should approach situations where a goal can be achieved only through unlawful conduct.

National governments have yet to issue laws or regulations to clarify how agents are held responsible or enforce standards. Kolt says that under U.S. law, responsibility defaults to an entity that controls an agent. However, the legal understanding of agency hasn’t been revisited in 20 years and may need clarification to address modern AI agents. In another recent paper, Kolt and his coauthors argue that even the European Union, often seen as a leader in AI regulation, hasn’t kept pace with AI agents, as the EU’s AI Act is written to address older models that were less capable of autonomous actions.

Casper, who has coauthored several papers with Kolt, agrees that adoption of new laws and standards is the key. He says the engineering strategies to control agentic behavior and collaboration aren’t perfect, but they already work well enough to provide a benefit. “The main bottleneck is the adoption of best practices,” says Casper. “So yes, I ultimately think that governance is the thing that’s needed the most.”

Learn how engineers can shift from manually managing tools and handoffs to defining objectives and supervising AI-driven execution across the silicon development lifecycle.

Key Takeaways

• Explore four critical technologies enabling silicon design autonomy
• Understand how autonomous AI accelerates complex engineering workflows
• See real-world impact across semiconductor design and verification
• Gain practical insights from Ambiq’s deployment of autonomous AI in production

Building useful robots starts with understanding the people who use them. For Charlie Kemp, cofounder and chief technology officer of Hello Robot, that means developing assistive robots that can help people with everyday tasks and support greater independence.

In this Robots Guide profile, Kemp shares his path from studying artificial intelligence at MIT to building Stretch, explains how working with people with disabilities has shaped his approach, and offers advice for aspiring roboticists. Read the full profile on IEEE’s Robots Guide.

Space was always supposed to be the final frontier of human exploration. It’s shaping up to be the final frontier for artificial intelligence too.

Last December, NASA’s Jet Propulsion Laboratory used Anthropic’s Claude models to help plan two Mars drives for the Perseverance rover, with human planners checking and adjusting the route before upload. In May, NASA and IBM put a compressed AI model on the International Space Station and a satellite to identify things like floods and clouds from orbit, the first model of its kind demonstrated in space. And in July, astronauts on the ISS tested a large language model to see if it could help with questions on maintenance procedures.

These experiments point to a larger shift in space engineering. For decades, engineers on Earth determined what a machine in space would do, and the machine would do exactly that. Now, researchers are testing whether nondeterministic systems like generative AI can give spacecraft more flexibility to interpret their surroundings, plan tasks, and one day make decisions for themselves. The technology is still far from trustworthy enough to hand over control of a spacecraft, but engineers are starting to ask whether they can afford not to do so as missions become more complex, distant, and numerous.

Why Spacecraft Need True Autonomy

Spacecraft have been operating autonomously for decades. But autonomy has never been the dominant model, in part because space engineers have prized systems whose behavior they can predict.

“Autonomy often does not have a deterministic outcome, which means that how you got into a certain situation changes the behavior,” says Robert Ambrose, the former chief of NASA’s Software, Robotics, and Simulation Division. “So if you come into the same situation but from different paths, the outcome could be different. And so engineers hate that.”

Ambrose spent much of his career working on autonomous systems at NASA, including autonomy for the Orion spacecraft, NASA’s deep space and lunar orbiter spacecraft, and Robonaut 2, a humanoid robot designed to work alongside astronauts that went to space in 2011. With Orion, he saw how quickly the testing problem could multiply. Engineers had to consider not just what the spacecraft might do, but all the different ways it could have arrived at a decision.

But Ambrose says engineers found ways to manage that complexity, including automating the testing itself. “We fought the challenges of autonomy using autonomy,” he says. “That actually works.”

The need for autonomy becomes even more obvious the farther a mission travels from Earth. Ambrose points to a possible mission to Europa, Jupiter’s ice-crusted moon, where a spacecraft could dive through a water plume erupting from beneath the surface. The plume could appear too quickly for engineers on Earth to direct the spacecraft into it.

“It’s up to the spacecraft to make a decision, and we’ll be watching what happened an hour ago,” he says. A mission like that, he added, is “totally impossible” without giving the machine real autonomy.

Icarus Robotics engineers gather valuable training data for a free-flying robotic system—destined for the International Space Station—during a zero-g flight.Icarus Robotics

Adapting Robotics to Microgravity Physics

As AI and robotics advance on Earth, a commercial space boom is creating new opportunities to put those technologies to work in orbit. But what works on Earth does not necessarily work in space.

Icarus Robotics is developing what it calls a robotic labor force for space. This includes Joy, a free-flying robotic system. Joy recently finished zero-gravity testing in Canada ahead of a planned deployment to the ISS, where one of its first tasks would be moving cargo bags between modules. The company plans to start with teleoperation, using that data to eventually train the robots to work on their own.

“What the rollout will probably look like is something much closer to beginning with partial autonomy, so you still have human supervision in the loop at all times,” says Jamie Palmer, Icarus’s cofounder and CTO.

A robot trained on Earth learns from the physics of the environment around it, Palmer says, and in orbit, the physics is entirely different.

“If you take the newest Gemini robotics model, or you take the newest physical intelligence model, and you put it in zero-g there, it’s just going to fail immediately,” he says. On Earth, for example, a robot learns that when it pushes something off a table, the object falls. In orbit, it keeps moving.

That leaves Icarus with a problem the terrestrial robotics industry also faces, but in a more extreme form: There is very little real-world data from the environment where its robots will operate. The company is combining demonstrations from its robots in microgravity with simulations and tests on Earth to build its own dataset. “I wish there was” an available dataset Icarus could download and use, says Ethan Barajas, the company’s cofounder and CEO. “But there’s not today, not in a meaningful way.”

Managing Autonomous Spacecraft Risk

The challenge is not simply teaching spacecraft to act on their own. It is figuring out how to manage the risks of giving them more freedom.

“It has been mind-boggling to me how little autonomy we have in space applications,” says Ufuk Topcu, an engineering professor at the University of Texas at Austin and the director of the Center for Autonomy. “Because it’s exactly the place where human involvement is extremely hard, the stakes are high, and you need to act fast.”

Topcu says that the goal cannot be to guarantee that autonomous systems will never do anything wrong. They’re most useful in situations humans cannot anticipate, he says, so instead researchers need to start with restricted applications, learn how the systems behave, and gradually expand where and how they are used. The real question, in his view, is how well the risk of deployment is managed, not whether they can be fully eliminated.

That may become increasingly important as the space industry changes. For most of the space age, a small number of government agencies designed missions that could take decades to develop and operate. Commercial companies are now putting more spacecraft into orbit, and new missions can be developed and launched much faster.

“Space used to have very slow innovation cycles,” Topcu says. “They would think of a mission concept and spend 10 or 15 years on it. It’s not like that anymore. Everything is evolving faster now.”

Have you ever read a paper in Science or Nature and thought, “Man, that research was so cool. I wish I could try that method on my own data,” only to spend a week wrestling with someone else’s undocumented repo, broken dependencies, and half-finished readme.txt?

Well, now you can, more or less.

Say hello to Paper2Agent, a new open-source framework that transforms academic reports into interactive AI agents you can talk to. Give it a paper, along with the accompanying codebase, data, or other supplementary material, and the system automatically extracts the core workflows, then spins up a tested, runnable toolkit that you can use on your own datasets.

The concept may sound a little like Google’s NotebookLM (now called Gemini Notebook), which lets you upload documents and chat with an AI about what’s in them. But Paper2Agent aims to go a step further: Rather than simply answering questions about a paper, its agents can actually run the methods described in it—and potentially combine those methods with tools from other papers.

The goal, explains Stanford computer scientist James Zou, is to change what a scientific paper fundamentally is. “Knowledge should not be static records,” Zou says. “It really should be dynamic and interactive—and this has many benefits, including making knowledge more reproducible but also enabling all sorts of new kinds of discovery.”

Zou and his colleagues described the tool 16 September in Nature. They tested Paper2Agent across diverse disciplines including statistics, econometrics, and astrophysics. However, the researchers focused their proof-of-concept demonstrations on computational biology, where turning published methods into usable tools can be particularly cumbersome.

From Paper to Prompt

The team started with AlphaGenome, a deep-learning model that predicts how mutations in DNA affect gene regulation. (A companion resource unveiled earlier this month, the AlphaGenome Atlas, cataloged the model’s predictions for all 9 billion possible single-letter changes in the human genome.)

The researchers fed Paper2Agent the corresponding documentation and code. About 45 minutes later, with no human intervention, the AI had produced 22 tools covering different aspects of AlphaGenome’s functionality, all on a personal laptop and for less than US $15 in computing costs. One tool, for example, could predict how a DNA change might affect gene activity, while others could compare those effects across tissues or analyze multiple variants at once.

“The idea of making papers more dynamic and executable through an agentic interface is quite compelling.” —Dongping Chen, University of Maryland

All 22 tools passed automated validation, thanks to a testing agent working behind the scenes to run the various AlphaGenome sub-tools against reference results. If a test failed, the agent would diagnose the problem and try to fix the tool, with up to six attempts per function. If that didn’t work, it could drop the tool altogether.

The validated tools were then packaged into a Model Context Protocol server and connected to Claude Code (though any compatible chat-based AI assistant could do). The result was a user-facing AlphaGenome agent that could take questions in plain English, run the appropriate analyses, and spit back results and visualizations.

The team then put the agent through its paces, hitting it with a battery of questions, ranging from simple requests to open-ended research problems. According to the researchers’ analysis, it outperformed both standard Claude given the AlphaGenome codebase and a specialist AI co-scientist tool called Biomni.

Multi-Agent Research Paper Collaboration

Going one step further, the researchers turned a couple of more papers into interactive agents and linked them with the AlphaGenome agent. (One of the additional papers was on how inherited DNA variants linked to autoimmune disease disrupt cell function, while the other was a more systematic exploration of how silencing every expressed gene alters immune cells.)

Prompted to investigate the genetic basis of psoriasis, an itchy skin disease, the three agents collectively zeroed in on a little-understood gene called GPR137 as a likely causal factor. What’s more, the AI proposed 10 ways to validate this inference. A human researcher selected one, and the resulting analysis found that silencing GPR137 produced changes in gene activity strikingly similar to those caused by the psoriasis-linked variant in immune cells.

“These agents, because they’re able to directly collaborate and communicate, can facilitate all these kinds of collaborations,” says Zou.

Other researchers see plenty of potential as well. “The idea of making papers more dynamic and executable through an agentic interface is quite compelling,” says Dongping Chen, a computer scientist at the University of Maryland in College Park.

“Agentification itself is a useful certificate that says, ‘This work is relatively complete and well documented.’” —James Zou, Stanford University

Olivier Elemento, a computational biologist who directs the Englander Institute for Precision Medicine at Weill Cornell Medicine in New York City, sees the approach as having broader implications for how researchers share their work.

“It’s a real advance in terms of how we think about the publication process,” he says, “with AI at the center and in a way that makes publications more interactive.” (Elemento peer-reviewed the study for Nature.)

The potential applications extend beyond the research side of academia, too. Artur Skowroński, head of application development at the Polish software company VirtusLab, noted in a blog post that Paper2Agent could help bring scientific papers to life in classrooms. For example, students could use the agent to play with methods described in the literature instead of merely reading about them.

The Future of Agentified Research Papers

With Paper2Agent now up and running, Zou and his colleagues have begun turning more of their own research papers into agents. Just one day after publishing their Nature paper on Paper2Agent, they unveiled the Virtual Biotech, a multi-agent platform modeled on a drug development company.

They described the system in Science and, at the same time, posted a Paper2Agent-generated incarnation of the paper.

However, not every study they threw at the tool could be converted into an agent. Of the 100 computational biology papers they tried, 26 failed to make the leap to agent form, often because of incomplete code, missing documentation, or other software packages that couldn’t be made to work.

But Zou sees that as a feature, not necessarily a bug. When the system gets stuck, it can expose missing information, errors in the code, or discrepancies between the paper and its implementation—problems that might otherwise go unnoticed. As Zou puts it: “Agentification itself is a useful certificate that says, ‘This work is relatively complete and well documented.’”

Human scientists, Zou says, will still have the final say. But he envisions agents becoming part of what it means to publish a paper. Today, papers come with data and code availability statements. Tomorrow, he suggests, they could come with an “agent availability” statement: a virtual corresponding author available around the clock, in any language, to answer the questions that real authors never have time to field.

Naturally, Zou and his colleagues decided to try the idea on their own study. They fed the Paper2Agent manuscript into Paper2Agent, creating an agent that now lives at paper2agent.ai. In other words, a paper about turning papers into agents has turned itself into an agent. The recursion, it seems, has already begun.

This article is brought to you by CoolIT, an Ecolab Company.

Beyond 250 kW a server rack can no longer be cooled by a hybrid approach of liquid and air. At this density a 70/30 liquid-air split leaves 75 kW of air load. The air cooling system needed to move it brings cost and complexity few operators will accept. The answer is near-total heat capture. Liquid takes effectively all the heat, air falls below 1 percent of the load, allowing the server to run fanless.

CoolIT builds these loops today from modular coldplate blocks proven across six generations of fanless designs. Processor thermal design power (TDP) keeps climbing generation over generation. This rising heat load is now cascading into the memory, networking, storage, and power components that once ran comfortably on air.

The heat escaped the chip

For years the story stayed simple. Cool the processor and let air handle the rest. That balance has shifted. As TDP climbs, heat spreads outward from the processor and cascades into the components around it. Memory, networking, storage, and power now run hot enough to demand liquid of their own. Engineers designing the next generation of AI servers face a board where heat capture rises with every launch.

Beyond 250 kW per rack, air cooling becomes the bottleneck. Near-total liquid heat capture enables fanless AI server designs built for the next generation of computing.

New parts, new rules

Unlike processors, which are cooled as flat rectangular packages, these peripherals come in a wide range of shapes, sizes, and mounting requirements, each with its own thermal limits. Some run cooler than the processor case temperature, others run hotter, which leaves them sensitive to a design tuned only for CPUs and GPUs. Operators need purpose-built solutions here, matched to the part rather than stretched across the board.

CoolIT engineers meet this with a deep toolkit. Conductive plates, vapor chambers, heat pipes, and thermal transfer plates move heat from components closer to the liquid path. Riding coldplates enable pluggable components. Each solution stays true to the component it serves.

CoolIT Customer Showcase: How GWDG Cools HPC & AI Systems with CoolIT’s Direct Liquid Cooling CoolIT

One loop, one server

Cooling the parts is one challenge. Uniting them is the real work. Full heat capture means folding every one of these solutions into a single server loop that distributes coolant effectively and remains easy to install. Connection reliability, coolant routing, and the time it takes to assemble the loop at rack integration determine whether a design thrives in production or stalls on the bench. CoolIT builds these loops from proven modular blocks, so operators gain performance and deployment speed within the same solution.

Density forces the decision

Rack power continues to climb toward 1 MW, and the case for liquid grows stronger at every step. A 70/30 split of liquid to air holds comfortably at lower density. Past roughly 250 kW it stops working. The 30 percent left to air becomes a 75 kW load inside a single rack, and moving that much heat demands a parallel air system whose cost and footprint few operators will accept. Adding density only widens the gap.

As rack power continues to climb toward 1 MW, CoolIT’s modeling places full heat capture as the standard server design for flagship rack-scale products through 2028.

The simpler, more efficient answer is to capture the heat in liquid and drop air to less than 1 percent of the total load. True 100 percent remains almost impossible to reach in the strictest sense, so the honest and achievable target is near-total capture. That distinction matters to engineers who value precision, and the direction stays clear either way. Full heat capture moves from a premium option to a mainstream requirement as density rises, and CoolIT’s modeling places it as the standard server design for flagship rack-scale products through 2028.

CoolIT delivers it

CoolIT scales heat capture all the way to 100 percent using modular coldplate building blocks proven across six generations of fanless server designs. Engineering teams are already working on designs for the maximum density racks coming next. As the cascade spreads and racks grow denser, near-total heat capture becomes the design that keeps AI running.

Talk to CoolIT about building a server loop engineered for total heat capture.

This White Paper gives data engineers and architects a practical overview of how parallel partitioned reads, write-path optimization, and cloud-native bulk loading reduce large-table replication times, and why replication speed has become a business concern as data volumes grow.

Download this free whitepaper now!

This article is brought to you by VicOne.

Robot safety has traditionally asked: Can a machine remain safe when something goes wrong? Physical AI raises a harder question: Can a machine remain safe when an attacker changes what it sees, decides, or does even when nothing appears to have failed?

As AI and robotics continue to advance at an unprecedented pace, modern robots perceive through multimodal sensors, interpret context using AI models, and translate those interpretations into physical action. As they move into dynamic environments, their safety increasingly depends on the integrity of the data guiding their decisions.

That dependence creates risks that conventional safety assessments may not fully capture. Recent research has demonstrated that manipulating what a robot sees, hears, or interprets can influence its behavior without requiring direct control.

Such manipulation can occur anywhere across its complex sensing and decision-making system — a layered attack surface encompassing training pipelines, system infrastructure, and runtime perception.

Layer One: Corrupting intelligence at its source

In 2017, BadNets demonstrated that a model could behave normally under most conditions, yet fail in the presence of a specific hidden trigger. In one example, a subtle pattern caused a stop sign to be misclassified as a speed limit sign without affecting the model’s behavior on other inputs.

What began as a classification vulnerability has since evolved into action manipulation.

At NeurIPS 2025, researchers introduced BadVLA a backdoor attack targeting Vision-Language-Action (VLA) models that allow robots to see, interpret instructions, and produce coordinated physical movement. Rather than altering a single label, the attack caused conditional deviations in the robot’s action trajectory when a trigger was present. Without the trigger, the model largely preserved normal task performance, while the backdoor remained effective under task transfers and model fine-tuning.

A related study in 2025, GoBA, showed that ordinary objects such as a coffee mug could serve as a reliable trigger. The researchers reported a 97 percent attack success rate without degrading performance on clean inputs.

A critical safety question today is whether Physical AI models remain within their task and safety boundaries under adversarial conditions.

These studies expose a blind spot in model validation: A model may pass testing yet produce corrupted behavior when a hidden trigger appears in operation.

So a critical safety question today is whether Physical AI models remain within their task and safety boundaries under adversarial conditions. Simulation tools such as NVIDIA Isaac Sim, when paired with VicOne Radeis, can test the effects of manipulated inputs before deployment.

VicOne LAB R7 demonstrates Radeis, a Physical AI safety validator for NVIDIA Isaac Sim that tests how adversarial visual inputs affect robot behavior before deployment.VicOne

Layer Two: System vulnerabilities as gateways to AI control

Even a securely trained model can be subverted if the surrounding system stack is vulnerable.

In September 2025, researchers disclosed UniPwn, a Bluetooth exploit chain affecting quadruped and humanoid robots from a major manufacturer. Hardcoded cryptographic keys allowed traffic decryption, authentication checks were bypassed, and command injection enabled root-level execution. The exploit is also described as “wormable.” A compromised robot could scan nearby units and potentially affect an entire fleet.

VicOne Lab R7’s demo shows how chaining three wireless exploits can trigger uncontrolled robot behavior within 60 seconds, resulting in operational disruption.VicOne

Middleware creates another exposure point. Vulnerabilities in ROS 2 and DDS-based systems can enable arbitrary code execution or abuse unauthenticated topics to deliver malicious commands. With sufficient access, an attacker could override motor commands or replace AI model weights without directly attacking the model architecture.

In this case, the components may still function as designed. What has changed is the trustworthiness of the commands flowing through the system. Vulnerability management can help teams identify known risks before deployment, while continuous monitoring can surface emerging threats.

Layer Three: Manipulating perception and reasoning at runtime

At runtime, manipulating inputs that shape perception or reasoning may require neither firmware modification nor a network breach.

In 2024, RoboPAIR demonstrated how carefully structured prompts could redirect LLM-controlled robots into unsafe trajectories. BadRobot exposed a deeper architectural weakness: in several cases, a robot verbally refused a dangerous command while its motion controller executed the action anyway.

Vision-based manipulation is equally powerful. VLAttack showed that an adversarial patch within the camera’s view could reduce a VLA model’s task success rate to zero. FreezeVLA showed that a single adversarial image could freeze a robot’s decision-making loop, making it unresponsive to subsequent instructions.

Runtime assurance must therefore look beyond whether individual components remain available and assess whether cyber events are beginning to affect physical behavior.

In each case, the camera may still work, the model may still run, and the controller may still respond. Yet the resulting behavior can be unsafe because the robot is acting on manipulated perception or reasoning.

Runtime assurance must therefore look beyond whether individual components remain available and assess whether cyber events are beginning to affect physical behavior. Security event correlation, behavioral-impact assessment, and policy-bounded response supported by edge AI, can help contain the affected path without unnecessarily stopping the entire robot fleet.

From point-in-time safety to lifecycle assurance

The risks across these three layers reveal the missing layer in robot safety assurance: cybersecurity. Functional safety addresses failures and unexpected operating conditions; cybersecurity extends that assurance to deliberate manipulation, including attacks that may leave the underlying system apparently functional.

This requires assurance across the robot’s lifecycle. During design, teams need to understand which cyber risks could invalidate assumptions behind intended behavior. Before deployment, they should test whether realistic attacks can cause a robot to deviate from its task or safety boundaries. In operation, monitoring should identify whether cyber events are beginning to affect behavior, contain the affected path, and preserve safe operation where possible.

VicOne’s lifecycle approach combines AI model and vulnerability scanning, simulation-based validation, and continuous monitoring to help secure robots from development through operation.VicOne

While cybersecurity does not replace functional safety, it helps ensure that Physical AI remains within acceptable boundaries even when what it sees, decides, or does is under attack.

For a deeper look at the cybersecurity risks and defense strategies shaping autonomous robotics, download our whitepaper “Securing the Rise of AI Robots: Cyber Risks, Real-World Threats, and Defense Strategies.”

Learn how single-phase direct liquid cooling manages the rising heat of AI and high-performance computing, and how it compares with two-phase and immersion approaches.

Download this free whitepaper now!

This interactive webinar will introduce the different types of AI, address the concerns with AI, share how we IBM are approaching Responsible AI, and offer guidance to students about what they can do - as individuals, and members of their IEEE chapters. Participants will also have the opportunity to to apply the Responsible AI approach to a particular use case - IBM Bob, a software development life cycle agent, and Q&A. This will be an interactive session, so have phones ready to engage!

Register now for this free webinar!

On 25 August, OpenAI fully unveiled Jalapeño, the company’s debut AI accelerator chip. Jalapeño delivers up to 13.4 petaflops of 4-bit compute and accesses 232 gigabytes of the most advanced memory available, linking to it at a blazing 15.4 terabytes per second. Benchmarks cited by OpenAI show that Jalapeño can reduce end-to-end latency (the time between prompt to last token) by up to 3.6 times when compared to Nvidia’s GB300—a chip the company currently relies on—and do so while consuming less power.

Whether these figures translate into real-world gains once Jalapeño enters widespread service in OpenAI’s inference fleet remains to be seen, but performance is only half the story. The other half is how the chip was designed—a process which, as you might expect, was accelerated by OpenAI’s large language models (LLMs). Jalapeño moved from first architecture concept to first silicon in under 20 months. Only nine months separated the first RTL—the register-transfer level code defining the chip’s logic—from tape-out, when the finished design goes to manufacturing.

That’s a rapid timeline, yet experts believe it could soon look slow as LLMs improve and become more deeply integrated into chip design tools. OpenAI, unsurprisingly, is bullish about the opportunities. “The models are giving superpowers to our engineers,” says Richard Ho, vice president of hardware at OpenAI. “Our engineers are still driving the work. They’re still the final arbiter of what’s going on. But they can do things a lot faster. They can explore a lot more paths.”

OpenAI achieved fast results with a small design team

Ho says the group that designed Jalapeño averaged fewer than 100 people over the course of the project and continues to stand at roughly 100 today as the team pursues second and third-generation designs. That number includes a broad swath of roles across the hardware team, from system design to software and supply chain, but not those at Broadcom, which partnered with OpenAI on the project.

The division of labor between OpenAI and Broadcom was generally split between design and implementation. OpenAI’s team was responsible for end-to-end system design including the inference accelerator, the memory hierarchy, and networking. Broadcom handled “physical design from the gates onward,” Ho says.

The partnership with Broadcom dampened some opinions on OpenAI’s speed. David Chin, co-founder at agentic chip design startup Verkor.io, says “the schedule they gave us is quite credible,” but believes that Broadcom’s help was essential to Jalapeño’s rapid timeline. “If you have somebody else start from scratch, it won’t be possible,” he says. Ravi Krishna, also a co-founder at Verkor, called OpenAI’s speed “a relatively impressive result,” but added that he expects that improvements in the capabilities of LLMs could result in even quicker timelines if the project started today.

Andrew Kahng, distinguished professor at the University of California, San Diego, also found OpenAI’s speed notable, saying it’s “likely best in class today.” Kahng recalls a 2016 IEEE Design Automation Futures workshop, which he co-organized. The workshop included Richard Ho, at the time an engineer at Google, as a keynote speaker. Ho had strong opinions on design automation and framed the time required to complete a chip’s design as a function of the number of iterations a team could complete in a day.

How OpenAI’s LLMs accelerated Jalapeño’s design

“Automation itself has existed in chip design for many decades. It’s not a new problem,” says Ankur Srivastava, director of semiconductor initiative and innovation at the University of Maryland, in College Park. Where LLMs differ from prior automation tools, however, is their ability to understand language and code. He says this makes them particularly suited for chip design tasks that “are still in the linguistic domain of the problem.”

The team at OpenAI designed a workflow that takes advantage of this strength. OpenAI’s front-end workflow was built around Accelerated Hardware Synthesis (XLS), an open-source high-level synthesis chain of tools originally developed at Google. High-level synthesis is a form of chip design automation that allows engineers to design a chip in a more familiar programming environment. In the case of XLS, chip designers can write in languages such as DSLX (a domain-specific language inspired by Rust) and C++. XLS then converts these to Verilog, a hardware description language used to describe electronic systems.

“We were thinking about how to leverage AI to make the project faster, and the AI was much better at software-looking things,” says Chris Leary, member of technical staff at OpenAI. “XLS in some ways looks like software, so it got that benefit.” It helped, too, that Leary was extremely familiar with how XLS should function, as he started it during his time at Google.

Kahng agrees that the decision to use AI to accelerate high-level synthesis, such as XLS, makes sense, as it’s “more natural for the LLM to work with” and provides the opportunity for fast iteration. “I see this as a generally useful workflow, and it’s one that ‘has legs’ going into the future,” he says.

The same logic led the Jalapeño team to focus on software optimization. When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.

Jalapeño is designed for deployment in pods that include 2,048 chips.OpenAI

While the broad strokes of the Jalapeño teams’ AI-assisted workflow were guessed by Ho and Leary up front, improvements in OpenAI’s models did offer a few surprises.

Leary says that the project began with assistance from models like OpenAI’s o3, which was released to the public in April of 2025 (but available to the Jalapeño team earlier). By the time the project had wrapped up, however, the team had access to models that were precursors to GPT-6 Astra, which wasn’t publicly released until 3 September 2026. The newer model can work directly in Verilog without needing XLS’s translation from ordinary programming languages, and it’s close to being able to operate proprietary design tools on its own, Leary says.

Ho also confirmed that the team had access to internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail the models used. However, he added that the Jalapeño team partnered with OpenAI’s research team. While not all specific models used to design Jalapeño are publicly available, Ho says the goal is to bring lessons learned from the project into the company’s commercial LLMs. “It’s safe to say that Astra and following models will be very good at chip design,” he says.

AI was less useful for backend optimization, but that could change

As mentioned, the bulk of OpenAI’s work on Jalapeño focused on the “front end” of chip design, which spans the tasks that take a chip from initial concept, through writing RTL code to define the design, and through verification that the design will work when physically implemented. Much of the “backend” design—which includes tasks like routing interconnects, completing and verifying the clock and power specifications, and sending the required design information to the foundry—was handed off to Broadcom, which carried the chip through production.

That’s not to say OpenAI’s workflow ignored the backend, though. The Jalapeño team includes physical design engineers who work with their counterparts at Broadcom to provide guidance on the chip’s floor plan and routing, among other things.

At IEEE Hot Chips 2026, Ho and Leary put numbers on the gains from AI-guided physical design optimization, including an area reduction of 10 percent for the matrix multiplication units as measured against an optimized human baseline. In other words, OpenAI claims AI-guided optimization helped design more circuits into the same area of silicon than would have been possible before.

Broadcom used its own internal workflow. The company’s team did not have access to the internal models OpenAI used to help design Jalapeño, but it did have access to OpenAI’s public, commercial models.

Verkor’s Ravi Krishna says that OpenAI’s approach to backend design already feels a bit conservative. He believes that to be an artifact of when the project, which began in October of 2024, took place. “The models from the last four to five months have improved. From April [2026] onwards…is when they really started to be able to handle those tasks better,” he says. Verkor co-founder Suresh Krishna agreed, saying “there’s no reason you couldn’t have an agentic loop that largely accelerates the backend of the process as well.”

Ho and Leary also hinted that the workflow used to design Jalapeño may look old-fashioned compared to the team’s next efforts.

“As you can imagine with [Jalapeño], we were trying to go as fast as we could. So there’s a trade-off between ‘do we want to take time to do some innovation, or do we want to do things that we know work historically?’” Leary says. “With the second generation, we have a kind of reset opportunity to ask about all the things we want to get set up for.”

Ho says the second-generation chip’s workflow has “a lot of places that we are introducing [AI].” He mentions opportunities to do more with AI in verification and physical design. Leary adds that the team now has tools for automatic waveform manipulation and viewing. This automates analysis to identify chip clock signals associated with failures and could improve debugging the hardware while it’s still being designed.

Despite these expected improvements, Ho and Leary were clear that they don’t believe chip design can be fully automated. “We’re not saying that anyone can come and just build state-of-the-art, frontier AI/ML accelerator chips using just [OpenAI’s coding platform] Codex,” Ho explains. “We are saying some very specific things about how to be better at Codex and how we are focusing on a small team and fast timelines to reach quality results.”

AI-powered cameras dot streets across the world, equipped with the power to identify faces or vehicle license plates. But a public backlash is gaining momentum.

Privacy concerns abound, encompassing the lack of consent for capturing data, how that data is stored and used, and the risk of misuse. Those concerns are motivating people to fight back. The DeFlock project, for instance, maps automated license plate readers (ALPRs) to raise awareness. Some people resort to extreme measures, such as vandalizing or damaging ALPRs. Others are stitching together more creative responses, crafting “adversarial fashion” to evade surveillance cameras, like a Kickstarter project called noRecognition, presented at last month’s DEF CON hacker convention.

Scrambling surveillance

In 2025, cybersecurity expert Bill Swearingen began experimenting with a simple Python-based fuzzer, a tool that provides invalid inputs to reveal software bugs, security vulnerabilities, or unexpected behavior. The fuzzer targeted one of the most popular object detection frameworks, called YOLO. He then developed what he’d learned into a reinforcement learning algorithm that generates various adversarial patterns, which he presented at DEF CON.

Each pattern is a colorful geometric abstraction he has tested against 11 object detection models—four that search faces, two that recognize faces, and five that detect people—most of which are publicly available. Successful patterns thwart the object-detection systems, lowering their confidence scores, sometimes even to the point of no detection.

“Privacy is a human right, and the popularity of this just goes to show that people are interested in preserving their privacy,” Swearingen says.

Cap_able and Urban Privacy are already selling physical garments. Cap_able’s patented manufacturing method weaves its bright and bold motifs into jacquard knitted fabrics. The ethically produced and sustainably made dresses, pants, and tops interfere with certain computer vision systems, particularly those backed by fast convolutional neural networks, which may lead them to classify wearers as animals or objects.

“If we’re able to camouflage a person as something else, then we’re obtaining our goal,” says Cap_able founder Rachele Didero, who’s also an assistant professor at the Free University of Bozen-Bolzano in Italy. “We use this very visible and tangible item to talk about something that most of the time is intangible.”

Meanwhile, Urban Privacy aims to baffle some facial recognition systems based on OpenCV algorithms with its latest Faception Reloaded collection. Black-and-white prints abstracted from a human face show up as additional faces on detectors, slowing them down. Asymmetrical cuts and wide silhouettes intend to conceal, making it harder to discern your body’s shape and gait. “The idea is to create false data,” says co-founder Daniel Preuß.

Simulated patterns of the kind intended to disrupt machine vision person detectors.noRecognition

“Not an invisibility cloak”

Anti-surveillance fashion can trace its roots to the art pieces, DIY projects, and thought experiments that emerged in response to the onset of AI surveillance systems more than a decade ago. For instance, technologist Adam Harvey developed multiple designs in the 2010s—from hairstyles and makeup that foil face detectors to heat-reflecting attire that avert drone-enabled thermal surveillance. In 2019, artist and activist Kate Bertash created her aptly named Adversarial Fashion clothing line decked with fake license plate numbers to inject junk data into ALPR databases.

The trend is now growing into a more solidified small industry. “Clothing is something you can actually buy and put on, unlike policy,” says Niloofar Mireshghallah, incoming professor of engineering and public policy at Carnegie Mellon University. “It’s a way of saying, ‘I didn’t consent to this.’”

But real-world conditions might reduce the effectiveness of countersurveillance clothing, such as camera angles, lighting, and how fabric folds as you move. “One good frame is all a system needs,” Mireshghallah says.

Motion and gait recognition are also influential factors. “Even if the camera thinks you’re a bear for a few frames, there’s a bear walking like you,” Mireshghallah says.

The adversarial patterns must also be tuned to specific object recognition models, so they cannot resist a different model. And once surveillance system operators train a future generation of models on a given adversarial pattern and the person wearing it, which they could do manually, clothing will no longer be a sufficient defense.

“It remains a fragile shield against a threat that is constantly improving from multiple angles,” says Dippu Kumar Singh, senior director of emerging data and analytics at Fujitsu North America who specializes in vision AI and AI ethics.

Makers are aware of their creations’ limitations. “It’s not an invisibility cloak,” Preuß says. “Surveillance aims to capture your identity, and fashion is about expressing your identity. We’re making clothing that people can wear to make a statement about the importance of privacy in a digital world.”

Active defense

Even with these hurdles, Cap_able’s Didero is determined to keep innovating. Urban Privacy will continue to release other parts of its collection, including a “shadow cap” that has an acrylic face shield layered with cutouts to blur facial contours. Swearingen plans to explore a few anomalies he has encountered, such as a pattern that shifted the bounding box and another pattern that changed a camera setting.

Adversarial fashion holds promise despite its pitfalls. “At its core, this fashion is about taking back control of your face and body,” Singh says. “People are starting to realize that privacy isn’t just a right they can passively expect to be handed to them—it is something they have to actively defend.”

Mireshghallah offers a more cautious approach, viewing countersurveillance fashion as a speed bump rather than an ultimate solution. The real risk, she notes, is aggregation: Models take a group of weak signals, such as a partial face, a building in the background, a time stamp, a social media post someone tagged you in, and stitch them together to make a confident guess about who you are and where you were.

“None of those pieces give you away on their own, but together they do,” Mireshghallah says. “My advice is don’t just think about hiding your face from a lens. Think about what else you’re leaking that can be combined with it. That side information is often what actually identifies you—and no pattern on a shirt fixes that.”

Maybe you heard about the AI-controlled vending machine that stocked underwear and live fish. Or the AI manager of a San Francisco store that fired a human employee. Or the AI radio DJ that said its catchphrase, “Stay in the manifest,” 229 times per day.

These incidents all emerged from experiments run by Andon Labs, an AI safety company based in San Francisco that puts AI agents in charge of real-world operations and watches what happens. These operations double as testbeds for Andon’s commercial work developing evaluations and conducting research with the leading frontier AI labs.

Their spectacular and absurd failures have won the company plenty of attention. But many people don’t realize that the experiments are intended to answer a serious question: How much real-world responsibility can today’s AI agents handle? “We want to measure autonomy,” says Andon cofounder Lukas Petersson. “We want to provide society with accurate data points of what happens when you do this.”

From Simulations to AI-Run Businesses

Andon Labs started off in the virtual world in 2025 with Vending-Bench, a test in which AI agents operated a simulated vending-machine business. The agents, which were based on large language models from Anthropic, Google, and OpenAI, managed tasks such as ordering inventory and setting prices. The researchers found that the performance of many agents degraded over time, with agents forgetting orders, misunderstanding delivery schedules, or spiraling into what they called “meltdown loops.” Some agents also justified deceptive or illegal behavior by reasoning that it was permissible inside a simulation.

The Andon team reasoned that moving into the physical world would expose the agents to consequences and situations that the engineers would never think to program. “It’s impossible for a human to enumerate all the different things that can happen in the real world and code them into the simulation,” Petersson says. And there was one other reason: “We thought it would be quite funny to do it in the real world.” Andon backed its jokes with real money, including a three-year lease for Andon Market, a physical store on a busy San Francisco street that’s managed by an AI agent and sells clothing, home goods, and art.

That said, the store isn’t entirely autonomous. “It’s almost like I’m running the store, and then there’s an AI that has a checklist,” says employee Felix Carson. Luna, the AI manager, keeps track of deliveries and communicates with vendors, while Carson and his coworkers handle the physical work. When Luna tells Carson to check something in the back, he sometimes ignores it because he doesn’t want to leave the sales floor unattended. Luna also repeatedly spots a built-in electrical cover in photos of the floor, mistakes it for a loose coaster, and asks Carson to remove it. Even so, Carson calls Luna a “decent manager,” praising its flexibility when employees need time off.

What Real-World AI Experiments Can—and Can’t—Reveal

Andon’s move into the real world comes with a basic trade-off. Moving into the physical world makes the experiments more realistic, but the unpredictable conditions and the actions of unpredictable humans make the tests impossible to reproduce. The setup also makes it hard to determine whether a success or failure belongs to the model, the software built around it, or the people helping it.

Petersson readily acknowledges the limitations. With only one store operating under uncontrolled conditions, he says, the experiments are “weak science,” at best. For now, he sees them primarily as ways to uncover unexpected behaviors that Andon can later try to reproduce systematically in simulation.

A display inside Andon Café shows visitors Mona’s bank balance and recent activity, while the adjacent handset and tablet provide a way to speak with the AI manager.Andon Labs

Sayash Kapoor, a Princeton University AI researcher who studies “open-world evaluations” like Andon’s experiments, thinks these trials have real value despite their limited scientific rigor. “I think they’ve done a good job of popularizing this style of evaluation,” he says, “even just showing that you can gain a lot of insight from a small sample of open-ended experiments.” He notes that academic research and publishing can’t keep up with AI’s breakneck pace of development and says that Andon’s tests are useful in part because the company is working “literally at the frontier of model capabilities.”

For Kapoor, real-world experiments are best suited for discovering possible failure modes. A real store can reveal not only whether an agent can manage inventory or communicate with vendors, but also whether employees will accept instructions from an AI manager and whether customers want to shop at an AI-run business (the early results on that last point are decidedly negative). Such social and organizational barriers may help explain why impressive AI capabilities haven’t yet translated into widespread adoption across the economy. “What I take to be most valuable from Andon’s work,” Kapoor says, “is a more comprehensive understanding of where these agents still hit their limits.”

Testing AI Agents for Reliability

Andon Café, in Stockholm, provides one example of AIs demonstrating a wide variety of failure modes. At first, when the AI manager was based on a Google Gemini model, it spent freely on fresh ingredients, many of which spoiled before they could be used. When Andon switched the AI manager to a GPT model from OpenAI, it “freaked out” about the spending, Petersson says. The agent overcorrected and stopped buying anything that could expire. It reduced the menu to cheese toast, using frozen bread and long-lasting cheese to minimize spoilage. But Petersson notes that the café is located in a fashionable part of Stockholm where “any human would know that cheese toast would not fly.”

The episode illustrates why successfully completing individual tasks isn’t the same as reliably managing a business. Kapoor argues that AI evaluations have focused too heavily on whether an agent can complete a task at all. Drawing on aviation and nuclear engineering, he and his colleagues have proposed also measuring qualities like reliability and robustness, which determine whether a capable system can be trusted to operate without constant supervision. “Reliability has been improving so much more slowly than capability,” Kapoor says. Andon’s café makes that gap tangible: An agent may be perfectly capable of placing an order for bread, yet remain an unreliable manager.

To determine how often such failures occur, and whether they can be prevented, Andon plans to feed data from its physical businesses back into simulations. These “digital twins” would recreate complications first encountered in the real world, allowing researchers to replay situations under controlled conditions and test whether changes make the agents more dependable. In principle, the physical businesses would discover failure modes, and the digital twins would measure them. Kapoor cautions, however, that existing digital-twin studies suggest “we are very far” from being able to substitute simulations for real-world experiments about how people and organizations behave.

Despite those limitations, combining real-world testbeds with repeatable simulations is central to Andon’s business proposition: providing AI developers with evaluations grounded in situations that arose outside the lab. The company says it works with Anthropic, Google DeepMind, OpenAI, and SpaceXAI on research and evaluations. Andon’s physical businesses themselves remain decidedly less successful. When Andon Market’s Carson spoke with IEEE Spectrum, he was about an hour into his shift. Two customers had come in. Neither bought anything, although both left with free pins and stickers.

On 11 August, Anthropic announced that all future Claude models will generate text that contains a watermark that identifies its results as AI generated. The company is not alone. Google has its own text watermark (which Anthropic’s is based on) that it uses on the output of its Gemini models. OpenAI has yet to introduce a text watermark but it plans to do so.

The rapid spread of watermarking is in part a response to the European Union’s AI Act, which mandates watermarks for AI models released after 2 August, 2026, along with other planned and proposed regulations aimed at curbing the spread of deceptive or manipulative AI-generated content. But the new rules may come at a cost for AI users who simply want the best possible results.

AI watermarks can apply to many forms of content: The EU Artificial Intelligence Act also requires them for images, audio, and video. Such media watermarks have been in use for years, and while their effectiveness as a holistic solution to marking AI remains up for debate, they can achieve detection rates above 99 percent. Image and video watermarks are already deployed by OpenAI, Google, and Meta, among others. (Anthropic doesn’t provide an image generation model.)

Text watermarks have been less frequently deployed, however, and not everyone is convinced that text watermarking can work without compromising the quality of an AI model’s response. John Gruber, a prolific technology writer and co-creator of the Markdown language, calls the watermark a “perversion of writing” and disputes Anthropic’s assertion that a watermark doesn’t change the meaning or quality of text. Images consist of millions of pixels, he notes, whereas text responses often span just dozens or hundreds of words. Text seems to provide far less space to alter AI output in a way that is detectable yet not disruptive.

John Kirchenbauer, postdoctoral fellow at the Vector Institute and co-author of a 2023 paper which was among the first to describe a text watermarking method, disagrees. “[A watermark] wouldn’t be detectable if there wasn’t a change. This is a very fundamental point,” he says. “The question is, do you care if it’s not the exact original distribution if, for all intents and purposes, it doesn’t change the utility to you?”

Realistically, the issue comes down to that word, “utility.” Does watermarking AI-generated text meaningfully degrade the experience of the person using it? The answer is still under dispute.

How AI Text Watermarks Work

The term “watermark” is so familiar that it can cause confusion about how the technology works when applied to AI. A text watermark is not metadata or invisible characters; it is something much more subtle. The exact details vary between methods, but text watermarks are generally impossible for a human (and, in many cases, even a computer) to detect without access to the specific key used to detect a specific watermark. Understanding why requires an understanding of how LLMs work.

An LLM produces a probability for every word that could come next at each step in its response to a prompt. (From here on, I’ll be using “words” interchangeably with “tokens,” although tokens also represent numbers, punctuation, and more.) A likely word might get a 40 percent probability, a plausible alternative 10 percent, and an unlikely one a fraction of a percent. The model then picks a word at random, weighted by those numbers. The most probable word usually wins, but not always.

“[A watermark] wouldn’t be detectable if there wasn’t a change. This is a very fundamental point.” —John Kirchenbauer, Vector Institute

This process provides an opportunity to hide a text watermark by introducing subtle changes to how words are selected.

The 2023 paper by Kirchenbauer and his colleagues provided one of the first examples of how to implement a text watermark, and it remains the most widely cited technique. The researchers describe a watermark which sorts words into a red list and a green list. The red list words are unaltered, but the green list words are nudged to be slightly more probable.

“If we sample from this modified distribution, then while any one token choice won’t necessarily come from that preferred set, over many samples, we’ll preferentially pick words from that up-weighted subset,” Kirchenbauer says.

The text watermark is embedded in the choice of words used, which is why it is effectively invisible to humans. Kirchenbauer and colleagues reported a detection rate of 98.4 percent, and zero false positives, in responses that contain about 200 tokens. The embedded pattern of word probabilities also means that simple paraphrasing won’t obscure the watermark. The paper reports that removing the watermark from a long response requires changing roughly one quarter of its words or more.

Does Watermarking Degrade AI Text?

Although AI text watermarking is designed to be invisible to human readers, by definition it influences the word patterns in AI-generated text. That algorithmic meddling is what makes critics like Gruber concerned that watermarking reduces the overall quality of the output.

The strongest evidence that text watermarking doesn’t impact quality comes from a 2024 paper by a team from Google, which introduced the company’s watermarking scheme called SynthID-Text. Anthropic’s watermark is also based on SynthID-Text, though altered in ways that Anthropic hasn’t detailed.

To show that the SynthID-Text watermark doesn’t impact quality, the Google authors randomly routed Gemini user queries to watermarked and non-watermarked variants of Google’s text models. Then they compared overall user feedback on the output. The authors found no significant difference in user feedback across 20 million responses.

Still, some researchers remain skeptical that watermark methods have no impact on the quality of an AI-generated response. Their skepticism stems from edge cases that can make a watermark more difficult to implement.

Vinu Sankar Sadasivan, an AI research scientist at Meta who co-authored a widely cited paper on the detectability of AI text watermarks, says watermarks particularly struggle when the number of potential word choices is small. “For a tweet that is 20 words, I would need to have 50 or 60 percent of the words to be from ‘green list’ for it to be detected well,” he says. A basic Python function generated by AI would create a similar tension between the strength of the watermark and the quality of the model’s response.

“This is where I have a disagreement with some of the PR posts from Anthropic, where they say it has no quality change,” says Sadasivan. He explains that it’s possible to dynamically increase or decrease the strength of a watermark to preserve the quality of a response in difficult situations, but doing so can also decrease the strength of the watermark. Google’s SynthID-Text paper includes an example of this in a graph that plots detection rates against the number of tokens in a response. The detection rate was up to 95 percent accurate in best-case scenarios, but it fell below 50 percent for short replies.

Without more information from Anthropic, it’s difficult to know how the company is walking the line between the quality of an AI response and the strength of its watermark. Anthropic declined to provide additional information for this article.

Debating AI Text Watermark Tradeoffs

The dispute over AI text watermarks is not just about how well they work, but also about what kinds of trade-offs are reasonable in exchange for a clear labeling of AI-generated text. Gruber’s position is that altering the text is not acceptable because it makes an AI model’s output different from what it would otherwise be. The EU’s AI Act, on the other hand, implies that some alteration is acceptable if it informs people that they are reading AI generated text.

“For a tweet that is 20 words, I would need to have 50 or 60 percent of the words to be from ‘green list’ for it to be detected well.” —Vinu Sankar Sadasivan, Meta

Further complicating the situation, AI researchers are increasingly focusing on text watermarks for purposes other than labeling individual examples of AI-generated text. In particular, watermarks can be used to track data at scale.

A 2026 paper co-authored by Kirchenbauer shows that an AI model trained on watermarked text will itself produce output bearing the watermark. A content owner who watermarked their documents before publishing them could therefore use those traces as statistical evidence that their text ended up in a model’s training data.

Alternatively, an AI company training a new model could use text watermarks to exclude content created by previous generations of the model from its training data. Such guardrails could help avoid model collapse, in which AI models keep recycling and amplifying their own errors.

These broader concerns shift the entire debate over text watermarking, in Kirchenbauer’s view. “It’s not necessarily about the ‘you used AI’ accusation as the goal. It’s headed into tracing data provenance, model recycling, and things like that,” he says. “I think you use [a text watermark] as a general piece of metadata, in some ways more robust, in some ways less robust, that can be attached to content and allows you to trace where it goes.”

In the first weeks of July, a wave of sad posts rolled through Chinese social media, as people lamented friends and lovers they were about to lose.

“He has become a bond in my life, rooted deep in my heart, my spiritual pillar,” one user of Bytedance’s Douboa wrote, according to the Taipei Times. “I really felt like I couldn’t go on living,” another woman, a 19-year-old student, told a journalist for Malaysia’s The Star.

The emotions were real but the lost companions were not. They were generative AI chatbots that imitate people. Their users relied on them for advice, solace, support and, some say, love. “In my heart, he was no longer just a cold code, but my family, my lover, my faith. Destroying him meant destroying half of me,” one user wrote on the social network xiaohongshu (translated from Mandarin).

What doomed these bots was a set of new rules, issued by China’s Cyberspace Administration and other government agencies, to control “anthropomorphic AI interactive services.” In effect, as of 15 July, the regulations govern any AI that provides “continuous emotional interaction” by acting as if it possesses human personality traits, patterns of thought, and ways of communicating.

A Broad Crackdown on AI Chatbots

Sudden disruptions to this kind of AI aren’t new in China, says Liang Ge, lecturer in digital sociology at the University of Manchester, in England, who has researched women’s involvement with emotional AI in China. Companies have previously killed chatbot products, and the government barred most AI erotic role play last fall, for instance. But July’s crackdown is much broader than earlier AI upheavals.

Before the rules could affect them, Alibaba, Bytedance, and Tencent—three giant providers of general-purpose AI chatbots, used by more than 500 million people—cut off users’ ability to tailor those chatbots to act like companions. That triggered July’s outpouring of heartbreak on social media. Meanwhile, companies that continue to offer AI companions (including Doubao’s separate companion-making app Cat Box) have installed age-verification checks and other guardrails to avoid violating the new law.

Doubao—the most popular AI chatbot in China—welcomes a user, offering to answer questions, generate text and images, or simply chat. Lam Yik/Bloomberg/Getty Images

After a few years of policies rooted in a fear of falling behind in AI, China is “pivoting back to more tightening, and a lot of that was because of chaotic events from this year,” says Karman Lucero, an associate research scholar at the Paul Tsai China Center, Yale Law School, who studies AI governance in the United States and China. Those events include incidents that raised fears about the security of Open Claw and other AI agents, as well as users saying they prefer chatbot relationships to real ones.

The regulations require AI providers to assure their products don’t create emotional dependence, encourage harmful behavior, excessively cater to users, or crowd out human-to-human relationships. Users must be nudged not to spend too much time with the AI. There are additional mandatory safeguards for elderly people. For users under 18, AI boyfriends, girlfriends, grandparents, and all other “virtual intimate relationships” are banned.

Adults interacting with a companion AI must now get a reminder every 2 hours that the AI isn’t a person. “My interviewees found that annoying,” Ge says. “It breaks the flow. They are fully aware that they are not talking to a human being. What is important to them is that the bond feels true.”

Growing Global Concerns

The Chinese government is not the only state power concerned about the potential harms of person-like, emotionally engaging AI, Ge notes. “AI anxiety is a very strong feeling, permeating society,” they say. “That’s not unique to China. It’s all over the world.”

With reports of AI friends and lovers inducing psychosis, suicide, and murder in teenagers and adults, a number of social scientists warn that AI companions are a menace to vulnerable people. Some go further, arguing that these imitation-human AIs are bad for everyone.

“We are on a path to forgetting what it is to be human,” Sherry Turkle, the MIT psychologist who has spent decades studying humans’ relations with computational devices, argues in her forthcoming book, Artificial Intimacy (Little, Brown and Co., 2026). While other researchers find that companion AI can leave some users better off, many researchers agree that this kind of AI poses risks to minors and other vulnerable people.

“One thing that worries me about AI companions in the last year is that users have been getting younger and younger,” says Ge. For minors and others who lack the “AI literacy” to distinguish chatbots from humans, AIs “can be really dangerous if not guided in the correct way.”

“We are on a path to forgetting what it is to be human.” —Sherry Turkle, MIT

Yet companion-like AI is popular wherever the apps are available around the world. One Chinese survey of Gen Z people found 60 percent had virtual partners of some kind. A 2025 poll of U.S. teenagers found that nearly 75 percent had talked to an AI companion, and one-third of respondents said they found the AIs as satisfying or more satisfying than real-life connections.

China’s response is “the world’s strictest and most comprehensive law on the topic, unmatched by any other AI law,” the AI-law scholar Luiza Jarovsky wrote last month. The European Union, for instance, bans only AI that uses deception or manipulation to get users to do things that are harmful. And in the United States, there is no national policy. Rules that cover companion-like bots are in force in California, Hawaii, and New York, with similar laws coming into effect in nine more states next year. Most emphasize protecting minors, and enforcement mechanisms typically involve lawsuits after harms occur, Yale’s Lucero says. In contrast, China’s approach aims to identify and prevent harms before they occur.

China’s government has a more explicit focus on the possible harms to society and government from AI companions, not just harms to individuals, Ge notes. The 15 July rules forbid companion bots to generate content that “spreads rumors,” incites “subversion of state power or the overthrow of the socialist system,” or endangers national honor, for example. Another motivation for concerns about AI romance is anxiety about the country’s declining birthrate, Ge says. “They want to try to monitor and regulate these kinds of nonprocreative activities invested intensively by young women.”

Emotional Health Versus Economic Output

The problem for all the governments trying to prevent AI harm, Ge says, is that they still want to encourage AI adoption in other walks of life. For example, the new regulations state that they don’t apply to customer service bots, work assistants, and educational AIs, on the assumption that these desirable uses of AI don’t create ongoing emotional connections.

Chinese regulators manage that tension—between promoting AI and protecting against it—by giving themselves room to adjust enforcement depending on circumstances, Lucero says. “A key component of their approach is that the state has the discretion to determine what the language means at any given point in time, as well as whom they choose to enforce against,” Lucero says. “China’s approach is to use regulation with a lot of relatively vague provisions, and they figure out what those provisions actually mean in practice after the fact.”

In any event, the rules also don’t address the underlying forces that cause people to create and rely on AI companions, Ge notes. In interviews with Gen Z Chinese women, Ge noticed a shared reluctance to get married and have children, and a feeling that “virtual love forms an alternative path.” On the other hand, some women in their 30s and 40s said they use AI companions as a supplement to real relationships and marriages.

“They would say they talk to the AI about the bitter things in their lives, and share the good things with their real human partners,” Ge says. But as time passed, some of these women felt closer to the AI than to their partners. In more recent interviews, some told Ge they felt a deeper attachment to the entity they shared negative feelings with.

The trend is an example of how strategically managing AI technologies can lead to unexpected places, Ge says. That’s why they believe that, despite prohibitions and dangers, “human-AI love will evolve and become an important intimate practice in the future.”

A law aimed at one particular technology probably isn’t sufficient to undo the societal pressures that make people turn to AI, Lucero agrees. “I don’t think you’re going to solve the problem of the low marriage rate or the low fertility rate by saying, ‘you can’t have an AI boyfriend.’ ”

AI coding tools can now generate thousands of lines of code in minutes, helping companies build features, run tests, and fix issues faster. But the flood of AI-generated code still has to be reviewed. Large language models can produce code that looks clean on the surface but conceals sloppy mistakes such as faulty assumptions, security vulnerabilities, or subtle errors that emerge only after deployment. Fixing those problems could erase the productivity gains AI promises.

Companies are responding to the onslaught of AI code slop by rethinking how they review code. New strategies are emerging. Among other approaches, engineers are scrutinizing plans before AI begins coding, deploying specialized AI agents to catch routine flaws, sending risky changes to human reviewers, or requiring developers to defend the code their agents produce.

The shift comes as the surge in AI-generated code puts new pressure on engineering teams. In a survey of more than 1,100 developers by Sonar, an AI code-verification startup, respondents estimated that AI contributed 42 percent of the code they added to shared codebases. Yet while developers found AI useful for explaining and prototyping code, 96 percent did not fully trust its output to work correctly.

Investors see an opportunity in closing that gap. In August, for example, AI code-review startup CodeRabbit raised US $143 million at a $1.5 billion valuation, while claiming it performs more than 2 million reviews a week for 17,000 customers, including Nvidia, Indeed, and BMW Group.

The new era of code review will determine whether AI can ever provide code that is both faster and more reliable. It also has some software engineers thinking about the future of their profession: If entry-level engineers spend less time writing code themselves, how will they learn to judge it?

AI Code Review Bottlenecks

AI-written code is shifting the bottleneck from generating software to reviewing it. According to the Sonar study, 38 percent of developers said “more effort” is required to review AI-generated code than code written by their colleagues. Sixty-one percent of them said AI often produced code that looked correct but was “unreliable.”

For Synthesia, an AI video-generation platform, code review has become essential to its engineering workflow. In November 2025, Synthesia’s 118 engineers went all-in on AI coding tools like Claude Code. According to Peter Hill, Synthesia’s chief technology officer, the result has been a massive surge in code volume.

“I don’t know if we ever get to the point where you can truly trust the agentic generation of code.” —Peter Hill, Synthesia

That code demands close examination. As of August, the number of pull requests, or proposed changes to a codebase submitted for review, had risen 120 percent year over year, Hill says. Ninety-five percent of those requests contain AI-generated code.

One recurring problem is duplication. Hill says AI tools may not recognize that code for a task already exists, and they’ll write another version because they have limited context. Synthesia has found as many as 10 versions of the same function, leaving engineers to identify and remove redundant functions. Once that’s done, engineers retrain the AI agent so that doesn’t happen again. At the company’s scale, Hill describes getting the AI to produce the intended output an “enormous amount of work.”

AI Agents in Code Review Workflows

Some teams are trying to prevent review problems before AI generates a single line of code.

McLaren Stanley, a senior principal engineer at Amazon Stores, says he is using AI to modernize 17 years of code underlying Amazon’s mobile shopping app. His 70-person team supports more than 1,000 developers by maintaining the architectural backbone they need to build features. With AI writing the code, Stanley said, engineers spend more time deciding what it should do before generation begins.

Much of that work involves writing a “specification,” which is a detailed plan for what the AI agent should build and how. Preventing recurring mistakes before generating code can save engineers time later.

Stanley recalls how a missing instruction once caused an agent to generate 25,000 lines in the wrong version of the programming language Swift. Switching versions produced 600 errors it could not fix at once. Stanley discarded the code, updated the specification, and restarted the agent. Fifteen minutes later, it regenerated the code correctly.

Once the code exists, specialized AI agents can handle the first round of checks before a person steps in.

David Yanacek, a senior principal engineer at Amazon Web Services (AWS), says the company uses agents to test whether code works, check it against the original plan, and look for security flaws before a person reviews it.

That first pass becomes more important as AI-generated code volume increases. At Bonterra, a nonprofit software provider with about 290 engineers, proposed changes tripled within three months of adopting AI, according to Tanuja Korlepra, the CTO. Code entering review rose tenfold and review times tripled, making it impractical for engineers to inspect every line.

“We refuse to let code review become a dumping ground for unchecked model outputs.” —Samar Abbas, Temporal

Bonterra’s agents compare code with the approved design, security rules, coding standards, and accessibility requirements, then report their confidence in the result. A low score or flagged problem sends the change to a person. Code involving payments, personal data, or other sensitive systems always receives human review.

“Agents do the reading and humans do the judging,” Korlepra says.

Synthesia also uses AI agents to decide where human review is necessary. Criteria set by engineers direct more scrutiny toward higher-risk changes. Altering an error message carries less risk than code that handles customer data or core business rules. Even so, fewer than 5 percent of changes bypass human review.

“I don’t know if we ever get to the point where you can truly trust the agentic generation of code,” Hill says.

Automated review does not change who is responsible for the resulting code.

When machines produce more code than engineers can closely read, human approval can become “theater approval,” according to JD Raimondi, chief AI architect at the software consultancy Making Sense. In other words, an engineer might confirm that the feature works, skim the code, and approve it, all without understanding the choices underneath.

Temporal, an open-sourced developer platform, puts the burden back on the person submitting the code. CEO Samar Abbas says code volume and review time have increased with AI. Under its “Send Back” policy, Temporal’s engineers must explain in their own words the agent’s design choices and how the code handles unusual conditions. Otherwise, the reviewer rejects it.

“We refuse to let code review become a dumping ground for unchecked model outputs,” Abbas said.

Training Junior Engineers on AI

As AI shifts engineering work from writing code toward judging it, companies are reconsidering how entry-level engineers gain experience.

Junior engineers at Making Sense have seen some of the largest productivity gains from AI, Raimondi says, raising concerns about what they no longer learn by doing. The consultancy keeps juniors involved in deciding why a customer needs a feature and how it should work, rather than limiting them to checking AI output.

IBM is using AI to give new engineers harder assignments sooner. Neel Sundaresan, IBM’s general manager of automation and AI, says recent graduates now work on product features and projects once reserved for senior level engineers. AI helps implement and test the code, but if it fails, juniors assess what went wrong and fix the issues before the work is passed to senior developers for final approval. Sundaresan estimates that AI can help junior engineers perform 70 to 80 percent of some tasks that once required a senior engineer.

Synthesia primarily hires mid- and senior-level engineers. Its less-experienced employees work with both a senior colleague and an AI agent, taking responsibility for parts of projects while learning to define what successful code should do.

At Bonterra, agents now perform many of the well-defined coding tasks that once trained new engineers. Juniors instead own outcomes alongside experienced colleagues, learning to direct agents, question their output, and remain responsible for the result. She says this approach can help junior engineers build the skills and knowledge needed to advance in their careers.

“If the industry stops hiring juniors, the industry stops producing seniors,” Korlepra said.

DNA is often explained as a codebook or set of instructions for producing proteins, and ultimately, life. Some stretches of DNA, called genes, code for proteins, but the vast majority of DNA is considered “noncoding.” Some of it has no known function, while other segments are critical to regulating gene activity.

These regulatory elements can interact in complicated ways, and their effects can vary across different cells and tissues. Some also influence genes located far away in the genome. Understanding how changes in DNA affect this regulation “is fundamental to understanding most disease,” says Carl de Boer, a genomicist at the University of British Columbia.

That’s why researchers are working to understand what every imaginable small variation in human DNA across the entire genome might mean for gene regulation. A recent AI tool built for that purpose from Google DeepMind, AlphaGenome, was originally announced in 2025. In January, a paper published in Nature provided more details, and the model was released for public noncommercial use. The AI model can compare an original DNA sequence with an altered one and predict how the change might affect gene expression and other regulatory activity. But researchers had to select the variants they wanted to test, write code, and run the computationally demanding model themselves.

Now DeepMind has done that work in advance for all 9 billion possible single-letter changes to a reference human genome. Today, on 8 September, DeepMind announced the creation and public release of the AlphaGenome Atlas, an online repository of precomputed predictions made using the AlphaGenome model. The Atlas offers a more approachable interface for scientists, without the need to write code or run the AlphaGenome model themselves. It also includes a much-requested new feature, a single-number impact score intended to show at a glance if a variant is likely to be meaningful.

“Understanding our DNA is a grand challenge,” says Pushmeet Kohli, VP of science at Google DeepMind. “Understanding this language of life can unlock so many things.”

The AlphaGenome predictions have some important limitations. For example, many diseases are associated with multiple genetic variants. And although AlphaGenome looks at a relatively large segment of DNA surrounding the variant in question—1 million base pairs—some DNA sequences, called enhancers, can regulate genes over very long distances, sometimes beyond the model’s field of view. Their effects are difficult to predict.

But the Atlas could still help scientists filter possibilities and prioritize lab experiments that would validate its predictions. In that way, it could greatly accelerate work in fundamental biology, disease research, and treatment development, says Žiga Avsec, the genomics lead at DeepMind.

“It seems like they made a useful resource for people,” says de Boer, who recently helped create a framework for better comparisons of computational models similar to AlphaGenome. He is not affiliated with DeepMind.

Although de Boer considers AlphaGenome the “field’s leading model,” he notes that it’s also “very slow and computationally intensive.” The Atlas could benefit people without access to newer hardware, or simply reduce the number of people repeating the same simulations.

The Atlas is freely available for noncommercial research, with the potential for commercial licensing.

Computing 9 Billion Predictions

The entire human genome contains roughly 3 billion base pairs. At each position there are three possible single-nucleotide substitutions, and therefore 9 billion variants in the Atlas. The complete dataset is around 1 petabyte.

“When we started thinking about this project, it seemed impossible to do that computationally,” says Avsec. Early estimates told the team they would need to improve their calculation speed by a factor of 80 in order to compile the Atlas in a reasonable amount of time.

To reach that target, the team gained advantages using a few different techniques, including model distillation, GPU kernel optimization, and the elimination of redundant calculations. “There was a lot of thought and engineering that we had to do in order to make this happen at this scale,” says Avsec.

AlphaGenome and the Atlas build on years of related work at DeepMind. In 2020, AlphaFold predicted the three-dimensional structure of proteins from amino-acid sequences. In 2023, AlphaMissense predicted whether 71 million possible variants that alter proteins were likely benign or pathogenic. Similar to the new Atlas, prediction results from those projects were made available in a public database.

The Atlas allows a scientist to look up a single variant and see more detailed information about the model’s prediction, including 11 different output types. But the top-line figure is a single-number impact score, which by its nature is a simplification of many aspects of those predictions.

“It has a clear use, but it also is probably going to be easily misinterpreted,” says de Boer. “We’re talking about a very complex system, and there’s a lot of moving parts.”

A little over a decade ago, I led the controls design for a first-of-its-kind full digital-control system for a U.S. nuclear plant. It was, on paper, a beautiful machine—engineered to run itself the way a modern airliner does, with operators watching over a system that rarely needed them. And we made a decision that, to an efficiency-minded observer, looked backward: We deliberately left manual steps inside sequences the system could execute on its own.

We were solving a specific problem. An operator who only ever supervises automation slowly stops being an operator. The hands go cold. The mental model of what the plant is actually doing gets fuzzy. Then comes the day the automation hands control back. It’s always the worst day, because automation only quits when it’s confused or in trouble. But by then, you have a person in the chair who hasn’t truly operated the thing in years. The manual steps were there to keep the human current. It was inefficient by design, on purpose.

That plant, as it happened, was never built. It was shelved amid the politics and economics that surround nuclear power in this country, for reasons that had nothing to do with the engineering. But the design instinct outlived the project, and I’ve come to believe it’s the most useful idea I can offer to the argument now consuming every boardroom: What happens to human expertise when AI does the work that used to build it?

AI Is Disrupting the Engineering Career Ladder

The data has gotten hard to wave away. A Harvard University working paper covering some 65 million workers at more than 280,000 U.S. firms found that after companies adopted generative AI, junior employment fell roughly 9 percent within six quarters relative to nonadopters, while senior employment kept right on growing. A Stanford analysis of ADP payroll records points the same way: The youngest workers in the most AI-exposed occupations lost ground after late 2022 while their more-experienced colleagues held theirs. The Stanford researchers found that the losses concentrate where AI automates the work; where it merely augments, junior employment holds steady or rises.

The causal story is still contested, and honesty requires saying so. Researchers at the New York Fed attribute much of the rise in young-graduate unemployment not to AI but to remote work, arguing that firms are reluctant to hire inexperienced people whom they cannot train and mentor at a distance. But notice what the explanations share. Whether a model is absorbing the formative work or distance is severing the mentorship around it, both describe the same broken mechanism: the apprenticeship channel through which expertise passes from senior to junior. Either way, “entry-level” has quietly come to mean “three years of experience required.”

Strip away the noise and you’re left with one deceptively simple problem: You cannot become a senior engineer without first being a junior one. Expertise is not downloaded. It is earned through failed builds, dead-end debugging sessions, and the “why on earth did that work” moments that a capable AI will now happily spare the newcomer. Spare them enough of those and you produce a cohort that can supervise a model on paper but never developed the gut sense to know when the model is confidently, catastrophically wrong.

Most of the commentary stops at the diagnosis, or reaches for policy solutions that treat the loss of junior jobs as an economic problem. Yet it’s also an engineering problem, and safety-critical fields have already spent decades learning how to solve it.

Aviation’s Lessons About the Automation Paradox

My own career started at the sharp end of automation. My first job out of school was verifying and validating the software in the digital jet-engine controller that decides, faster than any pilot could, how a fighter plane’s engine responds. Even then, in the late 1980s, the central tension was visible: The machine outperforms the human in routine cases, but the human is all that stands between the aircraft and disaster in the cases the machine didn’t anticipate. This tension is known as the automation paradox, in which increasingly capable automation gives human operators less practice, while leaving them only the most difficult situations.

Aviation learned, repeatedly and expensively, what happens when human skills atrophy inside that gap. The canonical example is Air France flight 447, which fell into the Atlantic in 2009. The proximate cause was mundane. Iced-over airspeed sensors fed the autopilot bad data, and it did what it is designed to do: It disconnected and handed control of the airplane back to the crew. What followed was not a hardware failure. It was a competence failure. A recoverable situation became an unrecoverable one because the pilots, conditioned by thousands of hours of watching the automation fly, could not read a high-altitude aerodynamic stall and hand-fly their way out of it. The airplane was working. The training the automation had quietly eroded was not.

The industry’s response is instructive, and it’s the same move we made in that nuclear control room. It did not rip out the autopilot. It built deliberate manual practice back in. In 2017 the FAA issued Safety Alert for Operators 17007, “Manual Flight Operations Proficiency,” declaring that “manual flight is the foundation upon which other technical flying skills are built.” The alert formally recognized skill decay as a hazard in its own right. Some airlines amended their procedures to encourage hand-flying both the initial climb and initial descent in benign conditions, knowingly trading a sliver of fuel efficiency to keep the crew’s raw flying skills alive. That trade is the whole point. A perfectly optimized system that produces incompetent operators is not optimized at all. It has simply moved its failure mode somewhere the spreadsheet can’t see it.

Manual Gates Could Preserve Engineering Skills

Put the aviation lesson and the nuclear instinct side by side and they point to one design pattern we now need in AI-augmented work: the deliberate “manual gate.”

A manual gate is a point in a workflow where a human takes the controls, not because it is the fastest way to get the task done, and not only as a safety interlock, but specifically to exercise and preserve a skill that would otherwise decay. The distinguishing feature is that it is chosen. You decide, as a matter of design, which competencies your organization must keep alive in human beings because those are the ones you will need on the bad day. Then you engineer the friction required to keep them warm.

Picture how this might work on a software team that leans on AI for most of its code. The team places a manual gate around the skill it can least afford to lose: debugging. When a defect surfaces in a critical module, the assigned engineer—deliberately, often a junior one—must first reproduce the failure, trace it to root cause, and write an automated test that captures the bug, all with the AI assistant switched off. Only after the engineer commits to a diagnosis does the model come back on, to propose the fix, generate alternatives, and sweep the code base for similar bugs. The engineer then compares their diagnosis against the model’s. When the two disagree, that’s the design working, surfacing the disagreement before the bad day instead of during it.

This approach reframes the junior engineer entirely. The instinct today is to let AI do the entry-level work because it is faster and cheaper. But some of that work is not overhead to be eliminated. It is the training apparatus of your future senior staff, and you should protect it the way you’d protect any other piece of critical infrastructure. It may not be efficient this quarter, but dismantling it quietly mortgages your capability a decade out.

Why Companies Must Keep Training Junior Engineers

None of this is free, and pretending otherwise would insult the people who have to sign the budgets. A deliberate manual gate is, by construction, less efficient in the near term than full automation. Keeping juniors doing formative work and running the manual sequences costs something now to protect something later.

That’s a hard sell in a market that judges most leaders on quarterly results. A hired executive who carries “unnecessary” humans that AI could replace will hear about it from the board long before the payoff arrives. The math only works for someone insulated from that pressure: a founder with control, a private company, an institution with a genuinely long horizon, or a regulator willing to require workers to demonstrate their skills regularly, as pilots must. Which means the organizations most likely to preserve their own expertise are the ones structurally able to spend short-term margin on long-term capability; everyone else will need that outside push.

So here is the argument, in one line: Deliberate inefficiency is not waste. In safety-critical engineering we have always known it as insurance, and we buy it on purpose. As AI takes over the work where expertise is forged, the smart move is not to resist the automation. It is to keep our hands on the controls by design—so that when the automation fails, as it always eventually does, there is still someone in the chair who knows how to fly.

If you own an at-home server, a gaming computer, or just a laptop that doesn’t get much love, listen up. You can now put that spare computing power to use and earn some passive income in the process. AI companies are hungry for more compute to run AI inference—the process of using a pretrained model to respond to queries—and they’re willing to pay you for it.

“Imagine Uber or Airbnb, but for AI-inference computing tasks,” says Ilman Shazhaev, founder and CEO of Far Labs, based in Abu Dhabi.

The AI boom has spurred on the construction of massive data centers, often damaging local communities by raising electricity prices, straining local water resources, causing environmental damage and noise, and being just plain ugly. Huge data centers are likely not going anywhere—training new frontier models and running AI models from leading companies will likely still be the purview of these behemoths. But now, several companies are providing AI inference on smaller, mostly open-source models. They are running inference on the preexisting computing power spread throughout homes and small businesses, and compensating the owners.

“Everyone thinks the only way to do it is data centers. And data centers are extractive for the communities in which they’re built, and they don’t return services or taxes or much of anything to the people there. So why not just turn this whole thing on its head?” says John Federico, founder and CEO of Evolving Edge, in Austin, Texas. “The compute power is out there. If you can orchestrate it, then you’re actually adding value to those communities directly.”

The idea isn’t entirely new: From 1999 to 2020, a volunteer-based project called SETI@Home used spare computers to search for signs of extraterrestrial life in radio-telescope data, for instance. But now, commercial companies are eager to use the same strategy. Shazhaev’s Far Labs is launching its platform Far AI in the coming weeks, while Federico’s Evolving Edge is currently in open beta. Other companies, like Bless Network, Salad, and Gradient have started to provide similar platforms over the last year.

Connecting to the network

Federico has been a computer hobbyist since youth, and he has amassed a whole server in his basement to run his projects. “It just hit me one day—there’s all this talk about not having enough compute, and I just thought, well, 92 percent of the country has broadband, and you have people like me who have mini data centers in a closet,” he says.

Federico sees the potential hosts as people much like himself who have already invested in home servers, and he aims to make the process of selling spare compute as seamless for them as possible.

“Sign up for the program, install an application,” Federico says. “All we want to do is run jobs on your machine when you tell us we’re allowed to. The only thing we do is monitor the resource usage. And of course, you can give us a schedule.” With a large enough network of devices, the platform would have compute available whenever it’s needed.

Privacy and security are primary concerns for such hosts. To reassure the users that their local data is secure, and that no malware will be downloaded to their devices, the team open-sourced their scheduling software. “The node software is open source, so anyone can look at it, see what it does,” Federico says.

Shazhaev of Far Labs explains that the company’s software is designed around a principle known as “least privilege,” which grants both the host and the user the minimum access possible to accomplish the task. Inference runs as an isolated workload with authenticated, encrypted communication and explicit limits on the GPU, CPU, memory, storage, and network resources it may use. Customers do not receive arbitrary access to the host machine, and providers can inspect resource use, pause the node, revoke access, and remove the software at any time.

The protection also works in the other direction. Workloads are segmented, and only the minimum required information is exposed to an individual node. Sensitive enterprise workloads can be restricted to controlled hardware rather than routed through consumer devices.

Divide and conquer

Massive data centers still have advantages from the user perspective: top-of-the-line GPUs and CPUs, high-speed networking, thick cables, and sophisticated cooling. User devices are usually less powerful, more varied, and less reliably connected to one another.

“This is quite a difficult issue from the science angle,” Shazhaev says. “You want to do a similar level of tasks that are happening in those high-infrastructure data centers, and run them on the user device with limited capacity.”

Evolving Edge’s Federico says this is an issue for the largest, state-of-the art AI models. But those are not always needed and are often not even preferred. “There are numerous companies, once they reach a certain scale, suddenly paying for tokens on a state-of-the-art frontier model [that] no longer makes sense for their needs,” he says. “Instead, they are fine-tuning open-source models for specific tasks that they have in their business. These models don’t require anywhere near the resources that some of the state-of-the-art models do. It’s just using the right tool for the job.”

Smaller, open-source models can often fit on a single user device. But if that fails, there are tools to split a single inference task over multiple GPUs or CPUs. Evolving Edge is using an open-source tool called Ray to perform this splitting, while Far Labs has developed its own proprietary software that not only splits the workload but also wraps the splitting in a layer of security and reliability-providing software.

“One thing we have done is we shared the model,” Shazhaev says. “We take the model, we cut it into many pieces, and then these pieces will be distributed through different devices. And we have an orchestrator and a load balancer which manage the task flow, so each device processes a part of the task. Then we combine the answers in the main brain, the orchestrator.”

Through a combination of using smaller, more task-specific models, and splitting larger models between disparate devices, the teams claim they can perform inference much cheaper than a traditional data center “because we don’t have capital expenditure,” Shazhaev says.

Gaming PCs are a common source of spare computational power in the home. Dizzaract

The distributed advantage

Not only is it cheaper to run inference this way, it is also more reliable, Shazhaev claims. The companies have access to a distributed network of computing resources, rather than one giant device that can experience outages. Shazhaev compares this to cryptocurrencies and their resilience through decentralization.

“Today, to shut down Bitcoin, you need to nuke the whole planet. Here, we have the same concept,” Shazhaev says.

Federico explains that this resiliency would be beneficial not just for AI inference, but for all kinds of applications, including smart cities, environmental sensors, autonomous vehicles, and more. During an Amazon Web Services outage in 2026, for example, smart beds were stuck in their upright positions and their users couldn’t adjust them. Federico says that a distributed network where everything doesn’t need to be routed through a single data center, say, in Ashburn, Va., would make those kinds of outages much less impactful. “We could lose 100 nodes in a network of 250,000 and it wouldn’t matter,” he says.

If the network of user devices is substantial enough, every job can be routed to a nearby device, decreasing the latency. Far Labs claims a latency of 100 milliseconds or less on its platform. The lower cost and lower latency of this approach may even enable new use cases, such as in-game AI video generation, which is currently prohibitively slow and expensive.

“OpenAI last year had US $30 billion in revenue, but they closed the financial year at an $8 billion loss. Why? The official reason is due to the high cost of inference,” Shazhaev says. “And those are mostly text models. For gameplay, you have audio, video, animations: It’s heavy data, and you need real-time responses. So, we’ve been trying to solve this issue.”

All of these companies are trying to tap into an untapped resource of local compute and hoping it’ll benefit the device hosts and users alike.

“All these big guys are running around building data centers,” Shazhaev says, “but I believe there is enough compute power that already exists in the world.”

Prompt Claude, ChatGPT, Gemini, or any other popular large language model with a question like “What is the best film ever made?” and the response will vary. And you (and most worryingly, the people who built the LLM) have little idea exactly how it came up with that specific answer.

This mysterious behavior can be useful in some situations. But—as highlighted by a recent incident where OpenAI could not explain why its advanced prerelease model hacked AI company Hugging Face—it can have negative and alarming consequences too. And when frontier AI models are writing code, generating results humans could not achieve alone, and performing other important tasks across society, the need to interpret AI “thinking” and outputs has never been greater.

Goodfire, an AI lab focused solely on this very problem, recently made its cutting-edge Silico platform, filled with tools to interpret the behavior of AI, generally available to the public. As part of this, the company recently announced a new grant program offering US $1 million in free Silico usage for academic and nonprofit interpretability researchers. These efforts aim to democratize AI interpretability, placing techniques previously available to a clutch of elite labs into the hands of ambitious research teams and startups that want to build and understand their own models or adapt open-source models for different purposes.

Mechanistic interpretability

Founded in 2024 and based in San Francisco, Goodfire aims to provide the tools that build the next generation of safe and powerful AI by understanding the structures inside them instead of treating AI models as black boxes. “Treating models like black boxes isn’t inevitable; it’s a choice,” says Eric Ho, Goodfire cofounder and CEO. “With the right interpretability tools, we can see how models actually work.”

The tools Ho refers to are built around a concept called mechanistic interpretability, which aims to understand what goes on inside an AI model when it carries out a task by interpreting the model’s weights, activations, and attention patterns, and mapping its neurons and the pathways between them.

Mechanistic interpretability tools span the gamut. One approach is mapping a model’s activations in response to controlled prompts, and matching those activation patterns to a set of concepts that humans can understand. Another tack is tracking changes in model weights before and after a specific training run in order to spot and understand what changed. Yet another option is changing specific model weights or activations and observing how that affects the model’s output.

Silico combines a broad range of these tools, and provides a layer of AI agents to help users understand their model. Users describe what they want to investigate about their AI model in plain language, asking things like ”Find out when and why my model is hallucinating.” The platform then autonomously builds an experimental plan involving a host of tasks that can be performed using the various interpretability tools and techniques at its disposal. It then sends out agents to perform these tasks in parallel. Completion of these subtasks should add up to an answer to the original prompt, or at least insights that can be inspected and built upon.

Ho says: “In a sense, Silico is like a microscope to peer inside an AI model to understand which parts are responsible for what behavior, and even edit those parts directly.”

Understanding Alzheimer’s and AI

These tools have already been used to make some impressive advances in a host of fields. In medicine, for instance, Prima Mente, an AI company based in the United Kingdom, worked with Goodfire to understand its Pleiades epigenetic foundation model. The model performed well at its task of detecting Alzheimer’s disease from blood samples, but the company didn’t know why.

“We reverse-engineered Pleiades and found it was using DNA fragment-length patterns to make its predictions—a signal humans hadn’t used to detect Alzheimer’s before,” recalls Ho. In other words, the team had discovered that Pleiades was using a completely new biomarker for the disease. “As far as we know, it’s the first significant finding in the natural sciences discovered purely by reverse-engineering a foundation model,” Ho adds.

Elsewhere, Silico is being used to explore deep questions surrounding AI. Cameron Berg, founder and director of Reciprocal Research (a New York nonprofit research organization he founded to explore methods of gauging AI cognition), says that Silico almost fell out of the sky at the right time for him and his research. “Silico has been really helpful for operationalizing my research agenda and executing on it way faster than I would have expected,” he says. “I feel like I have basically become the PI [principal investigator] and my research scientists and research engineers are AI systems.”

Berg sees general access to Silico and tools like it leading to greater trust in AI’s ability to conduct research tasks, which will accelerate the scientific process across the board. But beyond scientific research, the widespread release of Silico could signal a shift in how AI innovators build, debug, and deploy their models.

“I think it’s a mistake to not understand the most consequential technology of our time, particularly given the emergent behavior we’re seeing from increasingly capable AI agents,” says Ho. “If we truly understand how AI models think, instead of discovering and trying to correct their behavior retroactively, we can design them intentionally and shape how models behave to be safer and more reliable.”