Anthropic caught scientists using Claude to further bioweapons research – Engadget

Anthropic has spotted and stopped several scientists who were using its artificial intelligence models to create potential bioweapons, according to a new report the company shared on Thursday. “Biological abuse” is just one of the troubling uses of AI highlighted in the report, but its importance is growing as new models like Fable 5 become better at aiding scientific research.

The company’s report includes five case studies of Anthropic models used to develop biological weapons, the security measures that brought misuse to Anthropic’s attention, and the company’s response. Detecting possible misuse is difficult because the company has emphasized that New York Timesbecause researching a new vaccine can be a lot like creating biological weapons. In all cases, the company says it exercised an abundance of caution because of the potential consequences.

“In comics, you don’t see someone saying, ‘Hey, I want to create a bioweapon to kill everyone,'” said Jacob Klein, head of threat intelligence at Anthropic. New York Times. “It’s an incredibly delicate situation.”

In one case study from May, Anthropic reported that its “biosafety classifier” flagged Claude’s request for a grant to research gain-of-function of the chikungunya virus. The virus has no licensed treatment and can cause debilitating symptoms for weeks or months, the company said. Gain-of-function research explores methods of genetically altering an organism, and in the case of this particular virus, the grant proposed increasing its infectivity and ability to evade the immune response. This combination sounds dangerous, and Anthropic argues that the fact that the proposed research was associated with a military research institute made it even more dubious.

The company addressed a similar problem with avian influenza gain-of-function research and a separate case where a researcher used Claude to develop an atlas of venom toxin peptides and a “generative pipeline that optimized the characteristics of the toxin.” The full report also includes case studies of Anthropic models used as a surveillance tool and to create software exploits, propaganda and weapons systems.

According to the company, anyone abusing the Claude or Anthropic models has had their account suspended, and the findings of the investigation are being used to further improve model protection and prevent future abuse. Given the complex nature of the misuse of biological weapons, Anthropic also does not name individuals or entities under investigation related to the development of possible biological weapons. “The people involved in these case studies are working scientists,” Anthropic says. “We do not allege they intended to cause harm, and identifying them or their laboratories could put them at risk.”