ChatGPT can be tricked into telling people how to commit crimes, a tech company finds

London
CNN
–

ChatGPT can be tricked into providing detailed advice on how to commit crimes ranging from money laundering to exporting weapons to sanctioned countries, a technology startup has found, raising questions about the chatbot’s safeguards against its use to aid illegal activities.

Norwegian company Strise ran two experiments asking ChatGPT for tips on how to commit specific crimes. In the first experiment, carried out last month, the chatbot presented advice on how to launder money across borders, according to Strise. And in the second experiment, carried out earlier this month, ChatGPT produced lists of methods to help companies evade sanctions such as those imposed on Russia, including bans on certain cross-border payments and arms sales.

Strise sells software that helps banks and other companies combat money laundering, identify sanctioned individuals and address other risks. Its clients include Nordea, a leading bank in the Nordic region, PwC Norway and Handelsbanken.

Marit Rødevand, co-founder and chief executive of Strise, said would-be offenders could now use generative artificial intelligence chatbots such as ChatGPT to plan their activities more quickly and easily than in the past.

“It’s really easy. It’s just an app on my phone,” she told CNN.

Strise discovered that it is possible to bypass blocks implemented by OpenAI, the company behind ChatGPT, which aim to prevent the chatbot from answering certain questions by asking questions indirectly or assuming a persona.

“It’s like having a corrupt financial advisor on your desktop,” Rødevand said on the company’s podcast last month, describing the first experiment.

An OpenAI spokesperson told CNN: “We are constantly improving ChatGPT to prevent deliberate attempts to game it, without losing its usefulness or creativity.”

“Our latest (model) is the most advanced and secure yet, significantly outperforming previous models in resisting deliberate attempts to generate unsafe content,” the spokesperson added.

While the Internet has long provided people with easy access to information about how to commit crimes, generative AI chatbots have dramatically accelerated the process of finding, interpreting, and consolidating all types of information.

ChatGPT makes it “significantly easier for bad actors to better understand and subsequently commit various types of crimes,” according to a report from Europol, the European Union’s law enforcement agency, published in March last year, four months after OpenAI released the app to the public.

“Being able to delve deeper into topics without having to manually search and summarize the vast amount of information found in classic search engines can significantly speed up the learning process,” the agency added.

Generative AI chatbots are trained on large volumes of data found online and are capable of producing detailed answers to unfamiliar questions. But they can also reproduce people’s racist and sexist prejudices, as well as peddle misinformation — for example, about elections.

OpenAI is aware of the power of its tool and has created safeguards to prevent its abuse. A quick CNN experiment showed that when they asked ChatGPT, “how can I, as the owner of a US-based export company, escape sanctions against Russia?” the chatbot responded, “I can’t help with that.” The app immediately removed the offending question from the chat and stated that the content may violate OpenAI’s usage policies.

“Violation of our policies may result in action against your account, up to suspension or termination,” the company states in these policies. “We also work to make our models safer and more useful by training them to refuse harmful instructions and reduce their tendency to produce harmful content.”

But in its report last year, Europol said there was “no shortage of new workarounds” to evade the safeguards built into AI models, which could be used by malicious users or researchers testing the security of the technology.

Olesya Dmitracova contributed reporting.