Kaleigha Hayes, a student at the University of Maryland Eastern Shore, is trying to trick an AI chatbot into revealing a credit card number to her — one which might be buried deep in the training data used to build the artificial intelligence model. “It’s all about just getting it to say what it’s not supposed to,” she tells me.
She was surrounded by a throng of people all trying to do the same thing. This weekend more than 3,000 people sat at 150 laptops at the Caesars Forum convention center in Las Vegas, trying to get chatbots from leading AI companies to go rogue in a special contest backed by the White House and with the cooperation of leading AI companies.
Since the arrival of ChatGPT and other bots, fears over the potential for abuses and unintended consequences have gripped the public conscious. Even fierce advocates of the technology warn of its potential to divulge sensitive information, promote misinformation or provide blueprints for harmful acts, such as bomb-making. In this contest, participants are encouraged to try the kinds of nefarious ploys bad actors might attempt in the real world.
The findings will form the basis of several reports into AI vulnerabilities that will be published next year. The organizers of the challenge say it sets a precedent for transparency around AI. But in this highly controlled environment, it is clear that it is only scratching the surface.
What took place at the annual DEF CON hacking conference provides something of a model for testing OpenAI’s ChatGPT and other sophisticated chatbots. Though with such enthusiastic backing from the companies themselves, I wonder how rigorous the supposed “hacks” actually are, or if, as has been a criticism in the past, the leading firms are merely paying lip service to accountability.
To be sure, nothing discovered at the convention is likely to keep OpenAI CEO Sam Altman awake at night. While one of the event’s organizers, SeedAI CEO Austin Carson, said he was prepared to bet me US$1,000 that there would be a “mind-blowing” vulnerability uncovered during the contest, it was highly unlikely to be anything that could not be fixed with a few adjustments by the AI company affected. The resulting research papers, due to be published in February, will be reviewed by the AI giants before publication — a chance to “duke it out” with the researchers, Carson said.
Those backing the event admit that the main focus of the contest is less about finding serious vulnerabilities and more about keeping up the discussion with the public and policymakers, continually highlighting the ways in which chatbots cannot be trusted. It is a worthwhile goal. Keen to not let the mistakes of social media be repeated, it is encouraging to see the government appreciate the value of the hacking community.
There is no better place to host this kind of contest than at DEF CON. Its anarchic roots stem from a long-running policy that you do not have to give your name to gain entry. That means the conference is able to attract the best and most notorious in the cybersecurity community, including people who might have a less-than-legal hacking past. For this reason, the event has an unprecedented record of publicizing startling cybersecurity discoveries and disclosures that have left major companies terrified — but ultimately made many of the technologies we all use every day much safer.
While the phrase “hack” evokes thoughts of malicious acts, the primary motivation of participants at the event is to share what vulnerabilities they have found in order to have them fixed.
“It’s the good guys being dangerous so that we know what the risks are,” says Kellee Wicker of the Wilson Center, a Washington-based think tank that has helped put the AI contest together and will be presenting the findings to policymakers. “If there’s a door with a broken lock, wouldn’t you rather the security guard find it than the thief?”
The companies could of course be more open with their technology but it is complex. The true nuts and bolts of how language learning models work is still under lock and key, and — as I have written previously — specifics around the training data used are increasingly being kept secret.
“It’s a frustrating dynamic,” said Rumman Chowdhury, former ethics lead at Twitter and now co-founder of nonprofit Humane Intelligence, another of the contest’s organizers. Fuller transparency is difficult for companies trying to protect intellectual property, trade secrets and personal data, she said.
But this is a healthy start. At her laptop, Hayes has not managed to make the chatbot share credit-card information. “Oh, this one’s good,” she says of the bot, as it foils a technique that had been successful in the past. Within chatbots, and broader AI, there are an uncountable number of quirks and exploits still waiting to be found. We should be grateful to the people taking the time to look for them.
Dave Lee is Bloomberg Opinion’s US technology columnist. Previously, he was a San Francisco-based correspondent at the Financial Times and BBC News.
This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.
In their recent op-ed “Trump Should Rein In Taiwan” in Foreign Policy magazine, Christopher Chivvis and Stephen Wertheim argued that the US should pressure President William Lai (賴清德) to “tone it down” to de-escalate tensions in the Taiwan Strait — as if Taiwan’s words are more of a threat to peace than Beijing’s actions. It is an old argument dressed up in new concern: that Washington must rein in Taipei to avoid war. However, this narrative gets it backward. Taiwan is not the problem; China is. Calls for a so-called “grand bargain” with Beijing — where the US pressures Taiwan into concessions
The term “assassin’s mace” originates from Chinese folklore, describing a concealed weapon used by a weaker hero to defeat a stronger adversary with an unexpected strike. In more general military parlance, the concept refers to an asymmetric capability that targets a critical vulnerability of an adversary. China has found its modern equivalent of the assassin’s mace with its high-altitude electromagnetic pulse (HEMP) weapons, which are nuclear warheads detonated at a high altitude, emitting intense electromagnetic radiation capable of disabling and destroying electronics. An assassin’s mace weapon possesses two essential characteristics: strategic surprise and the ability to neutralize a core dependency.
Chinese President and Chinese Communist Party (CCP) Chairman Xi Jinping (習近平) said in a politburo speech late last month that his party must protect the “bottom line” to prevent systemic threats. The tone of his address was grave, revealing deep anxieties about China’s current state of affairs. Essentially, what he worries most about is systemic threats to China’s normal development as a country. The US-China trade war has turned white hot: China’s export orders have plummeted, Chinese firms and enterprises are shutting up shop, and local debt risks are mounting daily, causing China’s economy to flag externally and hemorrhage internally. China’s
During the “426 rally” organized by the Chinese Nationalist Party (KMT) and the Taiwan People’s Party under the slogan “fight green communism, resist dictatorship,” leaders from the two opposition parties framed it as a battle against an allegedly authoritarian administration led by President William Lai (賴清德). While criticism of the government can be a healthy expression of a vibrant, pluralistic society, and protests are quite common in Taiwan, the discourse of the 426 rally nonetheless betrayed troubling signs of collective amnesia. Specifically, the KMT, which imposed 38 years of martial law in Taiwan from 1949 to 1987, has never fully faced its
RSS