An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
Women’s accessories sold by some of the world’s most popular online shopping firms contained toxic substances sometimes hundreds of times above acceptable levels, authorities in Seoul said yesterday. Chinese giants including Shein, Temu and AliExpress have skyrocketed in popularity around the world in the past few years, offering a vast selection of trendy clothes and accessories at low prices. The explosive growth has led to increased scrutiny of their business practices and safety standards, including in the EU and South Korea, where Seoul officials have been conducting weekly inspections of items sold by online platforms. In the most recent inspection, 144 products from
Turning heads as they cruise past office buildings and malls, driverless taxis are slowly spreading through Chinese cities, prompting both wariness and wonder. China’s tech companies and vehicle manufacturers have poured billions of dollars into self-driving technology over the past few years in an effort to catch industry leaders in the US. Now the central city of Wuhan boasts one of the world’s largest networks of self-driving cars, home to a fleet of more than 500 taxis that can be hailed on an app just like regular rides. At one intersection in an industrial area of Wuhan, AFP reporters saw at least five
China and Vietnam yesterday inked 14 documents spanning cross-border railways to crocodile exports after Chinese President Xi Jinping (習近平) met with new Vietnamese leader To Lam in Beijing. The Vietnamese president’s visit to Beijing, his first overseas trip since becoming the general secretary of the Communist Party of Vietnam earlier this month, signals a desire between the two communist neighbors to strengthen ties, amid growing trade and investment, despite occasional clashes over boundaries in the South China Sea. “China has always regarded Vietnam as a priority in its neighborhood diplomacy, and supports Vietnam in adhering to the party leadership, taking the socialist
A former Saudi Arabian official alleged in a report that the kingdom’s Crown Prince Mohammad bin Salman forged the signature of his father on the royal decree that launched the kingdom’s years-long, stalemated war against Yemen’s Houthi rebels. Saudi Arabia did not immediately respond to a request for comment over the allegations made without supporting evidence by Saad al-Jabri in an interview published yesterday by the BBC, but the kingdom has described him as “a discredited former government official.” Al-Jabri, a former Saudi Arabian intelligence official who lives in exile in Canada, has been a years-long dispute with the kingdom as