An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
Seven people sustained mostly minor injuries in an airplane fire in South Korea, authorities said yesterday, with local media suggesting the blaze might have been caused by a portable battery stored in the overhead bin. The Air Busan plane, an Airbus A321, was set to fly to Hong Kong from Gimhae International Airport in southeastern Busan, but caught fire in the rear section on Tuesday night, the South Korean Ministry of Land, Infrastructure and Transport said. A total of 169 passengers and seven flight attendants and staff were evacuated down inflatable slides, it said. Authorities initially reported three injuries, but revised the number
‘BALD-FACED LIE’: The woman is accused of administering non-prescribed drugs to the one-year-old and filmed the toddler’s distress to solicit donations online A social media influencer accused of filming the torture of her baby to gain money allegedly manufactured symptoms causing the toddler to have brain surgery, a magistrate has heard. The 34-year-old Queensland woman is charged with torturing an infant and posting videos of the little girl online to build a social media following and solicit donations. A decision on her bail application in a Brisbane court was yesterday postponed after the magistrate opted to take more time before making a decision in an effort “not to be overwhelmed” by the nature of allegations “so offensive to right-thinking people.” The Sunshine Coast woman —
BORDER SERVICES: With the US-funded International Rescue Committee telling clinics to shut by tomorrow, Burmese refugees face sudden discharge from Thai hospitals Healthcare centers serving tens of thousands of refugees on the Thai-Myanmar border have been ordered shut after US President Donald Trump froze most foreign aid last week, forcing Thai officials to transport the sickest patients to other facilities. The International Rescue Committee (IRC), which funds the clinics with US support, told the facilities to shut by tomorrow, a local official and two camp committee members said. The IRC did not respond to a request for comment. Trump last week paused development assistance from the US Agency for International Development for 90 days to assess compatibility with his “America First” policy. The freeze has thrown
TESTING BAN: Satellite photos show a facility in the Chinese city of Mianyang that could aid nuclear weapons design and power generation, a US researcher said China appears to be building a large laser-ignited fusion research center in the southwestern city of Mianyang, experts at two analytical organizations said, a development that could aid nuclear weapons design and work exploring power generation. Satellite photos show four outlying “arms” that would house laser bays, and a central experiment bay that would hold a target chamber containing hydrogen isotopes the powerful lasers would fuse together, producing energy, said Decker Eveleth, a researcher at US-based independent research organization CNA Corp. It is a similar layout to the US$3.5 billion US National Ignition Facility (NIF) in northern California, which in 2022 generated
RSS