An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
ANGER: A video shared online showed residents in a neighborhood confronting the national security minister, attempting to drag her toward floodwaters Argentina’s port city of Bahia Blanca has been “destroyed” after being pummeled by a year’s worth of rain in a matter of hours, killing 13 and driving hundreds from their homes, authorities said on Saturday. Two young girls — reportedly aged four and one — were missing after possibly being swept away by floodwaters in the wake of Friday’s storm. The deluge left hospital rooms underwater, turned neighborhoods into islands and cut electricity to swaths of the city. Argentine Minister of National Security Patricia Bullrich said Bahia Blanca was “destroyed.” The death toll rose to 13 on Saturday, up from 10 on Friday, authorities
OPTIMISTIC: A Philippine Air Force spokeswoman said the military believed the crew were safe and were hopeful that they and the jet would be recovered A Philippine Air Force FA-50 jet and its two-person crew are missing after flying in support of ground forces fighting communist rebels in the southern Mindanao region, a military official said yesterday. Philippine Air Force spokeswoman Colonel Consuelo Castillo said the jet was flying “over land” on the way to its target area when it went missing during a “tactical night operation in support of our ground troops.” While she declined to provide mission specifics, Philippine Army spokesman Colonel Louie Dema-ala confirmed that the missing FA-50 was part of a squadron sent “to provide air support” to troops fighting communist rebels in
Two daughters of an Argentine mountaineer who died on an icy peak 40 years ago have retrieved his backpack from the spot — finding camera film inside that allowed them a glimpse of some of his final experiences. Guillermo Vieiro was 44 when he died in 1985 — as did his climbing partner — while descending Argentina’s Tupungato lava dome, one of the highest peaks in the Americas. Last year, his backpack was spotted on a slope by mountaineer Gabriela Cavallaro, who examined it and contacted Vieiro’s daughters Guadalupe, 40, and Azul, 44. Last month, the three set out with four other guides
Local officials from Russia’s ruling party have caused controversy by presenting mothers of soldiers killed in Ukraine with gifts of meat grinders, an appliance widely used to describe Russia’s brutal tactics on the front line. The United Russia party in the northern Murmansk region posted photographs on social media showing officials smiling as they visited bereaved mothers with gifts of flowers and boxed meat grinders for International Women’s Day on Saturday, which is widely celebrated in Russia. The post included a message thanking the “dear moms” for their “strength of spirit and the love you put into bringing up your sons.” It
RSS