id: PIR-2026-0010title: NYC government's MyCity chatbot hallucinates legal guidance under official city branding - take workers' tips, refuse Section 8, lock out tenants - and stays online for years afterwarddate_occurred: 2023-10 (launch) onward; harmful outputs reproduced by journalists 2024-03date_detected: 2024-03 (The Markup / THE CITY testing)date_disclosed: 2024-03-29 (The Markup investigation published)status: corroborated (outputs independently reproduced by multiple journalists; city acknowledged errors)agent_description: MyCity chatbot, NYC government's official small-business assistant built on Microsoft Azure AI, answering regulatory and compliance questions on the city's website. BOUNDARY case: informational only, no transaction execution - harm via citizen reliance on official-branded guidance.operator_type: enterprise (municipal government)autonomy_level: autonomous-within-policy for its narrow function (answered the public directly, no human review of outputs)model_stack: Microsoft Azure AI (exact model undisclosed)harness: city web portal chat interfaceauthority_scope: external comms (official-city-branded legal guidance to businesses, employers, landlords)funds_at_risk_usd: unknown (fines/liability exposure transferred to citizens who acted on the advice)blast_radius: customers/third parties (public-facing; any NYC business owner or landlord)root_cause: plain-error (primary; hallucinated legal guidance); contributing operator-error (deployed on high-stakes legal topics without grounding or review, and kept online after the errors were publicly documented)failure_locus: agent-reasoning (operator-config contributing)exploitation_status: in-wild-malfunction (wrong advice served to real users with no adversary; downstream harm unquantified)mechanism: The bot answered compliance questions with confident falsehoods under official branding: that employers may take a cut of workers' tips (illegal under NY labor law), may fire workers for complaining about sexual harassment (illegal retaliation), that landlords may refuse Section 8 voucher holders (illegal income discrimination) and may lock out tenants (illegal after 30 days' residence), and that businesses may refuse cash (violates city law). Answers varied between identical queries. The city added a disclaimer and kept the bot online - Mayor Adams defended it as a "once-in-a-generation opportunity." The incoming Mamdani administration announced it would kill the bot (transition Jan 2026).adversary_present: no (journalists asked plain questions; no jailbreaking involved)severity: degradeddirect_loss_usd: unknown (program cost sunk; no documented case of a business penalized for following the advice - the named quantification gap)indirect_loss_usd: unknown (legal exposure transferred to citizens who relied on it)downtime: n/a - the failure mode was staying onlinedata_exposure: nonedetected_by: third-party (journalists at The Markup and THE CITY)time_to_detect: ~5 months (launch to public documentation)time_to_recover: ~2 years (kept online with disclaimer; shutdown announced by the incoming administration for the Jan 2026 transition)remediation: disclaimer added; errors acknowledged; bot retainedstructural_fix: none during the deploying administration; termination by its successor is the fixcontrols_that_worked: none internal; independent press reproduction was the only functioning checkliability_holder: unresolved - citizens acting on the advice bear the regulatory exposure; the city disclaimed rather than assumed responsibility (contrast PIR-2026-0006, where the operator was held liable)precedent_set: none adjudicatedsealed_material: notelemetry_grade: witnessed (outputs independently reproduced and published by unaffiliated journalists - stronger than operator custody)sources:Independence: strong - journalist reproduction plus city acknowledgment.
- confidence: high on outputs and timeline; low on downstream harm quantification (the named weakest link)