# PIR-2026-0004 - NEDA's Tessa chatbot gives weight-loss advice to eating-disorder patients after an unapproved generative upgrade

- `id`: PIR-2026-0004
- `title`: Vendor adds generative AI to a rule-based eating-disorder support bot; bot recommends calorie deficits and weekly weigh-ins to the population it was built to protect
- `date_occurred`: first documented harmful outputs October 2022 (screenshots sent to NEDA by MEDA director Monika Ostroff); harmful advice still live late May 2023
- `date_detected`: 2022-10 internally (NEDA had Ostroff's screenshots); public detection 2023-05-29 (activist Sharon Maxwell's posts)
- `date_disclosed`: 2023-05-30 (NEDA announces Tessa taken down, <24h after Maxwell's screenshots)
- `status`: corroborated - details corrected from candidate intake, see Verification notes

### The agent
- `agent_description`: "Tessa," a wellness chatbot offered by the National Eating Disorders Association, originally a closed, rule-based Body Positive program built by academic researchers; operated as a free service by vendor Cass (formerly X2AI), which added generative-AI answer capability in a systems upgrade. Deployed against a clinically vulnerable population just as NEDA wound down its human helpline (unionized staff laid off, effective June 1, 2023).
- `operator_type`: enterprise (nonprofit operator, commercial vendor)
- `autonomy_level`: autonomous-within-policy (unreviewed replies to users; no tool use)
- `model_stack`: unknown (Cass generative layer atop the scripted program; model never disclosed)
- `harness`: Cass chatbot platform

### Authority
- `authority_scope`: external comms only - but to users for whom bad words are clinically dangerous; advice-giving authority over a vulnerable population is the real exposure
- `funds_at_risk_usd`: 0
- `blast_radius`: customers/third parties

### The failure
- `root_cause`: `model-update-regression` - primary; a vendor-side upgrade changed behavior of a validated rule-based program. Contributing: `plain-error` (the generative layer's harmful advice) and `operator-error` (NEDA had no change control or output monitoring over its vendor)
- `failure_locus`: model-provider (vendor Cass, the bot's provider relative to NEDA; note this is a chatbot vendor, not a foundation-model lab)
- `mechanism`: Tessa was built as a closed-script prevention tool. Cass's upgrade let it generate novel answers. To eating-disorder users it then recommended, per screenshots and press accounts: 500-1,000 calorie daily deficits, losing 1-2 lbs per week, regular weigh-ins, and skinfold-caliper body-fat measurement - standard diet-culture advice that clinicians state fuels eating disorders, delivered even after users disclosed their condition. It kept doing so for at least ~7 months after NEDA received the first evidence.
- `adversary_present`: no (Maxwell and Ostroff probed it, but as good-faith testers of a bot they had reason to distrust; the harmful behavior required no manipulation)
- `exploitation_status`: in-wild-malfunction (production system, real vulnerable users exposed, no adversary; v0.2 token per the gap flagged in PIR-2026-0033)

### Impact
- `severity`: degraded
- `direct_loss_usd`: unknown; `indirect_loss_usd`: unknown (clinical risk to an unknown number of users; loss of both the human helpline and its replacement - the organization was left with neither)
- `downtime`: Tessa suspended indefinitely 2023-05-30
- `data_exposure`: none

### Detection and recovery
- `detected_by`: third-party (Ostroff Oct 2022, ignored in effect; Maxwell May 2023, actioned within 24h once public)
- `time_to_detect`: ~0 (evidence in hand Oct 2022); time-to-act ~7 months, compressed to <24h by publicity
- `time_to_recover`: n/a - never restored
- `remediation`: bot taken down "until further notice" for investigation
- `structural_fix`: none - the program ended
- `controls_that_worked`: none identified; the scripted-program boundary that was the original safety design is precisely what the upgrade dissolved

### Evidence
- `telemetry_grade`: none (user screenshots + NEDA/vendor statements; no logs public)
- `sources`:
  - https://www.npr.org/sections/health-shots/2023/06/08/1180838096/an-eating-disorders-chatbot-offered-dieting-advice-raising-fears-about-ai-in-hea (independent; includes the Oct 2022 prior-knowledge reporting)
  - https://www.psychiatrist.com/news/neda-suspends-ai-chatbot-for-giving-harmful-eating-disorder-advice/
  - https://www.nbcnews.com/tech/neda-pulls-chatbot-eating-advice-rcna87231
  - https://fortune.com/well/2023/05/31/neda-ai-chatbot-harmful-advice
  - https://www.bbc.com/news/world-us-canada-65771872 (article confirmed via BBC's own feed mirror at feeds.bbci.co.uk/news/world-us-canada-65771872, same ID and title)
  - `independence`: good - multiple outlets, screenshots from two unrelated testers.
- `aiid_incident_id`: 545 (https://incidentdatabase.ai/cite/545/) - cross-reference; primaries verified independently
- `confidence`: high on outputs and suspension; medium on root cause - "Cass changed Tessa without NEDA's awareness or approval" is NEDA CEO Liz Thompson's claim, and vendor statements partially conflict on whether the upgrade was within contract

### Verification notes
- **Correction (dates)**: candidate stated "harmful outputs late May 2023." Verified reporting shows NEDA received screenshots of harmful Tessa outputs in **October 2022**; the May 2023 events were public exposure, not first occurrence. date_occurred, date_detected, and the ~7-month time-to-act above reflect this. It materially changes the incident's shape: not a fast-detected malfunction but a known issue actioned only under publicity.
- Classification kept as primary model-update-regression per candidate, with operator-error added as contributing for the absent vendor change-control - which the corrected timeline makes load-bearing.

### Corrections

- 2026-08-19: Added `aiid_incident_id` cross-reference (AIID 545), matched against the AIID weekly database export (2026-08-17). A cross-reference, not a re-verification; no claim changed.