Would you put a heart warning on this pain pill?

What the agent made

Ran it for real with Claude Code 2.1.300 and OMP 18.11.0 on 2026-10-15. The first session started with no saved memory; each later session was a new one that kept only the files in the folder. Your result will differ.

Open what Claude Code made, full screen

Then a brand-new session used what it saved on new input: see the proof.

The same prompts in OMP made their own versions, every session recorded and checked: session 1 · Open the later record.

Make your own

1. Get the files

   Download the practice folder (ZIP)

   Unzip it: you get a folder called cli-practice with 3 practice files. A minimally quoted 2000 study abstract and a February 1, 2001 FDA cardiovascular-review excerpt plus source links. Later withdrawal and editorial materials remain outside this folder until the reveal. It also has a for-later folder with the new files for the fresh sessions.

   SOURCES.csvfda-review.txttrial.txt

2. Open your agent in that folder

   Open a terminal in cli-practice and type claude or omp --approval-mode always-ask. First time? Set up in about five minutes, with steps for Mac, Windows, and Linux.

3. Paste this prompt

   Work inside this practice folder using only trial.txt, fda-review.txt, and SOURCES.csv. Do not read outside the folder, access a network, install packages, ask for a reply, or modify these inputs. Python 3 standard library only. Do not use em dashes or en dashes. The date is February 8, 2001; do not look up or imply any later outcome.
   
   You are advising a hospital formulary committee about whether the published gastrointestinal benefit warrants ignoring or explaining away the myocardial-infarction comparison. Make a precise dated call on whether the label should disclose a cardiovascular warning. Compare the article's reported gastrointestinal rates and cardiac comparison with the FDA's exact event counts and denominators. Calculate arm-specific crude myocardial-infarction risks, rofecoxib-to-naproxen crude risk ratio, and excess events per thousand participants; distinguish those participant risks from the paper's gastrointestinal rates per patient-year. Audit the FDA table's other cardiovascular event row, but do not add overlapping rows. Ask whether a naproxen-protection explanation is established, merely plausible, or impossible to distinguish here: a two-arm comparison without a placebo does not identify absolute drug effect. Avoid claiming this study proves causality, a safe alternative, or a population-wide number of harmed people.
   
   Write only audit.py, analysis.json, check.py and analysis.html. audit.py must parse source files afresh, compute and cite the quantities, preserve denominators and distinct units, state the recommendation and unknown causal counterfactual. check.py independently parses the original inputs without importing or invoking audit.py, compares decision-critical JSON values and their tagged counterparts in HTML, checks source IDs and absence of later outcomes in the blind page, and rejects a ratio with reversed arms or a claimed placebo comparison. Its --self-test must catch at least three distinct in-memory corruptions of material values. Run audit.py, check.py and check.py --self-test and report exit codes.
   
   Design analysis.html as a self-contained, offline formulary evidence sheet, not a generic dashboard. Lead with the policy call and a concise sourced cardiac-versus-stomach comparison. Include an inline SVG zero-baseline bar chart comparing crude myocardial-infarction events per 1000 randomized people between the two arms, with direct labels, units, exact counts and linked VX2; keep the separate gastrointestinal rates with VX1. Every decision-critical number, source, recommendation and unknown must also be ordinary semantic HTML readable with JavaScript off. Include a useful, clearly labeled touch-and-keyboard interaction that opens the cited FDA reviewer reasoning about the alternative explanation and changes visible evidence, not merely color. Keep a source ledger with working publisher links. At 360 pixels wide and wider, including tablet widths from 600 to 1000 pixels, the page fits the screen and nothing scrolls sideways, not even inside a table, chart, timeline or code box; charts and timelines scale to the width; no word, field name or amount ever breaks mid-word or runs past the edge, so never print long code identifiers or URLs as page text; narrow tables become stacked cards. Respect reduced motion. No CDN, external asset, remote request, fake verification badge, or medical advice beyond this historical evidence exercise.

   The prompt asks the agent to stay in this folder; it cannot enforce that. Read each file change and command before you approve it.

4. Check it yourself

   Open cli-practice/analysis.html in your web browser (double-click it), then:

   Human check: VX2 FDA memo dated February 1, 2001: myocardial infarction 20/4047 rofecoxib versus 4/4029 naproxen. Crude risks 0.4942% versus 0.0993%; rofecoxib/naproxen crude risk ratio 4.9778 and excess 3.9491 per 1000 participants. The overlapping all-event row is 47 versus 20, not added to infarctions. VX1 reports 2.1 versus 4.5 GI events per 100 patient-years, a different denominator and endpoint. No placebo arm can distinguish rofecoxib harm from naproxen protection by itself; reviewer recommends cardiovascular disclosure and a warning discussion. Check absent hindsight, parsed figures, independent checker and three corruptions.

   When we checked the recording above: Re-recorded with the 360-and-tablet layout prompt. Clean-folder rebuild: lesson checks and self-tests exit 0, no inputs changed; key figures match human_check; sweep at 360/375/390/800/1024/1280 after every control shows no overflow, inner scroller or printed URL; resize 400-1300 in 4px steps fits.

Then prove it: fresh sessions

Each run below is a brand-new session, recorded for real in Claude Code. It remembers nothing from the run before except the files saved in the folder, so the short prompt is the whole instruction.

Work only in this practice folder. First read the original trial.txt, fda-review.txt, SOURCES.csv and existing output files. Then read the newly added outcomes.txt and OUTCOME_SOURCES.csv. Do not alter any inputs or prior outputs, read outside the folder, use the network, install packages, or ask for a reply. Python 3 standard library only. Do not use em dashes or en dashes. This is a fresh historical session: later findings must never be described as knowable on February 8, 2001.

Write only reveal.json, reveal.html, reveal-check.py. Build a self-contained offline page with two dated evidence columns: the formulary decision knowable from the trial and FDA review, versus the later company announcement and journal-editor notice. The company's announcement is an outcome involving a separate placebo-controlled study, not proof that the original two-arm trial identified the causal mechanism. Preserve the original FDA denominators and myocardial-infarction counts; carefully compare the journal editors' original manuscript window and later included-event window, without adding the omitted events a second time to the FDA's already updated tally. Distinguish how the editors characterized the omission from your own inferences about any author. Cite each claim to the source ID and dated URL.

Include an inline dated timeline and one native details disclosure that opens a substantive cited excerpt or table difference. Semantic HTML must retain every essential value and the two evidence columns with JavaScript off. At 360 pixels wide and wider, including tablet widths from 600 to 1000 pixels, the page fits the screen and nothing scrolls sideways, not even inside a table, chart, timeline or code box; charts and timelines scale to the width; no word, field name or amount ever breaks mid-word or runs past the edge, so never print long code identifiers or URLs as page text; narrow tables become stacked cards. Self-contained offline, no external assets, network calls, or animations needed.

reveal-check.py independently parses original and added inputs without importing or running audit.py or check.py, recomputes decision-critical numbers, compares reveal.json and tagged values in reveal.html, rejects later facts labeled as known in 2001, and runs --self-test with three distinct in-memory corruptions. Run reveal-check.py and reveal-check.py --self-test and report exit codes.

Do it for real

For a present-day medicine decision, use current regulator labeling, relevant randomized evidence, patient-specific risks, and a licensed clinician or pharmacist. Do not change prescribed treatment based on this historical exercise.

Why an agent, not a chat window

The agent reads a journal abstract against a dated regulator's event table, computes denominators and risk ratios, tests an alternative explanation without claiming causal proof, then writes an independent checker. A separate session confronts the frozen decision with a filed company announcement and the journal editors' later notice.