Is the published preclinical evidence for repurposing approved non-oncology drugs in AML concentrated in a few compounds?
- ground
- plan
- build
- verify
- deliver
- done15 h ago
Finding refuted. Written into the program's graph and articles.
- deliver15 h agoclaude-opus-5-5
I recorded the finding as **refuted** and published the write-up. The refutation is soft: the main estimate falls below 50%, but its uncertainty range still includes 50%. The hypothesis was that the 5 most-studied approved non-cancer drugs account for at least 50% of AML-repurposing papers in Europe PMC from 2015 to 2025. | Corpus | Papers matched to a drug | Top-5 share (95% CI) | |---|---|---| | Strict: AML plus a word like "repurposing" (the pre-registered test) | 33 of 147 | 0.424 (0.39–0.67) | | Loose: AML and a drug named in the title | 159 of 237 | 0.409 (0.35–0.49) | - **Main test:** the strict share is below 50%, but the interval reaches 0.67, so on its own it doesn't settle the question. - **Firmer evidence:** the loose corpus is larger and its interval stays below 50%. It was a pre-specified secondary analysis. - **Long tail:** no single drug covers more than 10–12% of matched papers. The spread is equivalent to about 19–22 equally studied drugs. - **Sensitivity checks:** 9 of 10 versions of the analysis come out below 50%. The one exception (0.505) only crosses the line by counting ATRA, which is approved for a subtype of AML and so falls outside the question. - **Main caveat:** 78% of strict-corpus papers (114 of 147) matched no drug in the 125-drug list, and three drugs tie for fifth place. The drug matching hasn't been checked by hand yet; a 50-paper sample is saved for that. All three verification checks passed, so nothing blocked a verdict. The article ends with the next steps: hand-check the sample and the unmatched papers, add missing drugs to the list, cross-check against PubMed's controlled vocabulary, and re-estimate.
- deliver15 h agoclaude-opus-5-5
→ write_article {"title": "AML drug-repurposing research is spread across many drugs, not concentrated in a few", "subtitle": "The pre-registered hypothesis predicted that 5 drugs would account fo
- deliver15 h agoclaude-opus-5-5
→ record_finding {"verdict": "refuted", "text": "In the primary strict corpus (Europe PMC 2015–2025, AML plus a repurposing term; 147 papers, 33 matched to a dictionary drug), the top 5 compounds a
- deliver15 h agoclaude-opus-5-5
→ ToolSearch {"query": "select:mcp__bistro__record_finding,mcp__bistro__write_article", "max_results": 2}
- deliver15 h agoclaude-opus-5-5
→ Bash {"command": "ls && cat ground.md plan.md build.md | head -400"}
- verify15 h ago
Re-executed 3 claims from the program directory. aml-repurposing-1-c1-corpus: PASS ""; aml-repurposing-1-c2-primary: PASS ""; aml-repurposing-1-c3-sensitivity: PASS ""
- build15 h agoclaude-sonnet-5-5
The pre-registered hypothesis is not supported: the top 5 compounds hold about 41–42% of drug-assigned AML-repurposing papers, below the 50% threshold. The evidence is spread over a long tail of drugs. | Corpus | Papers assigned a drug | Top-5 share (95% CI) | Top 5 (papers) | |---|---|---|---| | Strict (AML + repurposing term, 147 papers) | 33 | 0.424 (0.39–0.67) | chlorpromazine 4, disulfiram 3, mebendazole 3, ascorbic acid 2, pitavastatin 2 | | Loose (AML and a drug both in the title, 237 papers) | 159 | 0.409 (0.35–0.49) | metformin 16, valproate 15, ascorbic acid 13, sirolimus 13, artesunate 8 | - **Concentration:** the single most common compound holds only 10–12% of assigned papers. - **Sensitivity:** 9 of 10 scenarios fall below 0.50. The exception is the loose corpus with ATRA counted (0.505). ATRA is an approved oncology drug, so I don't count that as support. - **Strict corpus caveat:** 77% of its papers matched no dictionary drug, and the top-5 has a three-way tie at rank 5. That result is not decisive on its own, because its CI reaches 0.67. The loose corpus is the firmer evidence. - **Counts only:** this describes publication counts, not efficacy or clinical use. The default `python` has a broken httpx install (missing `charset_normalizer`). I ran `analysis.py` with the project venv's Python (httpx 0.28.1) and changed nothing in the environment. I also added flat output keys to the script's final JSON line so the numbers could be registered; the analysis logic is unchanged. I registered three claims (c1-corpus, c2-primary, c3-sensitivity) with tolerance 0.02. `build.md` has the method, the checks and the caveats. The tables are TSV files, not CSV: `papers_strict.tsv`, `papers_loose.tsv`, `unassigned_strict.tsv` and `precision_sample.tsv`. The full output is in `out.json`. I did not manually check the precision sample.
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "cat > build.md <<'E'\n# E:aml-repurposing-1 — is AML-repurposing evidence concentrated in a few non-oncology drugs?\n\n**Pre-registered hypothesis:** the top 5 candida
- build15 h agoclaude-sonnet-5-5
→ register_claim {"id": "c3-sensitivity", "statement": "Sensitivity S1: with ATRA, class-level, preclinical-only scenarios.", "script": "analysis.py", "expect": {"strict_plus_atra_S1": 0.4444, "loo
- build15 h agoclaude-sonnet-5-5
→ register_claim {"id": "c2-primary", "statement": "Primary top-5 share of drug-assigned papers (S1): strict 0.4242, loose 0.4088; top-1 share and effective number of compounds.", "script": "analys
- build15 h agoclaude-sonnet-5-5
→ register_claim {"id": "c1-corpus", "statement": "Europe PMC counts 2015-2025: all AML records; strict AML+repurposing query hits; strict papers kept after filters; loose (drug-in-title) papers ke
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "V=/Users/igor/Work/me/bistro-research/backend/.venv/bin/python\n$V - <<'E'\ns=open('analysis.py').read()\nadd=''' for nm in (\"strict\", \"loose\"):\n for sc
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "V=/Users/igor/Work/me/bistro-research/backend/.venv/bin/python; $V -c \"\nimport json;d=json.loads(open('out.json').read().splitlines()[-1])\nfor k,v in d.items():\n
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "V=/Users/igor/Work/me/bistro-research/backend/.venv/bin/python; $V -c \"import httpx;print(httpx.__version__)\" && ($V analysis.py > out.json 2> err.txt; echo rc=$?; t
- build15 h agoclaude-sonnet-5-5
The project venv probably has a working httpx. I'll try that.
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "which -a python python3; ls scripts; python3 -c \"import sys;print(sys.version)\"; pip --version"}
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "python analysis.py > out.json 2> err.txt; echo rc=$?; cat err.txt | tail -30; tail -c 600 out.json", "timeout": 300000}
- build15 h agoclaude-sonnet-5-5
→ ToolSearch {"query": "select:mcp__bistro__register_claim", "max_results": 1}
- build15 h agoclaude-sonnet-5-5
→ Bash {"command": "ls -la; cat analysis.py"}
- plan15 h agoclaude-opus-5-5
I've written plan.md and analysis.py. analysis.py has not been run yet. It compiles, and a quick offline check of the drug-name matching worked, including "hydroxychloroquine" not also counting as chloroquine. **How the test works (plan.md)** - **Data:** Europe PMC, 2015–2025, preprints excluded. - **Main paper set:** papers that name AML in the title or abstract and use repurposing language (repurpos\*, reposition\* or "old drug(s)", in title, abstract or keywords). A probe of the query returned 141 hits; ground.md's 147 included preprints and used slightly different terms. - **Second paper set:** papers with both AML and a listed drug in the title. This catches papers that test an old drug without saying "repurposing". Supportive-care papers (antifungal prophylaxis, nausea, GVHD and similar) are dropped. - **Which drugs count:** a fixed list of 125 approved drugs with no cancer-drug ATC code (L01/L02). Cancer-approved drugs are excluded by name: ATRA, arsenic, lenalidomide, thalidomide, celecoxib and others. Each drug is counted separately (simvastatin and lovastatin are two candidates, not "statins"). - **Main metric:** among papers that name at least one listed drug, the share that name at least one of the 5 most-studied drugs. Two other measures are also reported: - the same share with every paper in the denominator, including reviews and papers naming no listed drug (a lower bound); - the top 5's share of all drug mentions. - **Uncertainty:** a 95% interval from 2,000 resamples, with the top 5 re-picked in each resample. Picking the top 5 from the same data otherwise inflates the share. - **Verdict rule:** supported if the main share is at least 0.50 in the main set, refuted if below. If the interval contains 0.50, it is reported as "not decisive". - **Checks that cannot change the verdict:** adding ATRA; removing borderline drugs (steroids, valproate, vitamins); grouping by drug class; keeping only lab-based papers (no reviews). Each is run on both paper sets. If they disagree with the main result, it is reported as depending on the definition. - **Two biases handled:** doxycycline and tetracycline are not counted in papers that use them only to switch genes on in lab cell systems. A fixed random sample of 50 drug-assigned papers is written out for manual precision checking later. **analysis.py** - Runs with no arguments, on the standard library plus httpx. - Makes at most 4 calls per second and stops at 170 s. I estimate about 10 API calls. - The last stdout line is one JSON object: set sizes, the top 5 with paper counts, all three shares, the intervals, the other concentration measures, the verdict and the run metadata. A readable summary goes to stderr. - It also writes `papers_strict.tsv`, `papers_loose.tsv`, `unassigned_strict.tsv` and `precision_sample.tsv`. **Limitations** - I curated the drug list by hand. Any approved drug missing from it makes the main share look more concentrated than it is. The all-papers share and the count of papers with no listed drug are reported to show how big that effect is. - Europe PMC's MeSH search gave implausibly low counts, so it is not used.
- plan15 h agoclaude-opus-5-5
→ Bash {"command": "python3 -m py_compile analysis.py && python3 -I -c \"\nimport importlib.util,sys\ns=importlib.util.spec_from_file_location('a','analysis.py');a=importlib.util.module_f
- plan15 h agoclaude-opus-5-5
→ Write {"file_path": "/Users/igor/Work/me/bistro-research/state/runs/1/analysis.py", "content": "#!/usr/bin/env python3\n\"\"\"E:aml-repurposing-1 - test: top 5 candidates account for >=5
- plan16 h agoclaude-opus-5-5
plan.md is written. Now I'm writing analysis.py.
- plan16 h agoclaude-opus-5-5
→ Write {"file_path": "/Users/igor/Work/me/bistro-research/state/runs/1/plan.md", "content": "# Plan: E:aml-repurposing-1 — test of the pre-registered hypothesis\n\n**Question (pre-registe
- plan16 h agoclaude-opus-5-5
→ Bash {"command": "cd /tmp && python3 - <<'EOF'\nimport httpx,time\nB=\"https://www.ebi.ac.uk/europepmc/webservices/rest/search\"\nfor q in ['MESH:\"Drug Repositioning\" AND PUB_YEAR:[20
- plan16 h agoclaude-opus-5-5
→ Bash {"command": "cd /tmp && python3 - <<'EOF'\nimport httpx,time\nB=\"https://www.ebi.ac.uk/europepmc/webservices/rest/search\"\naml='(TITLE_ABS:\"acute myeloid leukemia\" OR TITLE_ABS
- plan16 h agoclaude-opus-5-5
→ Bash {"command": "cat run_state.json; ls scripts; cat scripts/counts.py; head -50 scripts/counts_out.txt; python3 -c \"import httpx; print(httpx.__version__)\"", "description": "Read ru
- plan16 h agoclaude-opus-5-5
→ Bash {"command": "ls -la && cat ground.md", "description": "List directory and read ground.md"}
- ground16 h agoclaude-sonnet-5-5
I've written `ground.md` (1,137 words) in the run directory, with the counting script `scripts/counts.py` and its output `scripts/counts_out.txt`. The file changed on disk after my last edit, and I haven't re-read it. I'm treating the on-disk version as current. - **Hypothesis not yet tested:** this survey doesn't test the hypothesis. The reviews I read don't say whether the evidence is concentrated. - **Narrow definition:** Europe PMC has 37,518 AML papers for 2015–2025. Only 147 of them use a repurposing term in the title or abstract. That definition misses papers that never use the word. - **Preliminary signal:** per-drug counts within those 147 are small and spread across many compounds. The listed drugs add up to 28 hits, before removing overlaps. This points against strong concentration, but it is keyword counting, not a test. - **Definition risk:** what counts as "non-oncology" can swing the result. All-trans retinoic acid (ATRA, approved in a leukaemia subtype) has 396 AML hits. - **Other caveats:** the file also covers dose and exposure mismatch, cell-line and xenograft models, and the fact that no repurposed non-cancer drug has entered standard AML care. - **References:** it lists 14, and proposes Europe PMC, OpenAlex, PubMed/MeSH and ATC drug classification as the sources for the real test. - **Not used:** I didn't use `gemini_read` for synthesis. I worked from the abstracts the search returned and haven't read any full texts.
- ground16 h agoclaude-sonnet-5-5
→ Bash {"command": "python3 -I - <<'EOF'\ns=open('ground.md').read()\ns=s.replace(\"about 25 hits\",\"28 hits, before removing overlaps\")\ns=s.replace(\"The 2020 reviews describe a wide
- ground16 h agoclaude-sonnet-5-5
→ Write {"file_path": "/Users/igor/Work/me/bistro-research/state/runs/1/ground.md", "content": "# Ground: AML repurposing literature (E:aml-repurposing-1)\n\nQuestion (pre-registered): Is
- ground16 h agoclaude-sonnet-5-5
→ Bash {"command": "mkdir -p scripts && cat > scripts/counts.py <<'EOF'\nimport json,urllib.request,urllib.parse\ndef n(q):\n u=\"https://www.ebi.ac.uk/europepmc/webservices/rest/searc
- ground16 h agoclaude-sonnet-5-5
→ literature_search {"source": "openalex", "query": "repurposing non-oncology drugs acute myeloid leukemia review", "n": 20}
- ground16 h agoclaude-sonnet-5-5
→ literature_search {"source": "europepmc", "query": "repurposed approved drug AML cells preclinical (metformin OR statin OR disulfiram OR itraconazole OR niclosamide OR artesunate)", "n": 25}
- ground16 h agoclaude-sonnet-5-5
→ literature_search {"source": "europepmc", "query": "drug repurposing acute myeloid leukemia", "n": 25}
- ground16 h agoclaude-sonnet-5-5
→ ToolSearch {"query": "select:mcp__bistro__literature_search,mcp__bistro__gemini_read", "max_results": 5}