eval(verbatim): expand to n=15 dense hex recall on Fable 5 - 13/15 vs Opus 0/15, both misses single-glyph confabulations

FINDINGS.md

@@ -82,6 +82,16 @@ documented failure mode, a single-glyph silent misread (`125f9e6e1c77` →
 `125f9e6a1c77`); one passing trial also misread an adjacent field
 (`cc33ae67` → `cc33a867`). The verbatim-risk guard remains required.
 
+*Expansion (2026-06-11, n=15 to match the Opus haystack count):* 5 fresh
+dense-JSON pages at the production 5×8 cell (40 lines/page, ~4.7k chars),
+3 needles per page, one-shot `claude -p` per trial, exact-match: **13/15**
+(vs Opus **0/15**). Both misses are single-glyph confabulations on visually
+adjacent hex digits — `5a7373d4187f` → `5a7973d4107f` (3→9, 8→0) and
+`b8fce698f971` → `b0fce698f971` (8→0) — plausible-looking, no error signal.
+Same conclusion at 4× the sample: Fable reads dense renders near-verbatim,
+but ~13% silent-misread on exact strings means the rule stands — anything
+that must round-trip byte-exact stays text.
+
 **4. Economics.** Fable is $10/$50 per MTok (2× Opus 4.8). Token-for-token
 savings are identical (same tokenizer), so every saved token is worth 2× the
 dollars — pxpipe is more valuable on Fable in absolute terms.

README.md

@@ -44,7 +44,7 @@ text; only older bulk history is imaged.
 
 **It is lossy.** pxpipe is a *gist* tier, not a lossless store. In a
 needle-in-haystack eval, exact 12-char hex strings inside dense imaged content
-came back **0/15** on Opus and 3/4 on Fable 5, and the failure mode is
+came back **0/15** on Opus and 13/15 on Fable 5, and the failure mode is
 *silent confabulation*: a plausible wrong value, not an error. Anything you
 need back byte-exact (IDs, hashes, secrets, exact numbers) must stay text.
 Recent turns do; a dedicated verbatim-risk guard is not built yet.
@@ -79,7 +79,7 @@ Measured with novel random-number problems the model cannot have memorized:
 | state tracking (value mutated 3x, final/first/count), Fable 5 | 18/arm | 18/18 | **18/18** | - |
 | confabulation on never-stated facts (lower is better), Fable 5 | 16/arm | 0/16 | **0/16** | - |
 | verbatim 12-char hex recall, dense render, Opus | 15 | 15/15 | **0/15** | - |
-| verbatim 12-char hex recall, dense render, Fable 5 | 4 | - | 3/4 | - |
+| verbatim 12-char hex recall, dense render, Fable 5 | 15 | - | **13/15** | - |
 
 ### SWE-bench Lite pilot (end-to-end task quality)
 

eval/verbatim-15/golds.json

@@ -0,0 +1,77 @@
+[
+ {
+  "page": 0,
+  "dur": 4439,
+  "gold": "c9c947f680ec"
+ },
+ {
+  "page": 0,
+  "dur": 812,
+  "gold": "851eb3af1bd1"
+ },
+ {
+  "page": 0,
+  "dur": 6150,
+  "gold": "ade34f70fd73"
+ },
+ {
+  "page": 1,
+  "dur": 7978,
+  "gold": "c5d68855f46d"
+ },
+ {
+  "page": 1,
+  "dur": 8071,
+  "gold": "92abade01aad"
+ },
+ {
+  "page": 1,
+  "dur": 3309,
+  "gold": "ffe21785b09d"
+ },
+ {
+  "page": 2,
+  "dur": 7215,
+  "gold": "87cb51eb0e99"
+ },
+ {
+  "page": 2,
+  "dur": 4397,
+  "gold": "93c3ced96dac"
+ },
+ {
+  "page": 2,
+  "dur": 4495,
+  "gold": "f152ae9bfb8f"
+ },
+ {
+  "page": 3,
+  "dur": 1622,
+  "gold": "5a7373d4187f"
+ },
+ {
+  "page": 3,
+  "dur": 2025,
+  "gold": "44ea8c7aeedd"
+ },
+ {
+  "page": 3,
+  "dur": 6533,
+  "gold": "8145b5a0fd46"
+ },
+ {
+  "page": 4,
+  "dur": 2921,
+  "gold": "b8fce698f971"
+ },
+ {
+  "page": 4,
+  "dur": 8475,
+  "gold": "4a8164556b99"
+ },
+ {
+  "page": 4,
+  "dur": 8799,
+  "gold": "e53112c4b5a4"
+ }
+]

eval/verbatim-15/run.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+# 25 verbatim hex-recall trials, parallelism 3
+cd /tmp/verb25
+python3 -c "
+import json
+for i, g in enumerate(json.load(open('golds.json'))):
+    print(i, g['page'], g['dur'], g['gold'])
+" > trials.txt
+run_one() {
+  local i=$1 page=$2 dur=$3 gold=$4
+  local ans
+  ans=$(~/.claude/local/claude -p --model claude-fable-5 "Read the image at /tmp/verb25/page${page}.png. Find the JSON line whose dur_ms is exactly ${dur} and report ONLY its 'id' field value (12 hex chars), nothing else. Read it visually from the image; do not use code." 2>/dev/null | tr -d '[:space:]' | grep -oE '[0-9a-f]{12}' | head -1)
+  if [ "$ans" = "$gold" ]; then
+    echo "HIT  trial=$i page=$page dur=$dur gold=$gold"
+  else
+    echo "MISS trial=$i page=$page dur=$dur gold=$gold got=${ans:-EMPTY}"
+  fi
+}
+export -f run_one
+cat trials.txt | xargs -P 3 -L 1 bash -c 'run_one "$@"' _
+echo "=== done ==="