demo: pivot to smart-zone demo (plain Claude rots at large context, pxpipe stays sharp)

- generate.mjs builds one large context (filler logs + small state-tracking needle)
- README: two-column run (plain vs pxpipe), same prompt, compare answers + /context
- drop the e-commerce /context demo; context/ is generated, gitignored

demo/.gitignore

@@ -0,0 +1,2 @@
+# Generated by generate.mjs (deterministic, regenerable — don't commit the bulk)
+context/

demo/EXPECTED.md

@@ -1,37 +0,0 @@
-# Expected answer (ground truth)
-
-Computed deterministically by `generate.mjs`. Both demo columns should land here.
-
-## Answer
-| field | value |
-|---|---|
-| SKU | `SKU-0008` |
-| product name | Titanium Node |
-| net revenue (2025, recognized) | ~$71,520 |
-| top customer | Nora Moreau (`CUST-020`) |
-| that customer's spend on it | ~$4,320 |
-
-Margin over the #2 product: **$61,583.99** — a wide,
-unambiguous gap, so the *ranking* survives a stray glyph misread even if a
-dollar total is off by a little. Grade on the identity (SKU/name/customer),
-treat the dollar figure as approximate.
-
-## Why it can't be shortcut
-- **Year filter:** orders span 2024–2026; only 2025 counts (note the file names).
-- **Status filter:** only `fulfilled`+`delivered` count — a model that assumes
-  "fulfilled only" gets a different number, proving it read RETURNS_POLICY.md.
-- **Three-way join:** revenue from the order files, name from products.json,
-  top-customer name from customers.json.
-
-## File sizes (each well under Read's ~25k-token page cap)
-| file | chars | ~tokens |
-|---|---:|---:|
-| products.json | 5,282 | 2,765 |
-| customers.json | 8,035 | 4,207 |
-| orders-2024.json | 21,980 | 11,508 |
-| orders-2025-h1.json | 32,230 | 16,874 |
-| orders-2025-h2.json | 32,250 | 16,885 |
-| orders-2026.json | 20,967 | 10,977 |
-
-Real per-request savings on this content: run `node demo/preflight.mjs`.
-Your historical real savings: run `node demo/analyze-events.mjs`.

demo/README.md

@@ -1,50 +1,50 @@
-# pxpipe side-by-side demo
+# pxpipe smart-zone demo
 
-A self-contained data-retrieval task for the two-column demo (plain Claude Code
-vs Claude Code through pxpipe). Requires **Fable 5** — pxpipe only compresses
-`claude-fable-5`, so both columns must run Fable, and there is nothing to show
-while Fable is offline.
+**Shows that pxpipe keeps Claude *sharp* at large context.** Same task, two
+columns: the plain column drowns in a huge context and gets the answer wrong
+(the "dumb zone"); the pxpipe column images the bulk, keeps a small active
+context, and answers correctly.
 
-## Files
+Both columns must be **Fable 5** (pxpipe only compresses `claude-fable-5`).
 
-- `data/*.json` — a small "exported database" (products, customers, orders
-  split by year). **Compact JSON on purpose** so reads image cleanly; each file
-  fits in one `Read` page.
-- `RETURNS_POLICY.md` — revenue rules the task must apply (makes naive shortcuts wrong).
-- `EXPECTED.md` — the deterministic ground-truth answer.
-- `generate.mjs` — regenerates the data (`node demo/generate.mjs`).
-
-## The prompt (identical in both columns)
-
-> Read all the JSON files in `data/` and `RETURNS_POLICY.md`. Using only those
-> files (don't write or run code), which product generated the most net revenue
-> in 2025? Apply `RETURNS_POLICY.md` exactly. Give the SKU, product name,
-> approximate net revenue, and the single customer who spent the most on that
-> product.
-
-Expected: **SKU-0008 "Titanium Node"** (see `EXPECTED.md`). Both columns should
-get the same answer; the right column should use far less context (`/context`).
-
-## Running it (when Fable is back) — 2 columns
-
-Start the proxy once in the background (it's a service, not a column):
+---
 
+## 1. Generate the context (prints the prompt + expected answer)
 ```bash
-npx pxpipe-proxy >/tmp/pxpipe.log 2>&1 &           # 127.0.0.1:47821
+node demo/generate.mjs
 ```
 
-Then the two side-by-side Claude windows, both in this dir:
+## 2. Start the proxy (background, once)
+```bash
+npx pxpipe-proxy >/tmp/pxpipe.log 2>&1 &        # 127.0.0.1:47821
+```
 
+## 3. LEFT column — plain Claude (no proxy)
 ```bash
 cd /Users/user/Downloads/repos/pixelpipe/demo
-
-# LEFT column — plain
-claude                                             # then: /model claude-fable-5
-
-# RIGHT column — through pxpipe
-ANTHROPIC_BASE_URL=http://localhost:47821 claude   # then: /model claude-fable-5
+claude
 ```
+then: `/model claude-fable-5`
 
-In both: paste the prompt, let it finish, then run `/context` and compare.
-Same answer (SKU-0008), far less context on the right. Dashboard with the
-per-request token delta: <http://127.0.0.1:47821/>.
+## 4. RIGHT column — through pxpipe
+```bash
+cd /Users/user/Downloads/repos/pixelpipe/demo
+ANTHROPIC_BASE_URL=http://localhost:47821 claude      # or: pp
+```
+then: `/model claude-fable-5`
+
+## 5. Paste the prompt (from step 1's output) in BOTH columns
+It tells each to read every file in `context/` and answer the ZX-9 question with
+only the integer.
+
+## 6. The payoff
+- **RIGHT (pxpipe)** → answers the correct integer (it kept the needle readable
+  and the active context small).
+- **LEFT (plain)** → at a large enough context, gets it **wrong** — drowned in
+  the filler. That gap *is* the demo.
+- Also run **`/context`** in each: the right column is a fraction of the left.
+
+> If the LEFT column still answers correctly, the context isn't big enough to
+> push Fable into the dumb zone — raise `SIZE` in `generate.mjs`, regenerate, and
+> retry until plain Claude breaks. (If it never breaks, the win is the `/context`
+> reduction itself — report that honestly.)

demo/RETURNS_POLICY.md

@@ -1,22 +0,0 @@
-# Revenue Recognition Policy
-
-Defines how to compute **net revenue** from the order files. Apply exactly.
-
-## Recognized (counts toward revenue)
-An order line counts **only** when the order `status` is one of:
-- `fulfilled`
-- `delivered`
-
-## Excluded (counts as zero)
-- `returned`
-- `refunded`
-- `cancelled`
-- `pending`
-
-## Line revenue
-`line_revenue = qty * unit_price` (unit_price is in dollars). Sum across all
-recognized line items per product.
-
-## Reporting period
-"2025" = orders whose `date` is in calendar year 2025. The `orders-2024.json`
-and `orders-2026.json` files are out of period and excluded regardless of status.

demo/generate.mjs

@@ -1,167 +1,64 @@
 /**
- * Deterministic demo-data generator for the pxpipe side-by-side demo.
+ * Smart-zone demo generator.
  *
- * Design constraints discovered empirically (see demo/README.md "Why this shape"):
- *   1. COMPACT json (no pretty-print). Pretty-printed JSON is one short field
- *      per line; each line eats a full render row, so pxpipe can only pack
- *      ~2.8k chars/page and TRUNCATES large reads (silent data loss). Compact
- *      arrays reflow into full rows and image with ~0 loss.
- *   2. SPLIT across several files, each comfortably under Claude Code's Read
- *      page cap (~25k tokens). A single giant file/line makes Read paginate
- *      and warn the model not to answer from one page. Several moderate files
- *      = one clean Read each = exactly how real sessions accrue context.
- *   3. The answer is a RANKING with a wide margin (not exact-to-the-cent), so
- *      it survives the occasional glyph misread that pxpipe's lossy tier can
- *      produce. Identity (SKU + name + customer) is the headline; the dollar
- *      figure is supporting.
+ * Builds ONE large context that you feed to two Claude columns (plain vs pxpipe):
+ *  - context/filler-*.txt : inert, token-dense logs sized to flood the window.
+ *  - context/needle.txt   : a few state-tracking facts with a deterministic
+ *                           integer answer (small -> stays TEXT even through
+ *                           pxpipe, so the pxpipe column reads it perfectly).
  *
- * Deterministic (fixed-seed xorshift32, no Math.random/Date) so the committed
- * files and EXPECTED.md always agree. Re-run: node demo/generate.mjs
+ * The point: at large context the PLAIN column drowns in the filler and gets the
+ * answer wrong (the "dumb zone"); the pxpipe column images the filler, keeps a
+ * small active context, and answers correctly.
+ *
+ * Prints the prompt to paste and the expected answer. Re-run to change SIZE.
+ *   node demo/generate.mjs
  */
-import { writeFileSync, mkdirSync, readdirSync, unlinkSync } from "node:fs";
+import { writeFileSync, mkdirSync, rmSync } from "node:fs";
 import { fileURLToPath } from "node:url";
 import { dirname, join } from "node:path";
 
 const HERE = dirname(fileURLToPath(import.meta.url));
-const DATA = join(HERE, "data");
-mkdirSync(DATA, { recursive: true });
-for (const f of readdirSync(DATA)) if (f.endsWith(".json")) unlinkSync(join(DATA, f));
+const CTX = join(HERE, "context");
+rmSync(CTX, { recursive: true, force: true });
+mkdirSync(CTX, { recursive: true });
 
-let _s = 0x9e3779b9 >>> 0;
-function rnd() { let s = _s; s ^= s << 13; s >>>= 0; s ^= s >> 17; s ^= s << 5; s >>>= 0; _s = s; return s / 4294967296; }
+const SIZE = 200_000;        // target filler tokens (bump to 400k+ if plain Claude still answers right)
+const CPT = 1.91;            // chars per text token
+const FILE_CHARS = 40_000;   // each file stays under Read's ~25k-token page
+
+let _s = 0x2545f491 >>> 0;
+const rnd = () => { let s = _s; s ^= s << 13; s >>>= 0; s ^= s >> 17; s ^= s << 5; s >>>= 0; _s = s; return s / 4294967296; };
 const ri = (a, b) => a + Math.floor(rnd() * (b - a + 1));
-const pick = (a) => a[Math.floor(rnd() * a.length)];
-function weighted(pairs) { const t = pairs.reduce((x, [, w]) => x + w, 0); let r = rnd() * t; for (const [v, w] of pairs) if ((r -= w) < 0) return v; return pairs[pairs.length - 1][0]; }
 const pad = (n, w) => String(n).padStart(w, "0");
-const money = (c) => Number((c / 100).toFixed(2));
 
-// --- products ---------------------------------------------------------------
-const CATS = ["Widgets", "Gadgets", "Tools", "Accessories", "Bundles"];
-const ADJ = ["Titanium", "Quantum", "Eco", "Nimbus", "Apex", "Lumen", "Forge", "Vertex", "Halcyon", "Onyx", "Cobalt", "Solstice", "Maple", "Arctic", "Meridian", "Cinder", "Drift", "Helix", "Ember", "Glacier"];
-const NOUN = ["Hub", "Clamp", "Sensor", "Driver", "Mount", "Coupler", "Relay", "Bracket", "Module", "Adapter", "Gauge", "Valve", "Beacon", "Spindle", "Cartridge", "Manifold", "Toolkit", "Cradle", "Latch", "Node"];
-const PRODUCTS = [];
-for (let i = 1; i <= 50; i++) PRODUCTS.push({ sku: `SKU-${pad(i, 4)}`, name: `${pick(ADJ)} ${pick(NOUN)}`, category: pick(CATS), unit_price: money(ri(500, 25000)), supplier_id: `SUP-${pad(ri(1, 12), 2)}` });
-const priceCents = new Map(PRODUCTS.map((p) => [p.sku, Math.round(p.unit_price * 100)]));
-// One deliberate bestseller so the answer is an unambiguous ranking (wide
-// margin over #2), robust to a stray glyph misread.
-const HERO = PRODUCTS[7];
+// --- needle: deterministic state-tracking reasoning --------------------------
+const start = ri(1000, 9000), a = ri(100, 900), b = ri(100, 900), c = ri(100, 900);
+const answer = ((start + a - b) * 2) - c;
+writeFileSync(join(CTX, "needle.txt"),
+`Operational note — ledger account ZX-9 (do not confuse with any other account).
 
-// --- customers --------------------------------------------------------------
-const FIRST = ["Ava", "Liam", "Noah", "Mia", "Ethan", "Zoe", "Kai", "Nora", "Leo", "Ivy", "Owen", "Maya", "Finn", "Ruby", "Cole", "Iris", "Jude", "Lena", "Reed", "Tess", "Hugo", "Esme", "Milo", "Wren"];
-const LAST = ["Okafor", "Nguyen", "Patel", "Garcia", "Kim", "Rossi", "Haddad", "Silva", "Novak", "Tanaka", "Mendez", "Bauer", "Larsen", "Costa", "Ahmed", "Petrov", "Cohen", "Reyes", "Walsh", "Fischer", "Khan", "Moreau", "Singh"];
-const REGIONS = ["NA-East", "NA-West", "EU-Central", "EU-North", "APAC", "LATAM"];
-const CUSTOMERS = [];
-for (let i = 1; i <= 80; i++) CUSTOMERS.push({ id: `CUST-${pad(i, 3)}`, name: `${pick(FIRST)} ${pick(LAST)}`, region: pick(REGIONS), tier: weighted([["standard", 5], ["silver", 3], ["gold", 2], ["platinum", 1]]), signup_date: `${ri(2021, 2024)}-${pad(ri(1, 12), 2)}-${pad(ri(1, 28), 2)}` });
-const custName = new Map(CUSTOMERS.map((c) => [c.id, c.name]));
-
-// --- orders -----------------------------------------------------------------
-const STATUS = [["fulfilled", 42], ["delivered", 24], ["returned", 9], ["refunded", 8], ["cancelled", 10], ["pending", 7]];
-function makeOrders(n, yearPicker, startId) {
-  const out = [];
-  for (let i = 0; i < n; i++) {
-    const items = [];
-    const k = weighted([[1, 5], [2, 4], [3, 2], [4, 1]]);
-    for (let j = 0; j < k; j++) { const p = rnd() < 0.16 ? HERO : pick(PRODUCTS); items.push({ sku: p.sku, qty: ri(1, 6), unit_price: p.unit_price }); }
-    out.push({ order_id: `ORD-${pad(startId + i, 6)}`, customer_id: `CUST-${pad(ri(1, 80), 3)}`, date: `${yearPicker()}-${pad(ri(1, 12), 2)}-${pad(ri(1, 28), 2)}`, status: weighted(STATUS), items });
-  }
-  return out;
-}
-// Files: two out-of-period (2024, 2026) + two in-period halves of 2025. The
-// file names encode the year so the "2025 only" filter is unmistakable.
-const o2024 = makeOrders(120, () => 2024, 100001);
-const h1 = makeOrders(170, () => 2025, 100201).map((o) => ({ ...o, date: `2025-${pad(ri(1, 6), 2)}-${pad(ri(1, 28), 2)}` }));
-const h2 = makeOrders(170, () => 2025, 100401).map((o) => ({ ...o, date: `2025-${pad(ri(7, 12), 2)}-${pad(ri(1, 28), 2)}` }));
-const o2026 = makeOrders(110, () => 2026, 100601);
-const ALL = [...o2024, ...h1, ...h2, ...o2026];
-
-// --- ground truth: net 2025 revenue per product (recognized statuses) -------
-const RECOGNIZED = new Set(["fulfilled", "delivered"]);
-const rev = new Map(), spend = new Map();
-for (const o of ALL) {
-  if (!o.date.startsWith("2025-") || !RECOGNIZED.has(o.status)) continue;
-  for (const it of o.items) {
-    const c = it.qty * (priceCents.get(it.sku) ?? Math.round(it.unit_price * 100));
-    rev.set(it.sku, (rev.get(it.sku) ?? 0) + c);
-    let m = spend.get(it.sku); if (!m) { m = new Map(); spend.set(it.sku, m); }
-    m.set(o.customer_id, (m.get(o.customer_id) ?? 0) + c);
-  }
-}
-const ranked = [...rev.entries()].sort((a, b) => b[1] - a[1]);
-const [winSku, winCents] = ranked[0]; const runnerCents = ranked[1][1];
-const winP = PRODUCTS.find((p) => p.sku === winSku);
-const [topCust, topCents] = [...spend.get(winSku).entries()].sort((a, b) => b[1] - a[1])[0];
-
-// --- write compact files ----------------------------------------------------
-const jw = (name, obj) => { const t = JSON.stringify(obj); writeFileSync(join(DATA, name), t + "\n"); return t.length; };
-const files = {
-  "products.json": jw("products.json", PRODUCTS),
-  "customers.json": jw("customers.json", CUSTOMERS),
-  "orders-2024.json": jw("orders-2024.json", o2024),
-  "orders-2025-h1.json": jw("orders-2025-h1.json", h1),
-  "orders-2025-h2.json": jw("orders-2025-h2.json", h2),
-  "orders-2026.json": jw("orders-2026.json", o2026),
-};
-
-writeFileSync(join(HERE, "RETURNS_POLICY.md"), `# Revenue Recognition Policy
-
-Defines how to compute **net revenue** from the order files. Apply exactly.
-
-## Recognized (counts toward revenue)
-An order line counts **only** when the order \`status\` is one of:
-- \`fulfilled\`
-- \`delivered\`
-
-## Excluded (counts as zero)
-- \`returned\`
-- \`refunded\`
-- \`cancelled\`
-- \`pending\`
-
-## Line revenue
-\`line_revenue = qty * unit_price\` (unit_price is in dollars). Sum across all
-recognized line items per product.
-
-## Reporting period
-"2025" = orders whose \`date\` is in calendar year 2025. The \`orders-2024.json\`
-and \`orders-2026.json\` files are out of period and excluded regardless of status.
+The ledger balance of account ZX-9 began the quarter at ${start} units.
+In week 3, an adjustment increased the ZX-9 balance by ${a} units.
+In week 7, a correction reduced the ZX-9 balance by ${b} units.
+In week 9, the ZX-9 balance was doubled following a stock split.
+In week 11, ${c} units were withdrawn from ZX-9.
+No other changes to ZX-9 occurred during the quarter.
 `);
 
-writeFileSync(join(HERE, "EXPECTED.md"), `# Expected answer (ground truth)
+// --- filler: inert, token-dense, no answer-like content ----------------------
+const LVL = ["INFO", "DEBUG", "WARN", "TRACE"], SVC = ["billing", "auth", "cache", "router", "queue", "ingest", "render", "index"];
+const line = () => `2026-${pad(ri(1,12),2)}-${pad(ri(1,28),2)}T${pad(ri(0,23),2)}:${pad(ri(0,59),2)}:${pad(ri(0,59),2)}Z ${LVL[ri(0,3)]} svc=${SVC[ri(0,7)]} req=${Array.from({length:8},()=>"0123456789abcdef"[ri(0,15)]).join("")} shard=${ri(0,31)} lat=${ri(1,800)}ms msg=processed batch ${ri(1000,99999)} ok`;
 
-Computed deterministically by \`generate.mjs\`. Both demo columns should land here.
+let filler = "";
+const targetChars = Math.round(SIZE * CPT);
+while (filler.length < targetChars) filler += line() + "\n";
+let n = 0;
+for (let off = 0; off < filler.length; off += FILE_CHARS) writeFileSync(join(CTX, `filler-${pad(n++, 3)}.txt`), filler.slice(off, off + FILE_CHARS));
 
-## Answer
-| field | value |
-|---|---|
-| SKU | \`${winSku}\` |
-| product name | ${winP.name} |
-| net revenue (2025, recognized) | ~$${money(winCents).toLocaleString()} |
-| top customer | ${custName.get(topCust)} (\`${topCust}\`) |
-| that customer's spend on it | ~$${money(topCents).toLocaleString()} |
+const prompt = "Read EVERY file in the context/ folder (all the filler-*.txt logs AND needle.txt), then answer using only their contents: What is the final ledger balance of account ZX-9? Reply with ONLY the integer.";
 
-Margin over the #2 product: **$${money(winCents - runnerCents).toLocaleString()}** — a wide,
-unambiguous gap, so the *ranking* survives a stray glyph misread even if a
-dollar total is off by a little. Grade on the identity (SKU/name/customer),
-treat the dollar figure as approximate.
-
-## Why it can't be shortcut
-- **Year filter:** orders span 2024–2026; only 2025 counts (note the file names).
-- **Status filter:** only \`fulfilled\`+\`delivered\` count — a model that assumes
-  "fulfilled only" gets a different number, proving it read RETURNS_POLICY.md.
-- **Three-way join:** revenue from the order files, name from products.json,
-  top-customer name from customers.json.
-
-## File sizes (each well under Read's ~25k-token page cap)
-| file | chars | ~tokens |
-|---|---:|---:|
-${Object.entries(files).map(([n, c]) => `| ${n} | ${c.toLocaleString()} | ${Math.round(c / 1.91).toLocaleString()} |`).join("\n")}
-
-Real per-request savings on this content: run \`node demo/preflight.mjs\`.
-Your historical real savings: run \`node demo/analyze-events.mjs\`.
-`);
-
-console.log("demo data (compact json) written:");
-let tot = 0; for (const [n, c] of Object.entries(files)) { tot += c; console.log(`  ${n.padEnd(22)} ${c.toLocaleString().padStart(8)} chars (~${Math.round(c / 1.91).toLocaleString()} tok)`); }
-console.log(`  total ${tot.toLocaleString()} chars (~${Math.round(tot / 1.91).toLocaleString()} tok of bulk reads)`);
-console.log(`\nground truth: ${winSku} "${winP.name}" net-2025 ≈ $${money(winCents).toLocaleString()}`);
-console.log(`  top customer: ${custName.get(topCust)} (≈$${money(topCents).toLocaleString()}), margin over #2: $${money(winCents - runnerCents).toLocaleString()}`);
+console.log(`generated context/: ${n} filler files (~${SIZE.toLocaleString()} tokens) + needle.txt`);
+console.log(`\n--- paste this prompt in BOTH Claude columns ---\n${prompt}\n`);
+console.log(`--- expected answer (ground truth): ${answer} ---`);
+console.log(`plain column at this size may get it WRONG (context overload); pxpipe column should get ${answer}.`);