mirror of
https://github.com/teamchong/pxpipe.git
synced 2026-07-22 02:02:51 +02:00
fix(vision): patch-based Anthropic image-token model + doc sync (#27)
Anthropic bills images by a 28-px patch grid (⌈w/28⌉×⌈h/28⌉ after tier downscale), not (w·h)/750 — the old fit overcharged large images ~4–5%. - new src/core/anthropic-vision.ts: exact patch + downscale model matching Anthropic's reference (incl. extreme aspect ratios); wired into the unified visionTokensForModel router. - gate counts real 28-px patches; 10% bias renamed ANTHROPIC_GATE_MARGIN, gate-only. - removed ANTHROPIC_PIXELS_PER_TOKEN / IMAGE_COST_SAFETY_MARGIN; cols 313 → 312 (slab = 1568 px cap). - docs (README / RENDER_SIZING / TRANSFORM_INFO / audit) synced to shipped geometry (1568×728, 312 cols) and the patch billing model.
This commit is contained in:
@@ -13,6 +13,11 @@ follow from it.
|
||||
~2.8×4.4 px. We were paying full price for pixels destroyed in transit.
|
||||
- **Shipped:** page geometry clamped to **1568×728 = 1.14 MP** (WYSIWYG: billed/actual
|
||||
pixel ratio 3.25 → 1.04). No token-cost change per image; 614/614 tests green.
|
||||
- **Update (2026-07-04):** the `px/750` figure above is a ~4–5% continuous
|
||||
approximation. Anthropic's current docs bill the exact **28-px patch count**
|
||||
`⌈w/28⌉×⌈h/28⌉` (a 1568×728 page = 56×26 = **1456** tokens, ≈ the px/750 slope
|
||||
measured here). The gate and export now use that exact formula via
|
||||
`src/core/anthropic-vision.ts` (tiers: standard 1568/1568, high-res 2576/4784).
|
||||
- Reading exact strings off even **crisp** 5×8 tops out at **63%** (24-token blind
|
||||
test). Every miss is one of two classes the glyph-confusability matrix predicts:
|
||||
**case-normalization** (camelCase) and **single-glyph substitution** (hex/num).
|
||||
|
||||
@@ -64,9 +64,9 @@ The profitability gate resolves the same profile as the renderer and derives:
|
||||
- per-image token cost
|
||||
|
||||
Changing cell spacing or font therefore cannot leave cost prediction on a
|
||||
hardcoded 5×8 canvas. Claude uses the Anthropic pixel formula with a conservative
|
||||
margin, GPT uses model-specific tile/patch pricing, and Grok uses the measured
|
||||
tokens-per-megapixel rate.
|
||||
hardcoded 5×8 canvas. Claude uses the Anthropic 28-px patch formula (with a
|
||||
conservative margin at the gate), GPT uses model-specific tile/patch pricing, and
|
||||
Grok uses the measured tokens-per-megapixel rate.
|
||||
|
||||
## Why not square pages
|
||||
|
||||
|
||||
@@ -15,8 +15,9 @@ to re-read that text in token form — Anthropic prompt-caches it, and image
|
||||
blocks OCR cleanly at small font sizes. So pxpipe pulls the static prefix
|
||||
out of the JSON body, renders it as one or more grayscale PNG image blocks,
|
||||
and pins a single `cache_control` breakpoint on the last image. Anthropic
|
||||
charges roughly `ceil(W*H/750)` tokens per image; the current Anthropic profile
|
||||
caps pages at 1568×728 so rasterized pixels survive the provider's resize. A
|
||||
charges `⌈W/28⌉×⌈H/28⌉` visual tokens per image (28-px patches); the current
|
||||
Anthropic profile caps pages at 1568×728 (so `56×26 = 1456` tokens) and keeps
|
||||
rasterized pixels under the provider's resize. A
|
||||
large static slab becomes a small set of image pages on the first turn and a
|
||||
cache-read (billed at 0.10×) on subsequent turns. The trade is real text tokens
|
||||
for image tokens cached under the same stable prefix.
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* Anthropic image / vision INPUT-TOKEN cost model.
|
||||
*
|
||||
* Anthropic bills images by 28×28-pixel PATCHES, not by a pixel ratio: an image
|
||||
* costs `⌈width/28⌉ × ⌈height/28⌉` visual tokens, computed AFTER the image is
|
||||
* downscaled to fit the model tier's long-edge and visual-token limits.
|
||||
* (The older `(width×height)/750` figure was a ~4–5% continuous approximation of
|
||||
* this same 28²=784 px²/patch grid; it is no longer the documented formula.)
|
||||
* https://platform.claude.com/docs/en/build-with-claude/vision
|
||||
* https://platform.claude.com/docs/en/build-with-claude/vision-coordinates
|
||||
*
|
||||
* This module is the single source of truth for that math. It is the *documented
|
||||
* provider formula* only — any gate conservatism (safety margin) lives at the
|
||||
* gate, not here, so this stays honest about what Anthropic actually charges.
|
||||
*/
|
||||
|
||||
/** One visual token per 28×28-pixel patch. Also the Qwen2-VL grid; NOT OpenAI's
|
||||
* 32-px patch / 512-px tile model — keep those on the OpenAI path only. */
|
||||
export const ANTHROPIC_PATCH_PX = 28;
|
||||
|
||||
export interface AnthropicVisionProfile {
|
||||
readonly tier: 'high-res' | 'standard';
|
||||
/** Neither side may exceed this after downscale (px). */
|
||||
readonly maxLongEdge: number;
|
||||
/** ⌈w/28⌉×⌈h/28⌉ may not exceed this after downscale (visual tokens). */
|
||||
readonly maxVisualTokens: number;
|
||||
}
|
||||
|
||||
/** Model bases on Anthropic's high-resolution tier (max long edge 2576 px, max
|
||||
* 4784 visual tokens). Everything else is standard (1568 px / 1568 tokens).
|
||||
* Source: the Vision docs resolution-tier table. */
|
||||
const HIGH_RES_BASES = [
|
||||
'claude-fable-5',
|
||||
'claude-mythos-5',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-4-7',
|
||||
'claude-sonnet-5',
|
||||
] as const;
|
||||
|
||||
const HIGH_RES: AnthropicVisionProfile = { tier: 'high-res', maxLongEdge: 2576, maxVisualTokens: 4784 };
|
||||
const STANDARD: AnthropicVisionProfile = { tier: 'standard', maxLongEdge: 1568, maxVisualTokens: 1568 };
|
||||
|
||||
/** Resolve a model's vision tier. Unknown/blank models fall back to the
|
||||
* conservative (smaller) standard tier. Matches exact base or `<base>-suffix` /
|
||||
* `<base>[variant]` so aliases (e.g. `claude-fable-5-high`, `...[1m]`) tier alike. */
|
||||
export function anthropicVisionProfile(model: string | null | undefined): AnthropicVisionProfile {
|
||||
const m = (model ?? '').toLowerCase();
|
||||
const isHighRes = HIGH_RES_BASES.some((b) => m === b || m.startsWith(`${b}-`) || m.startsWith(`${b}[`));
|
||||
return isHighRes ? HIGH_RES : STANDARD;
|
||||
}
|
||||
|
||||
/** Raw 28-px patch count for a `w×h` image, i.e. the visual-token cost when the
|
||||
* image already fits the tier limits (no downscale). `⌈w/28⌉` inherently pads
|
||||
* the right/bottom edge up to the next 28-px multiple, as Anthropic documents. */
|
||||
export function patchTokens(width: number, height: number): number {
|
||||
const w = Math.max(1, Math.floor(width));
|
||||
const h = Math.max(1, Math.floor(height));
|
||||
return Math.ceil(w / ANTHROPIC_PATCH_PX) * Math.ceil(h / ANTHROPIC_PATCH_PX);
|
||||
}
|
||||
|
||||
/** True when a `w×h` image fits a tier with no resize: both padded patch edges
|
||||
* are within the long-edge limit AND the patch count is within the token
|
||||
* budget. Mirrors the `fits` predicate in Anthropic's reference resize. */
|
||||
function fitsTier(w: number, h: number, maxLongEdge: number, maxVisualTokens: number): boolean {
|
||||
return (
|
||||
Math.ceil(w / ANTHROPIC_PATCH_PX) * ANTHROPIC_PATCH_PX <= maxLongEdge &&
|
||||
Math.ceil(h / ANTHROPIC_PATCH_PX) * ANTHROPIC_PATCH_PX <= maxLongEdge &&
|
||||
patchTokens(w, h) <= maxVisualTokens
|
||||
);
|
||||
}
|
||||
|
||||
/** Largest aspect-preserving size that fits the tier, exactly as Anthropic's
|
||||
* reference does it: binary-search the long edge (short edge = round(long ×
|
||||
* aspect)), recursing with a swap for portrait images. Original if it fits. */
|
||||
function resizedSize(w: number, h: number, maxLongEdge: number, maxVisualTokens: number): [number, number] {
|
||||
if (fitsTier(w, h, maxLongEdge, maxVisualTokens)) return [w, h];
|
||||
if (h > w) {
|
||||
const [rh, rw] = resizedSize(h, w, maxLongEdge, maxVisualTokens);
|
||||
return [rw, rh];
|
||||
}
|
||||
// Landscape (w ≥ h): binary-search the largest long edge that still fits.
|
||||
const aspect = h / w;
|
||||
let lo = 1;
|
||||
let hi = w;
|
||||
let best = 1;
|
||||
while (lo <= hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (fitsTier(mid, Math.max(1, Math.round(mid * aspect)), maxLongEdge, maxVisualTokens)) {
|
||||
best = mid;
|
||||
lo = mid + 1;
|
||||
} else {
|
||||
hi = mid - 1;
|
||||
}
|
||||
}
|
||||
return [best, Math.max(1, Math.round(best * aspect))];
|
||||
}
|
||||
|
||||
/**
|
||||
* Anthropic visual-token cost of an image at `width×height` for `model`'s tier.
|
||||
* Applies the documented resize — the largest aspect-preserving size that fits
|
||||
* BOTH the long-edge limit and the visual-token budget, found by binary search
|
||||
* exactly as Anthropic's reference `count_image_tokens` — then the 28-px patch
|
||||
* count.
|
||||
*
|
||||
* Note: pxpipe's own pages are always ≤ 1568×728, so the resize never fires for
|
||||
* proxy output (both tiers charge the raw patch count). The resize path is for
|
||||
* correctness as a general-purpose estimator (e.g. arbitrary export input).
|
||||
*/
|
||||
export function anthropicVisionTokens(
|
||||
model: string | null | undefined,
|
||||
width: number,
|
||||
height: number,
|
||||
): number {
|
||||
const { maxLongEdge, maxVisualTokens } = anthropicVisionProfile(model);
|
||||
const w = Math.max(1, Math.floor(width));
|
||||
const h = Math.max(1, Math.floor(height));
|
||||
const [rw, rh] = resizedSize(w, h, maxLongEdge, maxVisualTokens);
|
||||
return patchTokens(rw, rh);
|
||||
}
|
||||
+5
-4
@@ -248,8 +248,9 @@ export function parseExportArgv(
|
||||
* Per-image vision-token cost for a rendered PNG at the given pixel dimensions.
|
||||
*
|
||||
* - **Claude / Anthropic models** (`model.startsWith('claude')` or
|
||||
* `model.includes('anthropic')`): uses Anthropic's billing formula
|
||||
* `ceil(width × height / 750 × 1.10)`, matching the live transform.
|
||||
* `model.includes('anthropic')`): Anthropic's documented 28-px patch model
|
||||
* `⌈w/28⌉×⌈h/28⌉` (tier-aware downscale), via `visionTokensForModel`. This is
|
||||
* the exact provider cost — no gate margin — for reporting.
|
||||
* - **OpenAI-shaped models**: delegates to the same exact-model router as the
|
||||
* proxy (Grok measured pixel rate; GPT tile/patch rate).
|
||||
*/
|
||||
@@ -280,7 +281,7 @@ export interface ExportTokenReport {
|
||||
* imageTokens = estimateImageCount(text, cols) × exportImageTokens(model, stripW, MAX_HEIGHT_PX)
|
||||
* textTokens = sourceText.length / REPORT_CHARS_PER_TOKEN
|
||||
*
|
||||
* `exportImageTokens` routes to the Anthropic billing formula (width×height/750×1.10)
|
||||
* `exportImageTokens` routes to the Anthropic 28-px patch model (⌈w/28⌉×⌈h/28⌉)
|
||||
* for Claude models, and to the GPT tile-pricing formula for GPT/o-series models.
|
||||
*
|
||||
* `factsheetItemCount` is the number of unique precision-critical identifier strings
|
||||
@@ -432,7 +433,7 @@ export async function runExportCore(
|
||||
});
|
||||
|
||||
// Compute token costs using actual rendered image dimensions (more accurate than estimate).
|
||||
// exportImageTokens routes to the Anthropic billing formula for claude-* models and to
|
||||
// exportImageTokens routes to the Anthropic 28-px patch model for claude-* models and to
|
||||
// the GPT tile-pricing formula for GPT/o-series models.
|
||||
const textTokens = Math.round(sourceText.length / REPORT_CHARS_PER_TOKEN);
|
||||
let imageTokens = 0;
|
||||
|
||||
+4
-3
@@ -28,11 +28,10 @@ import {
|
||||
countVisualRows,
|
||||
estimateImageCount,
|
||||
sha8,
|
||||
ANTHROPIC_PIXELS_PER_TOKEN,
|
||||
IMAGE_COST_SAFETY_MARGIN,
|
||||
type TransformInfo,
|
||||
type TransformOptions,
|
||||
} from './transform.js';
|
||||
import { anthropicVisionTokens } from './anthropic-vision.js';
|
||||
import { stripSchemaDescriptions } from './schema-strip.js';
|
||||
import {
|
||||
planGptCollapse,
|
||||
@@ -132,7 +131,9 @@ export const GROK_TOKENS_PER_MEGAPIXEL = 1000;
|
||||
* OpenAI tile/patch formula. Model-based, not endpoint-based. */
|
||||
export function visionTokensForModel(model: string, w: number, h: number): number {
|
||||
if (isClaudeModel(model)) {
|
||||
return Math.ceil((w * h / ANTHROPIC_PIXELS_PER_TOKEN) * IMAGE_COST_SAFETY_MARGIN);
|
||||
// Anthropic's documented 28-px patch model (tier-aware downscale). This is the
|
||||
// exact provider cost; the gate applies its own margin separately.
|
||||
return anthropicVisionTokens(model, w, h);
|
||||
}
|
||||
if (isGrokModel(model)) {
|
||||
const pixels = Math.max(0, w) * Math.max(0, h);
|
||||
|
||||
+5
-4
@@ -1,8 +1,8 @@
|
||||
/**
|
||||
* Text → PNG renderer. Blits atlas glyphs into a grayscale framebuffer, then PNG-encodes.
|
||||
* Iterates by codepoint so East Asian Wide chars (2-cell advance) and surrogate pairs handled correctly.
|
||||
* Pages capped at ~1932×1932 px: Fable/Opus 4.8 >20-image requests are held to ≤2000 px/side
|
||||
* (REJECTED if exceeded, not silently downscaled); ≤4784 token limit binds first at 1932 px.
|
||||
* Pages capped at 1568×728 px (1.14 MP): under both Anthropic tiers' limits, so every
|
||||
* page bills at its raw 28-px patch count with no server-side downscale (WYSIWYG).
|
||||
*/
|
||||
|
||||
import {
|
||||
@@ -141,7 +141,8 @@ function grayGlyph(codepoint: number, font: RenderFont | undefined): { atlas: Gr
|
||||
|
||||
/** Page-height ceiling. Measured (2026-07-01, count_tokens sweep, claude-sonnet-4-5 — see
|
||||
* /tmp/pxexp/LEVER1-findings.md): the API downscales any image to fit BOTH long-edge ≤1568
|
||||
* AND ~1.15 MP (≈1,143,750 px), then bills ≈ px/750 (≈1525 tok cap, applied per-image).
|
||||
* AND ~1.15 MP (≈1,143,750 px), then bills the exact 28-px patch count ⌈w/28⌉×⌈h/28⌉
|
||||
* (1568×728 = 56×26 = 1456 tokens/image; the old ≈px/750 slope was the 28²=784 approximation).
|
||||
* The old 1932×1932 page was billed at cap but resampled 0.555× → 5×8 glyphs reached the
|
||||
* encoder at ~2.8×4.4 px. New page shape 1568×728 = 1,141,504 px fits both bounds →
|
||||
* WYSIWYG for the vision encoder (also satisfies ≤2000 px/side for >20-image requests). */
|
||||
@@ -1103,7 +1104,7 @@ export async function renderTextToPngs(
|
||||
|
||||
const GUTTER_CELLS = 4;
|
||||
// Width hard-capped at the API's long-edge bound: anything wider is resampled server-side
|
||||
// (measured 2026-07-01: fit-within 1568-edge AND ~1.15 MP, then ≈px/750 billing).
|
||||
// (measured 2026-07-01: fit-within 1568-edge AND ~1.15 MP, then 28-px patch billing ⌈w/28⌉×⌈h/28⌉).
|
||||
const MAX_WIDTH_PX = 1568;
|
||||
|
||||
const GUTTER_DIVIDER_INK = 64; // pre-invert → 191 post-invert: light gray column separator
|
||||
|
||||
+18
-18
@@ -41,6 +41,7 @@ import { bytesToBase64 } from './png.js';
|
||||
import { collapseHistory, HISTORY_SYNTHETIC_INTRO } from './history.js';
|
||||
import type { GptHistoryOptions } from './openai-history.js';
|
||||
import { CACHE_CREATE_RATE, CACHE_READ_RATE } from './baseline.js';
|
||||
import { patchTokens } from './anthropic-vision.js';
|
||||
|
||||
/** Per-block descriptor passed to `TransformOptions.keepSharp`. */
|
||||
export interface KeepSharpBlock {
|
||||
@@ -142,7 +143,7 @@ const DEFAULTS: Required<TransformOptions> = {
|
||||
historyAmortizationHorizon: 1,
|
||||
priorWarmTokens: 0,
|
||||
priorWarmImageTokens: 0,
|
||||
// Multi-col off: single-col slab already holds ~50k chars; extra OCR risk not worth it.
|
||||
// Multi-col off: single-col slab already holds ~28k chars/page at the 1568×728 cap; extra OCR risk not worth it.
|
||||
multiCol: 1,
|
||||
reflow: true,
|
||||
keepSharp: () => false,
|
||||
@@ -162,11 +163,11 @@ export const CLAUDE_CODE_OAUTH_IDENTITY =
|
||||
|
||||
// --- per-block break-even check ---
|
||||
//
|
||||
// Image token cost is computed from pixel area (Anthropic formula: w×h/750,
|
||||
// empirically accurate to ~5% on dense PNGs). Constants bias CONSERVATIVE:
|
||||
// CHARS_PER_TOKEN=4 under-estimates text savings; multi-col cost is linearly
|
||||
// scaled from single-col + 10% margin. Mispredictions leave money on the
|
||||
// table; they never generate net-loss images.
|
||||
// Image token cost uses Anthropic's documented 28-px patch model (see
|
||||
// src/core/anthropic-vision.ts). Constants bias CONSERVATIVE: CHARS_PER_TOKEN=4
|
||||
// under-estimates text savings; the gate multiplies the patch count by
|
||||
// ANTHROPIC_GATE_MARGIN on top. Mispredictions leave money on the table; they
|
||||
// never generate net-loss images.
|
||||
|
||||
/** English ~4 chars per token average (conservative for code/JSON content). */
|
||||
const CHARS_PER_TOKEN = 4;
|
||||
@@ -189,16 +190,11 @@ export const HISTORY_CHARS_PER_TOKEN = 2.0;
|
||||
* of truth — src/core/export.ts imports this rather than redefining it. */
|
||||
export const REPORT_CHARS_PER_TOKEN = 3.7;
|
||||
|
||||
/** Anthropic image-billing formula: `tokens ≈ width × height / 750`.
|
||||
* https://docs.anthropic.com/en/docs/build-with-claude/vision#image-tokens
|
||||
* Accurate to ~5% on dense glyph PNGs (N=14 empirical calibration). The renderer
|
||||
* sizes height to content, so per-block images cost far less than full-canvas.
|
||||
* Exported so the export pipeline can reuse the same constant rather than hardcoding. */
|
||||
export const ANTHROPIC_PIXELS_PER_TOKEN = 750;
|
||||
/** Conservative 10% upward bias on Anthropic image token estimates — keeps the gate
|
||||
* on the safe (pass-through) side when the true cost is near the break-even point.
|
||||
* Exported so the export pipeline reuses the same value. */
|
||||
export const IMAGE_COST_SAFETY_MARGIN = 1.10;
|
||||
/** Gate-only conservatism: a 10% upward bias on the patch-count image estimate,
|
||||
* keeping the gate on the safe (pass-through) side near break-even. This is NOT
|
||||
* part of Anthropic's documented cost — the documented per-image formula lives in
|
||||
* `anthropicVisionTokens` (anthropic-vision.ts); this margin only tunes the gate. */
|
||||
export const ANTHROPIC_GATE_MARGIN = 1.10;
|
||||
|
||||
/** Width in px of a single-col PNG. Must stay in sync with `renderChunkToPng` (render.ts). */
|
||||
function singleColWidthPx(cols: number): number {
|
||||
@@ -241,8 +237,12 @@ function imageTokensForRows(
|
||||
const rowsInLast = Math.min(Math.max(1, linesInLast), rowsPerImage);
|
||||
const fullImageHeight = 2 * PAD_Y + rowsPerImage * CELL_H;
|
||||
const lastImageHeight = 2 * PAD_Y + rowsInLast * CELL_H;
|
||||
const totalPixels = fullImages * widthPx * fullImageHeight + widthPx * lastImageHeight;
|
||||
return Math.ceil((totalPixels / ANTHROPIC_PIXELS_PER_TOKEN) * IMAGE_COST_SAFETY_MARGIN);
|
||||
// Anthropic bills per-image by 28-px patches (not by summed pixel area), so
|
||||
// count each image separately. pxpipe pages are ≤ 1568×728, so no tier
|
||||
// downscale applies and the raw patch count is tier-agnostic.
|
||||
const patchSum =
|
||||
fullImages * patchTokens(widthPx, fullImageHeight) + patchTokens(widthPx, lastImageHeight);
|
||||
return Math.ceil(patchSum * ANTHROPIC_GATE_MARGIN);
|
||||
}
|
||||
|
||||
/** Exact image-token cost for `text`. Uses `countVisualRows` and optionally
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
anthropicVisionProfile,
|
||||
anthropicVisionTokens,
|
||||
patchTokens,
|
||||
ANTHROPIC_PATCH_PX,
|
||||
} from '../src/core/anthropic-vision.js';
|
||||
|
||||
// Anthropic bills images by 28×28-pixel patches: ⌈w/28⌉×⌈h/28⌉ visual tokens,
|
||||
// after downscaling to the model tier's long-edge (1568/2576) and token-budget
|
||||
// (1568/4784) limits. Numbers below are cross-checked against Anthropic's own
|
||||
// worked cost table (platform.claude.com/docs/en/build-with-claude/vision).
|
||||
|
||||
describe('patchTokens — raw 28-px patch count', () => {
|
||||
it('is one token per full 28×28 patch, padding partial edges up', () => {
|
||||
expect(ANTHROPIC_PATCH_PX).toBe(28);
|
||||
expect(patchTokens(28, 28)).toBe(1);
|
||||
expect(patchTokens(29, 29)).toBe(4); // ⌈29/28⌉² = 2² = 4
|
||||
expect(patchTokens(1000, 1000)).toBe(1296); // 36² (docs worked example)
|
||||
expect(patchTokens(1568, 728)).toBe(1456); // 56×26 — pxpipe's full dense page
|
||||
expect(patchTokens(1928, 1928)).toBe(4761); // 69² (old page, high-res)
|
||||
});
|
||||
});
|
||||
|
||||
describe('anthropicVisionProfile — tier by model', () => {
|
||||
it('puts the documented high-res models on the 2576/4784 tier', () => {
|
||||
for (const m of ['claude-fable-5', 'claude-mythos-5', 'claude-opus-4-8', 'claude-opus-4-7', 'claude-sonnet-5']) {
|
||||
expect(anthropicVisionProfile(m).tier).toBe('high-res');
|
||||
}
|
||||
// aliases / variant tags tier with their base
|
||||
expect(anthropicVisionProfile('claude-fable-5-high').tier).toBe('high-res');
|
||||
expect(anthropicVisionProfile('claude-opus-4-8[1m]').tier).toBe('high-res');
|
||||
expect(anthropicVisionProfile('claude-fable-5')).toEqual({ tier: 'high-res', maxLongEdge: 2576, maxVisualTokens: 4784 });
|
||||
});
|
||||
|
||||
it('falls back to the conservative standard 1568/1568 tier otherwise', () => {
|
||||
for (const m of ['claude-opus-4-5', 'claude-sonnet-4-6', 'claude-haiku-4-5', 'claude-3-5-sonnet', '', undefined, null]) {
|
||||
expect(anthropicVisionProfile(m as string).tier).toBe('standard');
|
||||
}
|
||||
expect(anthropicVisionProfile('claude-opus-4-5')).toEqual({ tier: 'standard', maxLongEdge: 1568, maxVisualTokens: 1568 });
|
||||
});
|
||||
});
|
||||
|
||||
// Independent translation of Anthropic's documented resize + patch count, used
|
||||
// as the oracle for the property test below. Kept separate from the production
|
||||
// code so a future edit to one is caught by the other.
|
||||
function refVisionTokens(maxEdge: number, maxTok: number, W: number, H: number): number {
|
||||
const pt = (w: number, h: number): number => Math.ceil(w / 28) * Math.ceil(h / 28);
|
||||
const fits = (w: number, h: number): boolean =>
|
||||
Math.ceil(w / 28) * 28 <= maxEdge && Math.ceil(h / 28) * 28 <= maxEdge && pt(w, h) <= maxTok;
|
||||
const resize = (w: number, h: number): [number, number] => {
|
||||
if (fits(w, h)) return [w, h];
|
||||
if (h > w) { const [rh, rw] = resize(h, w); return [rw, rh]; }
|
||||
const a = h / w;
|
||||
let lo = 1, hi = w, best = 1;
|
||||
while (lo <= hi) {
|
||||
const m = (lo + hi) >> 1;
|
||||
if (fits(m, Math.max(1, Math.round(m * a)))) { best = m; lo = m + 1; } else hi = m - 1;
|
||||
}
|
||||
return [best, Math.max(1, Math.round(best * a))];
|
||||
};
|
||||
const [rw, rh] = resize(Math.max(1, Math.floor(W)), Math.max(1, Math.floor(H)));
|
||||
return pt(rw, rh);
|
||||
}
|
||||
|
||||
const STD = 'claude-opus-4-5'; // standard tier: 1568 / 1568
|
||||
const HI = 'claude-fable-5'; // high-res tier: 2576 / 4784
|
||||
|
||||
describe('anthropicVisionTokens — documented cost with resize', () => {
|
||||
it('reproduces Anthropic\'s worked cost table on both tiers', () => {
|
||||
const rows: Array<[string, number, number, number, number]> = [
|
||||
// model, w, h, standard, high-res
|
||||
[STD, 200, 200, 64, 64], [STD, 1000, 1000, 1296, 1296], [STD, 1092, 1092, 1521, 1521],
|
||||
[STD, 1920, 1080, 1560, 2691], [STD, 2000, 1500, 1564, 3888], [STD, 3840, 2160, 1560, 4784],
|
||||
];
|
||||
for (const [, w, h, std, hi] of rows) {
|
||||
expect(anthropicVisionTokens(STD, w, h), `${w}x${h} standard`).toBe(std);
|
||||
expect(anthropicVisionTokens(HI, w, h), `${w}x${h} high-res`).toBe(hi);
|
||||
}
|
||||
});
|
||||
|
||||
it('is exact on extreme aspect ratios (where a continuous scale would drift)', () => {
|
||||
// These caught the old sqrt/decrement approximation (it gave 1008 / 896 / 1504).
|
||||
expect(anthropicVisionTokens(STD, 2510, 7800)).toBe(1064);
|
||||
expect(anthropicVisionTokens(STD, 1194, 4171)).toBe(952);
|
||||
expect(anthropicVisionTokens(STD, 2384, 1625)).toBe(1551);
|
||||
});
|
||||
|
||||
it('charges the raw patch count when the image already fits', () => {
|
||||
// pxpipe's full dense page (1568×728) fits BOTH tiers unchanged.
|
||||
expect(anthropicVisionTokens(HI, 1568, 728)).toBe(1456);
|
||||
expect(anthropicVisionTokens(STD, 1568, 728)).toBe(1456);
|
||||
// High-res leaves 1928² and 1568² unchanged; standard shrinks them to 1521.
|
||||
expect(anthropicVisionTokens(HI, 1928, 1928)).toBe(4761);
|
||||
expect(anthropicVisionTokens(HI, 1568, 1568)).toBe(3136);
|
||||
expect(anthropicVisionTokens(STD, 1568, 1568)).toBe(1521);
|
||||
expect(anthropicVisionTokens(STD, 1928, 1928)).toBe(1521);
|
||||
});
|
||||
|
||||
it('never exceeds the tier visual-token budget', () => {
|
||||
for (const [w, h] of [[8000, 8000], [4000, 500], [500, 4000], [7680, 1080]] as const) {
|
||||
expect(anthropicVisionTokens(HI, w, h)).toBeLessThanOrEqual(4784);
|
||||
expect(anthropicVisionTokens(STD, w, h)).toBeLessThanOrEqual(1568);
|
||||
}
|
||||
});
|
||||
|
||||
it('equals the independent reference across a spread of sizes and aspects', () => {
|
||||
// Deterministic LCG — no Math.random, reproducible.
|
||||
let seed = 0x9e3779b9;
|
||||
const next = (n: number): number => ((seed = (seed * 1103515245 + 12345) & 0x7fffffff) % n) + 1;
|
||||
for (let i = 0; i < 400; i++) {
|
||||
const w = next(9000);
|
||||
const h = next(9000);
|
||||
expect(anthropicVisionTokens(STD, w, h), `std ${w}x${h}`).toBe(refVisionTokens(1568, 1568, w, h));
|
||||
expect(anthropicVisionTokens(HI, w, h), `hi ${w}x${h}`).toBe(refVisionTokens(2576, 4784, w, h));
|
||||
}
|
||||
});
|
||||
|
||||
it('is below the retired w×h/750 approximation for the standard-tier page', () => {
|
||||
const legacy750 = Math.ceil((1568 * 728) / 750); // 1522
|
||||
expect(anthropicVisionTokens(HI, 1568, 728)).toBeLessThan(legacy750);
|
||||
});
|
||||
});
|
||||
+21
-22
@@ -16,6 +16,7 @@ import {
|
||||
} from '../src/core/export.js';
|
||||
import { extractFactSheetTokensAllPages, extractFactSheetTokens } from '../src/core/factsheet.js';
|
||||
import { DENSE_CONTENT_CHARS_PER_IMAGE } from '../src/core/render.js';
|
||||
import { patchTokens } from '../src/core/anthropic-vision.js';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Temp-dir helpers
|
||||
@@ -500,35 +501,33 @@ describe('runExportCore integration', () => {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('exportImageTokens model routing', () => {
|
||||
// Dense export page: width = 2*4 + 384*5 = 1928 px, height = MAX_HEIGHT_PX = 1932 px
|
||||
const W = 1928;
|
||||
const H = 1932;
|
||||
// Current full dense export page: width = 2*PAD_X + 312*CELL_W = 1568 px,
|
||||
// height = MAX_HEIGHT_PX = 728 px. Fits both Anthropic tiers unchanged, so the
|
||||
// cost is the raw 28-px patch count: ⌈1568/28⌉×⌈728/28⌉ = 56×26 = 1456.
|
||||
const W = 1568;
|
||||
const H = 728;
|
||||
|
||||
it('returns Anthropic-formula tokens for claude-sonnet-4-5', () => {
|
||||
// Anthropic formula: ceil(W*H/750 * 1.10)
|
||||
const expected = Math.ceil((W * H / 750) * 1.10);
|
||||
expect(exportImageTokens('claude-sonnet-4-5', W, H)).toBe(expected);
|
||||
expect(exportImageTokens('claude-sonnet-4-5', W, H)).toBe(patchTokens(W, H));
|
||||
expect(exportImageTokens('claude-sonnet-4-5', W, H)).toBe(1456);
|
||||
});
|
||||
|
||||
it('returns Anthropic-formula tokens for any claude-* model', () => {
|
||||
const expected = Math.ceil((W * H / 750) * 1.10);
|
||||
expect(exportImageTokens('claude-opus-4', W, H)).toBe(expected);
|
||||
expect(exportImageTokens('claude-haiku-3-5', W, H)).toBe(expected);
|
||||
expect(exportImageTokens('claude-opus-4', W, H)).toBe(patchTokens(W, H));
|
||||
expect(exportImageTokens('claude-haiku-3-5', W, H)).toBe(patchTokens(W, H));
|
||||
});
|
||||
|
||||
it('returns Anthropic-formula tokens when model includes "anthropic"', () => {
|
||||
const expected = Math.ceil((W * H / 750) * 1.10);
|
||||
expect(exportImageTokens('anthropic/claude-3-5-sonnet', W, H)).toBe(expected);
|
||||
expect(exportImageTokens('anthropic/claude-3-5-sonnet', W, H)).toBe(patchTokens(W, H));
|
||||
});
|
||||
|
||||
it('returns GPT (OpenAI tile) tokens for gpt-4o', () => {
|
||||
// OpenAI tile formula is much cheaper for this image size (~765 vs ~5464)
|
||||
it('routes gpt-4o to the OpenAI tile formula, not the Anthropic patch formula', () => {
|
||||
const gpTokens = exportImageTokens('gpt-4o', W, H);
|
||||
const claudeTokens = exportImageTokens('claude-sonnet-4-5', W, H);
|
||||
// GPT-4o tile formula for 1928x1932 px: scaled to 768x769, 2x2 tiles
|
||||
// = 85 + 170*4 = 765 tokens — far less than Anthropic's ~5464
|
||||
expect(gpTokens).toBeLessThan(claudeTokens);
|
||||
expect(gpTokens).toBeGreaterThan(0);
|
||||
// Different pricing model → different number (they happen to be close at this
|
||||
// size, but must not be computed by the same formula).
|
||||
expect(gpTokens).not.toBe(claudeTokens);
|
||||
});
|
||||
|
||||
it('uses measured Grok pixel pricing instead of the GPT fallback', () => {
|
||||
@@ -538,12 +537,12 @@ describe('exportImageTokens model routing', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('Claude image tokens are substantially higher than GPT for the same full-page image', () => {
|
||||
// The issue was a ~7x underestimate when using GPT formula for Claude.
|
||||
// Verify the ratio is at least 5x so the fix is clearly meaningful.
|
||||
const claudeTokens = exportImageTokens('claude-sonnet-4-5', W, H);
|
||||
const gpTokens = exportImageTokens('gpt-4o', W, H);
|
||||
expect(claudeTokens / gpTokens).toBeGreaterThan(5);
|
||||
it('caps a huge Claude image at the tier visual-token budget (patch downscale)', () => {
|
||||
// The old ceil(w*h/750*1.10) formula ignored Anthropic's downscale and grossly
|
||||
// overcharged large images (a 1928² page "cost" ~5464 when the API bills ≤1568
|
||||
// on the standard tier / 4761 on high-res). The patch model honors the cap.
|
||||
expect(exportImageTokens('claude-sonnet-4-5', 4000, 4000)).toBeLessThanOrEqual(1568);
|
||||
expect(exportImageTokens('claude-fable-5', 4000, 4000)).toBeLessThanOrEqual(4784);
|
||||
});
|
||||
|
||||
it('computeTokenReport uses Anthropic formula for default claude model', () => {
|
||||
|
||||
@@ -99,11 +99,12 @@ describe('visionTokensForModel (Claude on the Responses path)', () => {
|
||||
expect(isClaudeModel(undefined)).toBe(false);
|
||||
});
|
||||
|
||||
it('prices claude images by pixel area, not GPT tiles', () => {
|
||||
it('prices claude images by the 28-px patch model, not GPT tiles', () => {
|
||||
// Codex speaks OpenAI Responses; some models on that path are Claude, so
|
||||
// this path must bill images the Anthropic way: ceil(w*h/750 * 1.10).
|
||||
// 768x1932 → ceil(768*1932/750 * 1.10) = ceil(2176.8) = 2177.
|
||||
expect(visionTokensForModel('claude-opus-4-8', 768, 1932)).toBe(2177);
|
||||
// this path must bill images the Anthropic way: ⌈w/28⌉×⌈h/28⌉ visual tokens.
|
||||
// 768x1932 on the high-res tier (opus-4-8) fits without downscale →
|
||||
// ⌈768/28⌉×⌈1932/28⌉ = 28×69 = 1932.
|
||||
expect(visionTokensForModel('claude-opus-4-8', 768, 1932)).toBe(1932);
|
||||
// GPT models are unchanged (delegates to openAIVisionTokens).
|
||||
expect(visionTokensForModel('gpt-5', 768, 1932)).toBe(openAIVisionTokens('gpt-5', 768, 1932));
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user