# ADR-101: Pose Estimation Cog (WiFi-DensePose side)

- **Status:** Accepted — **v0.0.1 shipped 2026-05-19** (merged in PRs #642 + #643, signed binaries on GCS, live install on cognitum-v0)
- **Date:** 2026-05-19
- **Deciders:** ruv
- **Companion ADR (v0-appliance side):** v0-appliance ADR-225 (cognitum-pose-estimation crate)

## Context

ADR-079 designed the 17-keypoint COCO pose-estimation training pipeline. ADR-100 formalised the Cognitum Cog packaging spec. This ADR is the bridge: it specifies how the wifi-densepose training pipeline produces an artifact that ships as a Cog (`cog-pose-estimation`) onto the Cognitum V0 appliance and out to the Pi+Hailo cluster.

It is the next product step beyond the published `presence` Cog (binary head trained from the contrastive encoder on Hugging Face at `ruvnet/wifi-densepose-pretrained`). Where `presence` reports a single boolean per tick, `cog-pose-estimation` reports 17 (x, y) keypoints per person, per tick.

## Decision

### Pipeline

```
                         (training side — ruvultra GPU)
ESP32 / rvcsi  ─►  collect-ground-truth.py + sensing-server recording
                         │
                         ▼
                   data/paired/*.paired.jsonl   (CSI window + camera keypoints)
                         │
                         ▼
                   v2/crates/wifi-densepose-train  ──►  Rust + libtorch trainer
                   (uses RTX 5080 / CUDA 12.x)         │
                   init from ruvnet/wifi-densepose-pretrained
                                                       │
                                                       ▼
                                                  model.safetensors  (encoder + pose head)
                                                       │
                                          ─────────────┴─────────────
                                          │                         │
                                          ▼                         ▼
                                  v2/crates/cog-pose-estimation     export to ONNX
                                  (this repo)                       │
                                   • emits manifest.json            ▼
                                   • produces cog binary       cognitum-hailo
                                   • signs + uploads to GCS    (v0-appliance side)
                                                                    │
                                                                    ▼
                                                           cog-pose-estimation.hef
                                                                    │
                                                                    ▼
                              (appliance side — cognitum-v0 + Pi+Hailo cluster)
                                                           
                              gs://cognitum-apps/cogs/{arm,hailo8,hailo10}/cog-pose-estimation-<arch>
                                                                    │
                                                                    ▼
                              `cognitum-cog-gateway` pulls artifact + manifest, verifies signature, installs
                              into /var/lib/cognitum/apps/pose-estimation/
                                                                    │
                                                                    ▼
                              run loop: read CSI frames from local sensing-server
                              → encoder → pose head → emit `{ts, persons: [{keypoints: [...17 x,y...] }]}`
                              on stdout as the Cog runtime contract requires
```

### Architecture (model)

| Stage | Module | Notes |
|-------|--------|-------|
| Input | `[56 subcarriers × 20 frames]` per CSI window | matches today's `data/paired/wiflow-p7-*.paired.jsonl` |
| Encoder | TCN-lite or contrastive encoder lifted from HF presence model | 128-dim embedding; weights init from `ruvnet/wifi-densepose-pretrained/model.safetensors` |
| Pose head | 2-layer MLP `(128 → 256 → 34)` | 34 = 17 × (x, y) |
| Output | `[B, 17, 2]` keypoints in `[0, 1]` image-normalised coords | confidence is implicit in keypoint variance over time; ADR-079 P9 will add explicit per-joint confidence |
| Loss | Confidence-weighted SmoothL1 (frame-level) + bone-length regulariser + temporal smoothness | per ADR-079 Phase 3 refinement |
| Init | Encoder = HF presence weights (frozen for 50 epochs, then jointly fine-tuned) | unblocks the sigmoid-saturation failure mode observed in #645 |
| Training | `v2/crates/wifi-densepose-train` with libtorch backend on RTX 5080 | replaces the pure-JS SPSA trainer that produced 0% PCK in #645 |

### Repo layout

```
v2/crates/cog-pose-estimation/        # NEW (this ADR)
├── Cargo.toml
├── src/
│   ├── main.rs                # CLI: run | health | version | manifest
│   ├── lib.rs
│   ├── inference.rs           # ONNX runtime + Hailo HEF runtime dispatch
│   ├── frame_subscriber.rs    # local sensing-server subscriber
│   └── publisher.rs           # emits structured JSON events per Cog contract
├── cog/
│   ├── manifest.template.json
│   ├── config.schema.json
│   ├── README.md
│   ├── icon.svg
│   └── Makefile               # build-arm | build-x86_64 | sign | upload
└── tests/
    ├── manifest_signature.rs
    └── inference_smoke.rs
```

### Runtime contract

Honours ADR-100's per-Cog CLI contract:

- `cog-pose-estimation version` → `pose-estimation 0.0.1`
- `cog-pose-estimation manifest` → JSON
- `cog-pose-estimation health` → 0 if encoder+head load and a synthetic frame produces a finite output
- `cog-pose-estimation run --config /etc/cognitum/cogs/pose-estimation/config.json` → long-running; emits one JSON event per inferred frame:

```json
{
  "ts": 1779210883.444,
  "level": "info",
  "event": "pose.frame",
  "fields": {
    "tick": 12345,
    "n_persons": 1,
    "persons": [
      {"keypoints": [[0.48, 0.31], [0.52, 0.28], ...], "confidence": 0.81}
    ]
  }
}
```

### Hardware deployment

| Target | arch | runtime | notes |
|--------|------|---------|-------|
| ruvultra (dev) | `x86_64` | ONNX Runtime CPU/CUDA | development & smoke tests |
| cognitum-v0 (Pi 5) | `arm` | ONNX Runtime ARM | reference deploy; ~20 ms/frame |
| Pi + Hailo-8 hat | `hailo8` | Hailo HEF runtime via `cognitum-hailo` | ~2 ms/frame, 26 TOPS budget |
| Pi + Hailo-10 hat | `hailo10` | Hailo HEF runtime via `cognitum-hailo` | ~1 ms/frame, 40 TOPS budget |

### Acceptance gates

1. **Validates:** `cargo test -p cog-pose-estimation` green; `cog-pose-estimation health` returns 0 against a synthetic CSI window.
2. **Benchmarks:** end-to-end frame latency on each target arch logged in `target/criterion/`; published in `docs/benchmarks/pose-estimation-cog.md`.
3. **Optimised:** the Hailo-targeted ONNX graph passes through Hailo Dataflow Compiler without quantisation-aware-training warnings.
4. **Published:** signed binary at `gs://cognitum-apps/cogs/<arch>/cog-pose-estimation-<arch>`; manifest valid against the JSON schema in ADR-100; appliance installer can pull and run it.

PCK@20 is intentionally **not** an acceptance gate of this ADR. Achieving the ADR-079 ≥35% target is a separate, data-bound milestone tracked in #645. This ADR ships the **vehicle**, not the model accuracy.

### First measured run — v0.0.1 (2026-05-19)

A Candle-on-CUDA training run on `ruvultra`'s RTX 5080 against the same 1,077-sample paired session that produced the 0%/0% baseline in #645 yielded:

- **PCK@20 = 3.0%**, **PCK@50 = 18.5%**, **MPJPE = 0.093** (normalized).
- 400 epochs in **2.1 s** wall time (~5 ms/epoch, full-batch).
- Loss reduction 13× (0.181 → 0.014, eval 0.010).
- Strongest signal at `r_hip` (PCK@50 = 76.9%), `r_knee` (35.2%), `l_elbow` (26.4%).

This confirms the pipeline trains end-to-end and produces a signal-bearing model. The remaining gap to PCK@20 ≥ 35% is data-bound (1,077 samples is ≪ the ADR-079 target of ~30K). See `docs/benchmarks/pose-estimation-cog.md` for the full result dump.

## Consequences

### Positive

- First Cog from this repo that integrates with the appliance/cog-gateway pipeline. Future cogs (e.g. `cog-vitals`, `cog-fall-alert`) follow the same template.
- Closes the loop from data collection → training → quantisation → cluster deployment with a single repo-anchored artifact.
- Forces a real signature on cog binaries (per ADR-100), which improves supply-chain hygiene across the whole appliance.

### Negative

- Adds a hard dependency on the Hailo Dataflow Compiler, which lives behind a self-hosted runner — Hailo-targeted PRs land more slowly.
- The first published binary will have low PCK (data + training time gap, #645) — UX needs to surface this clearly so end users do not interpret bad keypoints as a bug.

### Risks

- **Model size on Hailo**: the encoder fits comfortably in Hailo-8's on-chip SRAM, but the pose-head expansion to `[17×2]` plus required temporal stacking pushes us close to the Hailo-8 envelope. Mitigation: Hailo-10 path is the primary deploy target; Hailo-8 is a stretch.
- **Sensing-server schema drift**: the cog subscribes to `/api/v1/sensing/latest` JSON. If the appliance's sensing-server schema changes, the cog fails open (logs warning, emits nothing). The `frame_subscriber.rs` module pins to schema version `2`.

## Migration / rollout

1. Land this ADR + ADR-100 on `main` of RuView.
2. Land companion ADR-225 + crate on `main` of v0-appliance.
3. First release `cog-pose-estimation@0.0.1` ships **only** to `ruvultra` and `cognitum-v0`. Not pushed to the cluster Pis yet.
4. After P7→P9 data work (#645) brings PCK above a usable threshold, rebuild + re-publish; only then enable cluster rollout via `cognitum-cog-gateway`'s OTA channel.

## v0.0.1 shipping status — 2026-05-19

PRs `#642` (scaffold + arm release + ONNX + live install) and `#643` (x86_64 release) landed on `main`. Acceptance gates from ADR-100 met as follows:

| Gate | Status |
|------|--------|
| Cog binary exists per arch | ✅ arm (`3,741,976 B`) + x86_64 (`4,548,856 B`) on GCS |
| Manifest matches schema | ✅ `cog/artifacts/manifests/{arm,x86_64}/manifest.json` |
| Binary sha256 + Ed25519 signature | ✅ both signed with `COGNITUM_OWNER_SIGNING_KEY`, round-trip verified |
| Public-readable GCS | ✅ anonymous HTTP GET works, SHA matches |
| Live install on a real appliance | ✅ `/var/lib/cognitum/apps/pose-estimation/` on `cognitum-v0` (Pi 5), same layout as `anomaly-detect` |
| Runtime contract (`version \| manifest \| health \| run`) | ✅ all four return correct output; `run` emits `pose.frame` events |
| Real weights loaded (not stub) | ✅ `cargo test` asserts `backend.starts_with("candle-")` + non-zero confidence |
| ONNX artifact (for downstream HEF) | ✅ `pose_v1.onnx` (12 KB), parity vs torch = 8.94e-8 |

| Metric | Value |
|--------|-------|
| Training time (RTX 5080 / Candle CUDA) | 2.1 s for 400 epochs |
| PCK@20 / PCK@50 / MPJPE (1,077-sample seated-desk session) | 3.0% / 18.5% / 0.093 |
| Cold-start: Windows x86_64 | 76 ms |
| Cold-start: ruvultra x86_64 | **5.4 ms** |
| Cold-start: Pi 5 aarch64 | **8.4 ms** |
| Tests | 5/5 pass |

Open follow-ups carried forward from this ADR's "Acceptance gates" section:

- **Hailo HEF cross-compile** — `pose_v1.onnx` is ready; still gated on Hailo Dataflow Compiler + self-hosted runner provisioning. Tracked separately.
- **PCK@20 ≥ 35%** — explicitly not an acceptance gate of this ADR, but the limiting factor on practical usefulness. Tracked in [#645](https://github.com/ruvnet/RuView/issues/645): needs ~30× more paired samples + multi-room camera framing. Today's seated-desk session is the demonstrated bottleneck.

## See also

- ADR-079: Camera-supervised pose training pipeline (the model we're shipping).
- ADR-100: Cog packaging specification (the format we're shipping in).
- v0-appliance ADR-225: cognitum-pose-estimation crate (the appliance-side runtime).
- v0-appliance ADR-220: cog management surface (where this cog appears in the dashboard).
- Issue #645: PCK gap (current 3% / 18.5% → ≥35% target).
- `docs/benchmarks/pose-estimation-cog.md`: full benchmark log, all measured numbers.