Phase 10 · v1.1 · Shipped 2026-05-26

UI evidence & deterministic triage.

Robots capture uiEvidence before validating events. A pure classifier maps (uiEvidence × rule × outcome) → failureKind. Extends Phase 7's enum with param-mismatch, duplicate, under-counted, unclassified. The headline BI win: kills the false-positive class where "article has no gallery → gallery_view missing" gets reported as tracking-broken.

Gathered2026-05-26 ModeBehavioral change Depends onPhase 9 · Phase 12
Phase boundary

Two coupled changes; everything else stays.

1. UI evidence capture per page load. 2. Deterministic triage classifier extending Phase 7's FailureKind. Never silently passes a row.

Out of scope: Slack output (Phase 11), the rule schema itself (Phase 9), the inventory (Phase 8), new/repeated dedup (Phase 11).

Decisions

Eight design questions, every answer integrated with cross-phase findings.

Q1 · UI evidence schema · expanded

Eleven base fields + CNC-specific signals from self-review.

export type UiEvidence = {
  pageType: PageType | "unknown";
  device: "desktop" | "mobile" | "tablet";
  hasArticleBody, hasGallery, hasVPlayer, hasVideo: boolean;
  hasPaywall, hasPremiumBadge: boolean;
  isLoggedIn, isPremiumUser: boolean;
  consentAccepted, modalBlocking: boolean;
  // CNC-specific (added in review)
  consentProvider: "cpex" | "didomi" | "other";
  paywallVariant: "hard" | "soft" | "metered" | null;
  collectedAt: string;
  selectorHits: Record<string, boolean>;
};

consentProvider routes Phase 12 Z3's TestError classifier correctly. paywallVariant: "metered" shouldn't expect paywall_view until the meter trips.

Q2 · Robot signature change

Append one positional param at the tail.

CLAUDE.md says "fixed positional signature" — preserved by appending, not changing existing positions. Verified against src/robots/homepage.ts and pagination.ts — both accept trailing optional params already (dynamic?: boolean).

Alternative considered: read uiEvidence from a module-level singleton. Rejected — makes the call site implicit, breaks the explicit positional contract robots already have.

Q3 · Selector packs · per-property JSON

Plus a nightly evidence:selftest for rot.

{
  "hasGallery": ["[data-role='gallery']", "//div[contains(@class,'gallery-component')]"],
  "hasVPlayer": ["#vplayer-instance", "[data-vplayer]"],
  "hasPaywall": [".paywall-overlay", "#cnc-paywall"],
  "isPremiumUser": ["[data-user-tier='premium']"]
}

Cross-property defaults in _default.json. npm run evidence:selftest <propertyKey> probes every selector on a known-good URL nightly; warns on ≥30% miss.

Q4 · Triage precedence · strict order

First-match-wins; never silently passes.

#ConditionVerdict
0uiEvidence === undefined or status:"collection-failed"Fall through to Phase 7 behavior
1Technical pre-check failed (consent, login, timeout, modal blocking)robot-broken / timeout / site-broken
2Rule's appliesWhen.uiEvidence doesn't matchoutcome: "rule-not-applicable" (non-failure)
3Rule applies, expected event missingtracking-broken
4Event present, count rule violated (too many)duplicate
4bEvent present, count rule violated (too few)under-counted
5Event present, count OK, required param missing & strictparam-mismatch
6All checks passpassed: true
7No rule matched, no technical failureunclassified → triage

Edge case documented: a row with BOTH technical failure AND tracking miss reports technical (branch 1 wins). Once consent is blocking, no tracking conclusion is trustworthy.

Q5 · New FailureKind values

param-mismatch · duplicate · under-counted · unclassified.

Keeping duplicate and under-counted separate (not merged into count-mismatch) — they route differently in Phase 11: duplicate inflates BI numbers and is urgent; under-counted may be a timing issue and is investigative.

Q6 · consent & modal as authoritative

If the overlay still intercepts pointers after retry, it's robot-broken — not tracking-broken.

Evidence collector retries consent once, then sets modalBlocking: true. Triage branch 1 short-circuits the row.

Q7 · Backward compat

Rows without uiEvidence behave like Phase 7.

Every classifier read of uiEvidence is guarded by if (!uiEvidence) return phase7Classify(...). EV2-parity test asserts bytewise equality for fixture rows with EVENT_RULES_V2=false.

Q8 · Collector failure mode

Never crashes the run.

collectUiEvidence catches its own errors and returns {status:"collection-failed",error:"..."}. Classifier branch 0 falls through to Phase 7.

Success criteria

Including the headline BI regression test.

  1. uiEvidence populated on every ReportDetailRow written after Phase 10 ships.
  2. Triage classifier truth table test covers all 8 precedence branches with at least one fixture each.
  3. False-positive regression test: on a fixture page with no gallery, the gallery_view row is outcome: "rule-not-applicable", NOT a tracking failure. Primary BI ask.
  4. Duplicate detection test: subscription_click fixture with 2 captured events triggers failureKind: "duplicate".
  5. Unclassified bucket test: a captured event with no matching rule produces an unclassified row — proves we never silently drop.
  6. With EVENT_RULES_V2=false, behavior is byte-identical to Phase 7 — proven by a regression test running both paths against the same fixture.
  7. npm run check + npm run test:logic pass; no new npm deps.
Files

Collector, classifier, selector packs, parity test.

newsrc/lib/uiEvidence.ts
newdata/ui-evidence-selectors/_default.json + <propertyKey>.json × 12+
newsrc/utils/triageClassifier.ts
modsrc/lib/Types.tsUiEvidence type, extended FailureKind
modsrc/utils/eventUtils.tsdelegate to classifier under flag
modsrc/bi-measurement.tscollect evidence after consent
modsrc/robots/*.tsaccept uiEvidence positional param
newtests/triage-classifier.test.js, ui-evidence.test.js, ev2-parity.test.js
newdocs/triage-rules.mdBI-facing decision tree
Review

Self-review (codex CLI rate-limited at retry time).

Reviewed against the same 7-point checklist using cross-cutting findings from the Phase 9 / 11 / 12 codex reviews.

i

Schema gaps closed

SELF · INTEGRATED

Added consentProvider (cpex / didomi / other) — different surfaces have different cleanup paths. Added paywallVariant (hard / soft / metered) — different UX produces different expected event sequences.

Positional contract preserved

SELF · VERIFIED

Appending one positional param doesn't violate the CLAUDE.md hard-no; existing robots already accept trailing optional params (dynamic?: boolean). Module-level state alternative rejected as too implicit.

i

Collector robustness

SELF · INTEGRATED

Original draft didn't specify what happens when collectUiEvidence itself throws. Added {status:"collection-failed"} sentinel + branch 0 in the classifier.

i

EV2-parity maintenance burden

SELF · RESOLVED

Parity test fixtures are frozen eventStorage.json snapshots. New rules added to data/event-rules.json don't invalidate them — with EVENT_RULES_V2=false, the runtime reads the old mapping. Test owners maintain snapshots, not BI authors.

← Previous Phase 9 · Event rules catalog Next → Phase 11 · Slack routing & templates