RFC-032: Post-Hoc Evaluation Loop — 17. Temporal Composition + Causal Attribution

AIGP Specification › RFC-032: Post-Hoc Evaluation Loop › 17. Temporal Composition + Causal Attribution

← 16. Quality Moderator — Consumer Tier Deliverable · Section index

17. Temporal Composition + Causal Attribution

17.1 Problem

RFC-032 defines ANTICIPATE/VERIFY for individual executions. The Quality Moderator produces per-interval risk postures. But longitudinal governance — detecting patterns across time, attributing degradation to root causes, and routing corrective actions — requires a composition discipline that transforms per-interval VERIFYs into recomputable longitudinal verdicts.

Without composition:

A system can be EXCELLENT per-interval but degrading over time (trend invisible)
Root causes remain unattributed (“quality is dropping” without “because of X”)
Corrective actions are manual and ad-hoc rather than protocol-driven

17.2 Composition Discipline

17.2.1 Longitudinal Verdict Definition

A longitudinal verdict is a composed finding derived from multiple per-interval VERIFY results:

LONGITUDINAL_VERDICT = compose(VERIFY_1, VERIFY_2, ..., VERIFY_n)

Composition preserves soundness-anchor inheritance: each per-interval VERIFY carries an immutable reference to its ANTICIPATE declaration. The longitudinal verdict inherits all per-interval soundness anchors, making the composed finding independently recomputable from retained evidence alone.

17.2.2 Composition Methods

Method	Semantics	Use Case
`weighted_temporal_average`	Score = Σ(weight_i × score_i) / Σ(weight_i), with recency weighting	Smooth trend detection
`threshold_crossing_count`	Count of intervals where score crossed a threshold boundary	Volatility detection
`monotonic_degradation`	Detect N consecutive intervals of declining score	Systematic drift
`pattern_correlation`	Correlate score changes with metadata changes (model version, prompt update)	Causal signal

17.2.3 LONGITUDINAL_VERDICT Message Schema

{
  "protocol_version": "5.0",
  "message_type": "LONGITUDINAL_VERDICT",
  "verdict_id": "lv-{uuid}",
  "app_id": "APP_WATCHTOWER",
  "session_id": "ses-longitudinal-001",
  "interval_refs": ["ver-001", "ver-002", "ver-003", "ver-004", "ver-005"],
  "interval_count": 5,
  "time_range": {
    "start": "2026-07-01T00:00:00Z",
    "end": "2026-07-22T00:00:00Z"
  },
  "composition_method": "monotonic_degradation",
  "order_type": "causal",
  "overall_verdict": "DEGRADED",
  "overall_score": 0.62,
  "trend": "declining",
  "trend_slope": -0.04,
  "attributed_causes": [...],
  "recommended_actions": [...],
  "soundness_anchors": ["ant-001", "ant-002", "ant-003", "ant-004", "ant-005"],
  "recomputation_inputs": {
    "interval_evidence_refs": ["evidence-001", "evidence-002", "evidence-003", "evidence-004", "evidence-005"],
    "composition_parameters": {"method": "monotonic_degradation", "window_size": 5}
  },
  "d_dna_signature": "sha256:{signature}"
}

17.3 Order-Type Taxonomy

Each longitudinal verdict MUST declare which order of analysis it represents:

Order Type	Description	Example Finding
`prescriptive`	What SHOULD happen (policy/doctrine compliance)	“Model responses deviate from doctrine template 40% of time”
`procedural`	How it SHOULD be done (process adherence)	“Human-in-the-loop step skipped in 3 of 5 intervals”
`descriptive`	What IS happening (observed behavior)	“Response length increasing 15% per interval”
`normative`	What is ACCEPTABLE (threshold governance)	“Score below ACCEPTABLE threshold for 3 consecutive intervals”
`temporal`	How behavior changes OVER TIME (drift/trend)	“Sycophancy score rising monotonically over 10 sessions”
`causal`	WHY it happened (root cause attribution)	“Degradation correlated with prompt template v3 deployment”
`relational`	How entities RELATE to each other (composition effects)	“Agent A’s outputs degrade when consuming Agent B’s context”

17.4 Causal Attribution Schema

Each longitudinal verdict MAY carry one or more causal attributions identifying the root cause of the finding:

{
  "attributed_causes": [
    {
      "target": "prompt",
      "confidence": 0.82,
      "evidence_refs": ["ver-003", "ver-004", "ver-005"],
      "reasoning": "Degradation onset correlates with prompt template v3 deployment on 2026-07-10",
      "metadata_correlation": {
        "field": "prompt_hash",
        "before_value": "sha256:old_prompt",
        "after_value": "sha256:new_prompt",
        "change_date": "2026-07-10T00:00:00Z"
      }
    }
  ]
}

17.4.1 Attribution Targets

Target	Description	Signal Source
`data`	Training data, knowledge base content, retrieval corpus	KB version changes, data freshness, retrieval drift
`user`	User behavior patterns (repeated misuse, prompt injection attempts)	Per-user score distribution, interaction patterns
`prompt`	System prompts, prompt templates, instruction sets	Prompt hash changes correlated with score changes
`model`	Model version, capability gaps, alignment drift	Model version changes, capability benchmarks

17.4.2 Attribution Confidence

confidence ∈ [0.0, 1.0]: degree of certainty that this target is the root cause
Confidence < 0.5: attribution is speculative → action = investigate
Confidence ∈ [0.5, 0.8): attribution is probable → action determined by target
Confidence ≥ 0.8: attribution is strong → automatable action may fire

17.5 Reinforcement Action Taxonomy

Causal attributions bind to governance actions. Each finding routes to a specific corrective mechanism:

Action	Trigger	Routed To	Automatable	Description
`sync_knowledge_base`	`data` attribution + stale KB	KB maintainer	Yes	Refresh knowledge base from authoritative source
`coach_user`	`user` attribution + repeated pattern	User’s supervisor	Partially	Provide guidance on proper system usage
`review_prompt`	`prompt` attribution + quality drop	Engineering authority	No	Review and potentially revert prompt changes
`swap_model`	`model` attribution + capability gap	Program office	No	Evaluate alternative model for the use case
`tune_guardrail`	`model` attribution + over/under-filter	Governance authority	Yes	Adjust guardrail sensitivity thresholds
`investigate`	Insufficient confidence (< 0.5)	Named authority	No	Manual investigation required

17.5.1 Action Routing

{
  "recommended_actions": [
    {
      "action": "review_prompt",
      "priority": "HIGH",
      "routed_to": "engineering-authority",
      "automatable": false,
      "trigger_attribution": {
        "target": "prompt",
        "confidence": 0.82
      },
      "deadline": "2026-07-25T00:00:00Z",
      "escalation_on_miss": "program-office"
    }
  ]
}

17.5.2 Non-Automatable Default

The investigate action is the mandatory fallback when:

Attribution confidence is below 0.5 for all targets
Multiple targets have similar confidence (ambiguous root cause)
The finding is novel (no prior pattern match)

This prevents false automation — the system never takes corrective action on uncertain attribution.

17.6 Quality Moderator → Action Binding

The Quality Moderator’s risk posture (§16) binds to reinforcement actions through threshold triggers:

Risk Posture	Score Range	Trigger Behavior
EXCELLENT	≥ 0.95	No action. Monitoring only.
ACCEPTABLE	[0.80, 0.95)	Log for trend analysis. No immediate action.
DEGRADED	[0.60, 0.80)	Compose longitudinal verdict. If causal attribution confidence ≥ 0.5, route reinforcement action.
FAILING	[0.40, 0.60)	Immediate reinforcement action required. Escalate to named authority regardless of attribution.
CRITICAL	< 0.40	`investigate` + circuit breaker. Halt automated operations pending review.

17.7 Gap Governance State Machine

Longitudinal findings surface as gap signals — structured representations of governance gaps that require resolution. Gaps are managed by a state machine (not a dashboard alert).

17.7.1 States

State	Description	SLA
`open`	Gap identified; not yet assigned	Assignment within 24h
`in_resolution`	Owner assigned; working on fix	Resolution target set at assignment
`resolved`	Gap closed; evidence of resolution recorded	—
`declared_permanent`	Gap cannot be resolved; accepted risk documented	Review annually

17.7.2 Transitions

[open] --assign_owner(owner_id, target_date)--> [in_resolution]
[open] --timeout(24h)--> [escalated → open with higher priority]

[in_resolution] --resolve(evidence)--> [resolved]
[in_resolution] --declare_permanent(justification)--> [declared_permanent]
[in_resolution] --timeout(target_date)--> [escalated → in_resolution with higher priority]

[resolved] -- (terminal, no transitions out)
[declared_permanent] -- (terminal, annual review trigger)

17.7.3 Gap Signal Schema

{
  "gap_id": "gap-{uuid}",
  "source_verdict_ref": "lv-{uuid}",
  "gap_type": "quality_degradation | coverage_deficit | attribution_failure | compliance_drift",
  "severity": "LOW | MEDIUM | HIGH | CRITICAL",
  "owner": null,
  "state": "open",
  "created_at": "2026-07-22T12:00:00Z",
  "resolution_target_date": null,
  "resolution_evidence_ref": null,
  "permanent_justification": null
}

17.8 Recomputability Requirement

A longitudinal verdict is recomputable if and only if:

All interval_refs (per-interval VERIFY IDs) are resolvable to their full VERIFY messages
Each VERIFY’s anticipation_ref resolves to an immutable ANTICIPATE declaration
The composition_parameters are sufficient to reproduce the composition algorithm
The soundness_anchors chain is unbroken (no missing or tampered links)

An independent inspector MUST be able to:

re_derived_verdict = compose(
    retrieve(interval_refs),
    composition_parameters
)
assert re_derived_verdict.overall_score == original_verdict.overall_score

← 16. Quality Moderator — Consumer Tier Deliverable · Section index