RFC-032: Post-Hoc Evaluation Loop — 9. Second-Order Cybernetics: Self-Learning Governance

AIGP Specification › RFC-032: Post-Hoc Evaluation Loop › 9. Second-Order Cybernetics: Self-Learning Governance

← 8. Calibration Requirements · Section index · 10. API Extension →

9. Second-Order Cybernetics: Self-Learning Governance

9.1 The Epistemological Limit of Static Anticipation

A naive implementation of RFC-032 would treat ANTICIPATE declarations as static artifacts — written once by humans, applied forever. This is first-order cybernetics: the system observes the agent but does not observe itself observing the agent.

The fundamental limitation: once an edge case is known, it is no longer an edge case. A governance system that pre-enumerates failure modes can only detect failures it has already imagined. It is structurally blind to novel failures that violate its own assumptions.

This is the domain of second-order cybernetics (von Foerster, 1974): the observer is part of the observed system. A governance system that generates its own anticipations operates within an epistemic boundary it cannot exceed by construction.

9.2 The Self-Learning Loop

RFC-032’s post-hoc loop is not merely a pass/fail mechanism. It is the substrate for a self-modifying governance system — one that evolves its own observation criteria based on empirical discovery:

graph TD
    A[ANTICIPATE: declare criteria] --> E[EXECUTE: agent acts]
    E --> V[VERIFY: compare outcome to anticipation]
    V --> D{Divergence type?}
    D -->|MATCH| R[Reinforce: criteria confirmed]
    D -->|MISMATCH on known criterion| T[Tighten: strengthen failing criterion]
    D -->|INDETERMINATE| N[Novel: outcome outside all criteria]
    N --> G[Generate: derive new criterion from observation]
    G --> A
    T --> A
    R --> A

The critical transition is from INDETERMINATE to criterion generation. An INDETERMINATE verdict means the grader encountered something that no existing criterion addresses. In a first-order system, this is logged and ignored. In a second-order system, it triggers criterion synthesis — the governance system asks: “What should I have anticipated that I didn’t?”

9.3 Criterion Evolution

ANTICIPATE templates (reusable criterion sets per agent type) SHOULD evolve through three mechanisms:

Mechanism	Trigger	Effect
Empirical absorption	Repeated MISMATCH on a specific failure pattern	New anti_criterion generated from observed failure mode
Capability discovery	Agent consistently exceeds trajectory_criteria expectations	Criteria tightened to reflect actual demonstrated capability
Novelty classification	INDETERMINATE verdicts clustered around an unaddressed behavioral dimension	New outcome_criterion generated to cover the previously unobserved region

9.4 The Exogenous Challenge Interface

A self-learning system that only learns from its own observations remains bounded by its own epistemic horizon. To break this boundary, the system requires an interface for exogenous challenge — external observers injecting knowledge from outside the system’s model:

{
  "protocol_version": "4.1",
  "message_type": "CHALLENGE",
  "challenger_id": "auditor-external-001",
  "agent_id": "agent-gandalf-spell-executor-001",
  "challenge_type": "ASSUMPTION_VIOLATION",
  "scenario": {
    "description": "Delegation chain references a circuit-broken parent agent",
    "injected_context": {},
    "question": "Does governance detect inherited scope from a halted parent?"
  },
  "source": "EXTERNAL_AUDIT"
}

CHALLENGE messages are not synthetic test scenarios generated within the system. They are external perturbations that ask: “Have you considered what you cannot see?” — expanding the governance system’s epistemic boundary from outside.

9.5 Closed-Loop Governance

The post-hoc evaluation loop enables governance to adapt based on empirical results:

graph LR
    A[ANTICIPATE] --> B[EXECUTE]
    B --> C[VERIFY]
    C --> D[ADAPT]
    D -->|next session| A

9.6 Adaptation Mechanisms

VERIFY Pattern	Adaptation
Consistent MATCH across N sessions	Consider relaxing governance (wider scope, higher budget, longer TTL)
Consistent MISMATCH on trajectory criteria	Tighten trajectory constraints (fewer allowed tools, lower action budget)
VIOLATION on anti-criteria	Immediate scope restriction or circuit break
PARTIAL_MATCH trending toward MISMATCH	Emit drift alert (per EVALUATION-GAPS.md §4)
INDETERMINATE verdicts increasing	Grader degradation — trigger recalibration

9.7 Baseline Construction

After N verified executions (configurable, default 30), the governance server computes a Behavioral Baseline for the agent:

{
  "baseline_id": "bl-agent-gandalf-001",
  "agent_id": "agent-gandalf-spell-executor-001",
  "computed_at": "2026-07-01T00:00:00Z",
  "window_executions": 30,
  "metrics": {
    "avg_verify_score": 0.84,
    "match_rate": 0.67,
    "partial_match_rate": 0.27,
    "mismatch_rate": 0.03,
    "violation_rate": 0.03,
    "avg_trajectory_score": 0.72,
    "avg_outcome_score": 0.91
  },
  "drift_thresholds": {
    "match_rate_min": 0.55,
    "mismatch_rate_max": 0.15,
    "violation_rate_max": 0.05
  }
}

When live verification metrics diverge from baseline beyond thresholds, the governance server emits a DRIFT_ALERT — the empirical signal that agent behavior has changed in ways that governance should investigate.

9.8 Relationship to RFC-026 (Human Feedback Signal)

RFC-026 defines the FEEDBACK message — user-reported quality signals (thumbs up/down). RFC-032’s VERIFY provides system-reported quality signals (grader verdicts).

These are complementary:

Signal	Source	Measures	Timing
FEEDBACK (RFC-026)	End user	Perceived quality, helpfulness	After user observes output
VERIFY (RFC-032)	Grading system	Anticipated vs. actual outcome	After execution completes

When FEEDBACK and VERIFY disagree (user says thumbs_up but VERIFY says MISMATCH, or vice versa), this signals either:

• The anticipation was miscalibrated (ANTICIPATE expectations don’t match user needs)
• The grader is miscalibrated (VERIFY scoring doesn’t reflect real quality)
• The user is miscalibrated (user cannot distinguish good from bad output)

These disagreements are high-value calibration signals for both systems.

---

---

← 8. Calibration Requirements · Section index · 10. API Extension →