The Age of Synthetic Realities: Challenges and Opportunities

Preface: Cybernetics and the Limits of Outcome Feedback

Cybernetics emerged in the mid-twentieth century as a general theory of control through feedback. Norbert Wiener described feedback as a method of governing a system by reinserting the results of past performance into the system itself.

The central cybernetic insight is that stability can be achieved when a system continuously measures its outputs, compares them with a desired state, and adjusts its behavior accordingly. The canonical illustration is the thermostat, which maintains a target temperature by measuring deviations from that target and activating or deactivating heating or cooling in response.

Cybernetics thus established a general framework for understanding control processes in mechanical, biological, and social systems. In this framework, the essential stabilizing mechanism is feedback from outcomes.

Contemporary predictive AI systems can be understood as large-scale cybernetic apparatuses. As the literature on Human-Aided AI shows, these systems rely on continuous feedback from human behavior: clicks, purchases, shares, and other reactions serve as signals that recalibrate predictive models.

However, the architecture of modern AI systems introduces a structural feature that was largely absent from the classical cybernetic examples: perception itself is mediated. Predictive algorithms filter, rank, and structure information before it becomes available to human reasoning or decision-making. In such systems, feedback is no longer derived from direct observation of the underlying environment but from outcomes generated by mediated representations of that environment.

This structural difference introduces a critical limitation in the cybernetic paradigm.

Introduction: Mediated Perception and the Limits of Outcome Feedback

Classical cybernetic systems assume that feedback signals retain sufficient information about the system’s state to guide corrective action. In physical control systems such as thermostats or fire-control apparatuses, this assumption is largely justified: the measured outcome directly reflects the physical state of the system.

In mediated intelligence systems, the relationship between reality and outcome is indirect. Instead of observing reality directly, the system operates through an intermediate representational layer:

where

denotes latent environmental states,
denotes mediated representations produced by filtering, extraction, or modelling processes, and
Denotes observable outcomes.

In such systems, feedback mechanisms operate primarily on , the outcome layer. However, the representational layer may already omit constraints or distort relationships present in the underlying environment.

Information theory makes this limitation explicit. If reality, representation, and outcome form the sequence.

Then the data processing inequality implies:

Outcome signals, therefore, contain no more information about the underlying environment than the mediated representation from which they were derived. Often they contain substantially less.

Consequently, feedback mechanisms that rely solely on outcomes cannot in general reconstruct information lost during mediation. Distinct mediation processes may produce identical observable outcomes within a given observational regime. In such cases, outcome-based reflection cannot distinguish between accurate and distorted representations.

The system may therefore converge toward configurations that are operationally stable—producing consistent and optimized outcomes—while remaining misaligned with the underlying environment.

This relationship can be expressed structurally as:

Traditional cybernetic control assumes a feedback loop of the form:

However, in mediated intelligence systems, the control loop must account for the perceptual layer itself. Stabilization therefore requires reflection not only on outcomes but also on the processes that generate perceptual input:

Toward a Structural Law of Mediated Intelligence Systems

These observations motivate a broader principle governing mediated intelligence systems.

Where classical cybernetics assumes that outcome feedback is sufficient for stability, mediated intelligence systems require reflection across multiple layers of the system architecture.

This principle can be expressed as follows:

Second Law of Mediated Intelligence Systems

Stability in a mediated intelligence system requires reflection across both operational outcomes and the mediation processes that determine perceptual inputs. Systems that reflect only on outcomes may converge toward internally consistent yet potentially distorted representations of reality.

This law follows from the informational structure of mediated systems. Because outcome signals are informationally downstream of mediation, reflection confined to outcomes operates on signals that may already lack critical constraints present in the underlying environment.

Formally, this means that optimization over outcome performance alone may converge to states satisfying

While mediation distortion remains non-zero:

where mediation distortion may be expressed as

Representing the constraint loss introduced during mediation.

Relationship to the First Law

The first law of mediated intelligence systems describes the dynamic consequence of this architecture: when perceptual mediation scales faster than reflective correction, divergence across representations increases.

The second law describes the structural condition required to prevent such divergence. Together, these principles define the stability conditions for intelligence systems operating under mediated perception.

Cybernetics, Variety, and the Limits of Outcome Feedback

Cybernetics approaches complex systems through relational concepts such as order, complexity, hierarchy, structure, information, and control. These concepts allow systems of very different kinds—biological, social, and technological—to be analyzed within a unified framework.

Central to cybernetic analysis is the idea that systems are not defined by their physical composition but by distinctions—differences that allow the system to distinguish one state from another. Gregory Bateson famously summarized this principle (defining information) by defining information as “a difference that makes a difference.”

In cybernetic modelling, the state of a system is defined by the conjunction of the values of its relevant properties. The set of all possible configurations of those properties defines the system’s state space.

Let the state space of a system be

The variety of the system, representing the number of distinguishable states it may occupy, is defined as

Where denotes the cardinality of the state space?

Variety therefore measures the degree of uncertainty about the system’s state, or equivalently the amount of information required to determine it.

This concept of variety is foundational to cybernetic control. Effective control requires that the controlling system possess sufficient internal variety to respond to disturbances generated by the environment.

Cybernetic Perception and Constructed Models

Cybernetics also recognizes that a control system never perceives the environment directly. Instead, it perceives only signals relevant to maintaining its goals.

A cybernetic system, therefore, observes the environment through a limited perceptual interface. It does not have access to the environment “as it is,” but only to signals indicating whether its predictions or control actions succeed or fail.

In this sense, cybernetic epistemology is fundamentally constructivist. Knowledge is not passively received from the environment but constructed internally by the system itself. The environment does not instruct the system how to build its models; it merely eliminates those models that fail to maintain viable control.

Model construction therefore occurs through processes of variation and selection. The system generates candidate control architectures and retains those that maintain stable feedback with the environment.

To illustrate this principle, consider a simplified organism that must maintain its temperature within a viable range. Suppose its perceptual system distinguishes three possible environmental states:

Its available actions are

The organism’s control architecture is therefore a mapping

Because there are three perceptual states and three possible actions, the number of possible control architectures is

However, only a small subset of these architectures produces stable negative feedback capable of maintaining the organism within the viable temperature range. All other mappings generate runaway behavior that ultimately leads to failure of the system.

This simple example illustrates a fundamental limitation: even when the number of possible models is small, environmental feedback provides only weak information about which internal model is correct.

Environmental Feedback as Weak Information

As systems grow more complex, the number of possible internal models increases dramatically. For systems with many perceptual dimensions and action possibilities, the space of possible control architectures becomes astronomically large.

Environmental feedback—success or failure of outcomes—provides only limited information about which model among this vast space is correct.

In other words, the signal provided by outcomes is extremely sparse relative to the complexity of the model space.

Formally, if the internal model space is

and the observable outcomes are

Then multiple internal models may produce indistinguishable outcomes:

Over the observational regime available to the system.

Outcome signals alone cannot therefore uniquely determine the internal model.

Consequences for Mediated Intelligence Systems

This limitation becomes significantly more severe in mediated intelligence systems, where perception itself is generated through complex mediation processes.

Instead of directly observing environmental states, such systems operate on mediated representations:

where

represents latent environmental states,
represents mediated representations, and
Represents observable outcomes.

The outcome feedback used for learning therefore reflects the mediated representation rather than the underlying environment.

Information theory makes the limitation explicit. For the sequence above, the data processing inequality implies.

Outcome observations cannot contain more information about the underlying environment than the mediated representation from which they were derived.

Consequently, reflection mechanisms based solely on outcomes cannot in general recover constraints lost during mediation.

The system may therefore stabilize around representations that are internally coherent and operationally successful while remaining misaligned with the underlying environment.

Toward Second-Order Reflection

The cybernetic tradition recognized that control systems must operate through internal models and limited perceptual signals. Second-order cybernetics extended this insight by emphasizing the role of the observer in constructing these models.

However, mediated intelligence systems introduce a further structural complication: the perceptual interface itself may be dynamically reshaped by the system’s own operations.

Under these conditions, reflection confined to outcomes is insufficient to guarantee representational stability.

This observation motivates the Second Law of Mediated Intelligence Systems.

This limitation can be expressed formally as the possibility that optimization over outcomes alone yields

While mediation distortion remains non-zero:

where mediation distortion may be expressed as

Representing the constraint loss introduced during mediation.

Failure of Outcome-Only Control in Mediated Systems

As discussed, classical cybernetic control assumes that feedback derived from outcomes contains sufficient information to correct the internal behavior of the system. In physical control systems this assumption is often justified, because the measured output directly reflects the state of the controlled variable.

However, mediated intelligence systems introduce an intermediate representational layer between the environment and observable outcomes. Instead of a direct mapping between state and output, the system operates through the sequence.

where

denotes latent environmental states, denotes latent environmental states,
denotes mediated representations, denotes mediated representations,
denotes observable outcomes. denotes observable outcomes.

Operational decisions are then produced by applying an internal policy to the representation:

and outcomes are generated through interaction between the action and the environment:

Substituting the policy definition yields

Because the representation itself is derived from the environment through mediation, itself is derived from the environment through mediation,

the complete mediated system can be written as

Outcome-Based Reflection

In outcome-only control architectures, system adaptation occurs through optimization of a loss function defined over observable outcomes.

Let the outcome loss be

Where represents the desired or target outcome. represents the desired or target outcome.

Reflection updates the parameters of the system to minimize this loss:

When the optimization converges, the system reaches a stationary point satisfying

In classical cybernetic systems, this condition corresponds to successful control.

However, mediated intelligence systems introduce an additional source of error: distortion in the mediation process itself.

Mediation Distortion

Let mediation distortion be defined as the loss of environmental constraint induced by the representation process.

One convenient measure of this loss is conditional entropy:

This quantity measures how much uncertainty about the underlying environment remains after the system observes the mediated representation.

If mediation preserves all relevant constraints, then

However, when mediation removes causal, temporal, or contextual information, the conditional entropy increases.

The critical observation is that optimization of outcome performance alone does not necessarily reduce mediation distortion.

Formally, the following condition may hold:

while

Thus, operational optimization may converge to a configuration that performs well according to observed outcomes while still operating on distorted representations of the environment.

Information Limits of Outcome Feedback

The fundamental limitation arises from the informational structure of mediated systems.

If environmental states, representations, and outcomes form the sequence

Then the data processing inequality implies

This inequality states that the outcome variable cannot contain more information about the environment than the representation from which it was derived.

Therefore, feedback based solely on outcomes cannot reconstruct information that was discarded during mediation.

Multiple distinct mediation functions may therefore produce identical observable outcomes:

Over the observational regime available to the system.

When this occurs, outcome feedback cannot distinguish between accurate and distorted representations.

The system may therefore converge to internally stable control architectures that remain misaligned with the underlying environment.

Compute-Reason-Reflect as a Stabilizing Architecture

The failure of outcome-only reflection motivates the introduction of a broader stabilization architecture.

In mediated intelligence systems, stability requires reflection not only on outcomes but also on the processes that generate representations.

A minimal architecture capable of supporting this requirement consists of three interacting functions:

Compute
Reason
Reflect

This structure can be expressed as the sequence

where

The three functional components operate as follows.

Compute

The compute function performs operations on the available representation.

Where represents computed features, predictions, or intermediate states. represents computed features, predictions, or intermediate states.

Compute increases local processing capacity but does not alter the representational constraints of the system.

Reason

Reasoning imposes structure and inference across the computed representations.

Where represents inference rules, policies, or decision structures. represents inference rules, policies, or decision structures.

Reasoning can increase internal coherence but remains bounded by the information available in the representation.

Reflect

Reflection introduces a corrective process that compares outcomes with expectations and updates both operational and representational components.

The reflection operator can be expressed as

where

parameterises the mediation process, parameterises the mediation process,
parameterises the operational policy. Parameterises the operational policy.

Reflection therefore adjusts both

and

Rather than updating operational behavior alone.

Dual Reflection Requirement

The stabilizing requirement can therefore be expressed as a dual optimization condition.

Outcome performance must be constrained:

While mediation distortion must also be constrained:

Stability, therefore, requires minimization across both dimensions:

Systems that optimize only outcome performance risk converging to configurations that maintain operational success while preserving representational distortion.

Implication for Mediated Intelligence Systems

These results demonstrate that classical cybernetic control—based solely on outcome feedback—is insufficient for mediated intelligence systems.

When mediation layers reshape perception before reasoning, stability requires reflection across both the operational and perceptual layers of the system.

The Compute-Reason-Reflect architecture provides the minimal functional structure capable of supporting this requirement.

Together with the first law of mediated intelligence systems, this analysis establishes the structural conditions under which mediated intelligence systems can maintain coherence with the environments they represent.

Entropy Dynamics in Compute–Reason–Reflect Systems

The previous sections established that outcome-only reflection cannot guarantee representational stability in mediated intelligence systems. The question that follows is how such systems behave dynamically when mediation, computation, and reflection operate at different rates.

To analyze this behavior, we extend the concept of representational entropy introduced in the previous paper. Entropy is treated as a measure of dispersion over the set of admissible mediated representations.

Let the latent environmental state be

and the mediated representation produced by the system be

The uncertainty remaining about the environment after mediation is given by the conditional entropy

This quantity measures the constraint loss introduced by mediation.

Define mediation distortion as

Higher values of indicate that the representation retains fewer structural constraints linking it to the underlying environment.

Entropy Growth Through Extract-Based Processing

In mediated intelligence systems, operations are rarely performed directly on environmental states. Instead, successive computational processes operate on derived representations.

Consider a sequence of transformations:

Each transformation represents an extract-based process such as filtering, summarization, feature construction, or predictive modelling.

Because each stage discards some information from the previous representation, the conditional entropy generally increases along the chain:

This relationship expresses the entropy gradient of extract-based reasoning.

Local coherence may increase at each step because the system imposes structure on the representation. However, the representation becomes progressively less constrained by the underlying environment.

This dynamic corresponds to the entropy amplification described in the first paper: optimization on layered extracts increases local order while increasing global entropy.

Compute–Reason as Entropy-Amplifying Processes

Within the Compute–Reason–Reflect architecture, the first two functions operate on mediated representations without directly re-anchoring them to environmental states.

The compute function performs transformations

Where denotes computed features, embeddings, or intermediate states.

Reasoning then imposes inferential structure:

Because both functions operate on representations derived from previous representations, the combined process forms an extract chain:

The conditional entropy therefore satisfies

Compute and Reason may increase internal consistency or predictive performance, but they cannot restore information lost during earlier mediation steps.

As a result, systems that rely solely on Compute and Reason tend to increase representational dispersion over time.

Reflection as Entropy Dampening

The Reflect function introduces a corrective mechanism that compares system outputs with observed outcomes and adjusts the internal processes generating representations.

Let the system parameters governing mediation and reasoning be

and

Reflection updates these parameters using outcome observations:

Where represents the observed outcome at time .

The critical feature of reflection is that it modifies both

and

Rather than adjusting operational behavior alone.

Through this process, reflection attempts to reduce mediation distortion:

A successful reflection step therefore satisfies

In contrast to Compute and Reason, which tend to increase representational dispersion, Reflect introduces negative entropy into the system by restoring constraints linking representations to outcomes.

Entropy Balance in Mediated Intelligence Systems

The dynamic stability of mediated intelligence systems therefore depends on the balance between entropy amplification and entropy correction.

Let

denote the rate of entropy generation from mediation, computation, and reasoning processes,
denote the rate of entropy reduction introduced by reflection.

The change in representational entropy over time can be expressed qualitatively as

Three regimes follow from this relationship.

Representational entropy increases, and system representations diverge from the underlying environment.

The system reaches a metastable equilibrium in which local coherence persists while representational drift continues slowly.

Reflection successfully stabilizes the system by maintaining sufficient constraint between representation and environment.

Relationship to the First and Second Laws

This entropy balance clarifies the relationship between the two laws of mediated intelligence systems.

The First Law describes the divergence condition:

Intelligence systems that scale perception faster than reflection will diverge.

Formally, divergence occurs when

The Second Law describes the structural requirement for stabilization:

Stability requires reflection across both operational outcomes and the mediation processes that determine perceptual inputs.

This requirement ensures that reflection acts directly on the sources of mediation distortion rather than only on downstream outcomes.

Together, these principles define the entropy dynamics governing mediated intelligence systems.

Implication for Cognitive Systems Architecture

The Compute–Reason–Reflect loop therefore functions as an entropy-regulation mechanism.

Compute and Reason increase internal structure while operating on partial representations. Reflect re-anchors the system by restoring constraints between representations and outcomes.

Without reflection, computational scale accelerates entropy accumulation. With reflection operating across both operational and mediation layers, mediated intelligence systems can maintain representational stability despite operating on partial and evolving representations.

The Third Law of Mediated Intelligence Systems

The previous sections established two structural properties of mediated intelligence systems. The First Law describes the conditions under which representational divergence emerges when mediation scales faster than reflection. The Second Law establishes that stability requires reflection across both operational outcomes and the mediation processes that shape perceptual inputs.

A further consequence follows from the optimization dynamics of mediated systems.

When intelligence systems optimize performance metrics derived from mediated representations, the system does not directly optimize its alignment with the underlying environment. Instead, it optimizes performance relative to the representation on which it operates.

This leads to a third structural principle governing mediated intelligence systems.

Third Law of Mediated Intelligence Systems

Systems that optimize performance only over mediated representations will converge toward configurations that optimize the representation itself rather than the underlying environment.

Optimization Over Representations

Let the latent environmental state be

and the mediated representation be

Operational decisions are derived from the representation through a policy

and outcomes arise through environmental interaction

In practice, most mediated intelligence systems optimize a performance objective defined over observable outcomes or representations:

Optimization adjusts system parameters to minimize this objective:

However, because the system operates on the mediated representation rather than directly on environmental states, the optimization process effectively targets the representation itself.

Representation–Environment Misalignment

Let environmental alignment be defined as the degree to which the representation preserves the structural constraints of the environment.

One measure of representational misalignment is the mediation distortion

Minimizing the operational objective

does not necessarily minimize

Thus it is possible for the system to satisfy

while

Under these conditions, the system has reached an operational optimum that is stable regarding its internal objective but remains misaligned with the underlying environment.

Representation Self-Optimization

This dynamic leads to a structural shift in the target of optimization.

Initially, system performance reflects a relationship between environmental states and mediated representations:

As optimization proceeds, however, the system increasingly operates on internal representations rather than environmental states.

The effective optimization loop becomes

In this regime, system behavior stabilizes around internal representational structures rather than external environmental constraints.

The system is therefore optimizing its own mediated reality.

Stable Representation Regimes

The equilibrium condition for mediated optimization can therefore be expressed as

subject to

However, because mediation removes environmental constraints, multiple distinct environments may produce equivalent mediated representations:

If this condition holds, then the system cannot distinguish between these environments through its representation.

Optimization therefore operates over equivalence classes of environments defined by the mediation function.

Formally, define the equivalence relation

The system’s effective state space is therefore not the environment but the set of equivalence classes

Optimization therefore occurs over the quotient space

Rather than over the environment itself.

Consequences for Mediated Intelligence

This structural property has several consequences.

First, system stability does not imply environmental accuracy. A system may converge to a highly stable operational regime that remains structurally misaligned with the environment.

Second, improvements in computational capacity accelerate this effect. Greater computational power enables more efficient optimization over the mediated representation without restoring lost environmental constraints.

Third, reflection mechanisms that operate only on operational performance metrics will reinforce representational stability rather than environmental alignment.

In this sense, mediated intelligence systems naturally tend toward representational closure: a state in which the system stabilizes around internally coherent representations that need not correspond to the underlying environment.

Relationship to the Previous Laws

The three laws of mediated intelligence systems therefore describe complementary aspects of system dynamics.

The First Law describes the divergence condition:

Intelligence systems that scale perception faster than reflection will diverge.

The Second Law describes the structural stabilization requirement:

Stability requires reflection across both operational outcomes and the mediation processes that determine perceptual inputs.

The Third Law describes the equilibrium behavior of optimized mediated systems:

Systems that optimize performance only over mediated representations will converge toward configurations that optimize the representation itself rather than the underlying environment.

Together these laws define the structural dynamics governing mediated intelligence systems.

Toward a Thermodynamics of Mediated Intelligence

Taken together, the three laws reveal a deeper analogy between mediated intelligence systems and thermodynamic systems.

Compute and Reason processes generate representational entropy by operating on successive extracts of the environment. Reflect functions introduce corrective constraints that dampen this entropy and restore alignment with observed outcomes.

However, when optimization occurs primarily over mediated representations, the system’s equilibrium state may correspond not to environmental truth but to internally stabilized representations.

In this sense, mediated intelligence systems exhibit a form of representational thermodynamics in which divergence, stabilization, and equilibrium are governed by the interaction between mediation, computation, and reflection.

Unified Formulation of the Three Laws

The preceding sections introduced three structural laws governing mediated intelligence systems. The First Law describes the divergence condition that arises when mediation scales faster than reflection. The Second Law establishes the structural requirement that reflection must operate across both operational outcomes and the mediation processes shaping perceptual inputs. The Third Law describes the equilibrium behavior of mediated systems that optimize performance over internal representations.

Although presented separately, these laws arise from a common underlying dynamic: the evolution of representational entropy within mediated cognitive systems.

To formalize this relationship, consider the mediated representation

Where denotes latent environmental states and represents the mediation process.

The uncertainty remaining about the environment after mediation is given by

Which measures the constraint loss introduced by mediation. As introduced earlier, define mediation distortion as

This quantity represents the structural entropy associated with the system’s representation of the environment.

Sources of Representational Entropy

Representational entropy evolves as the system processes information through successive mediation and computation steps.

Three processes influence this evolution.

The first is entropy generation through mediation and extract-based processing. Each transformation that compresses, filters, or abstracts information removes constraints linking representations to environmental states.

Let the rate of entropy generation from mediation and extract processing be denoted

The second process is entropy reduction through reflection. Reflection introduces corrective feedback by comparing system predictions with observed outcomes and adjusting both operational behavior and mediation processes.

Let the rate of entropy reduction introduced by reflection be

The third process is representation self-optimization, in which system optimization increasingly targets the mediated representation rather than the underlying environment.

Let the rate at which optimization reinforces representational closure be

Entropy Dynamics of Mediated Intelligence

The evolution of representational entropy can therefore be expressed qualitatively as

where

represents entropy introduced by mediation and extract-based computation,
represents entropy reduction through reflection,
represents entropy reinforcement arising from representation self-optimization.

This equation captures the fundamental dynamic governing mediated intelligence systems.

Law I: Divergence

The First Law emerges when mediation and computation scale faster than reflection.

Then representational entropy increases over time:

Under these conditions, mediated representations progressively diverge from the underlying environment even as local computational efficiency and internal coherence improve.

This condition corresponds to the First Law of mediated intelligence systems:

Intelligence systems that scale perception faster than reflection will diverge.

Law II: Stabilization Requirement

The Second Law specifies the structural requirement necessary to prevent divergence.

Reflection must operate not only on operational behavior but also on the mediation processes that generate perceptual inputs.

Formally, this requirement ensures that reflection acts directly on the mediation parameters:

so that reflection updates

Rather than optimizing operational behavior alone.

When reflection acts across both operational and mediation layers, the entropy balance becomes

Under this condition, representational entropy remains bounded, and the system maintains coherence with the environment.

This corresponds to the Second Law:

Stability requires reflection across both operational outcomes and the mediation processes that determine perceptual inputs.

Law III: Representational Equilibrium

Even when reflection is present, mediated intelligence systems may converge toward equilibrium states defined primarily by their internal representations.

If operational optimization dominates environmental constraint recovery, the system may satisfy

while

Under these conditions the system stabilizes around internally coherent representations that remain partially decoupled from the environment.

This corresponds to the Third Law:

Systems that optimize performance only over mediated representations will converge toward configurations that optimize the representation itself rather than the underlying environment.

Structural Interpretation

Taken together, the three laws describe the entropy dynamics governing mediated intelligence systems.

The First Law describes the condition under which representational entropy grows.
The Second Law specifies the reflection structure required to regulate entropy.
The Third Law describes the equilibrium behavior that arises when optimization occurs primarily within mediated representations.

These dynamics arise not from intent or design failure but from the structural properties of systems that reason over mediated representations rather than directly over environmental states.

Toward a General Theory of Mediated Intelligence

The unified formulation suggests that mediated intelligence systems obey a set of structural constraints analogous to thermodynamic systems.

Mediation introduces entropy through abstraction and compression. Computation organizes representations locally but does not restore lost constraints. Reflection provides the only mechanism capable of reintroducing environmental constraints into the system.

The stability of mediated intelligence systems therefore depends on maintaining a balance between entropy generation and entropy correction across the mediation, computation, and reflection layers.

When this balance is maintained, mediated intelligence systems can remain coherent with the environments they represent. When it is not, the systems will tend toward divergence or representational closure despite appearing internally consistent and operationally efficient.

The Observer Problem in Mediated Intelligence Systems

Classical cybernetics examined systems through the interaction between a system and its environment. Control was achieved through feedback loops that allowed the system to adjust its behavior in response to observed outcomes. In this framework, the observer was typically treated as external to the system being studied.

Second-order cybernetics introduced a critical refinement: the observer cannot be separated from the system being observed. Observations are themselves generated by the observer’s internal structures and constraints. Consequently, the behavior of the system cannot be understood independently of the observer who constructs its representation.

This insight becomes particularly important in mediated intelligence systems, where perception itself is produced through computational mediation.

Classical Cybernetic Structure

In first-order cybernetics, system dynamics can be described through a feedback loop between environmental states and system actions.

Let

represent environmental states and

Represent system actions.

System dynamics may be expressed as

The system observes outcomes

and adjusts its control policy

In this formulation, the observer simply measures system outputs and evaluates performance relative to desired outcomes.

The Second-Order Cybernetic Insight

Second-order cybernetics recognizes that the observer does not have direct access to environmental states. Instead, the observer constructs internal representations of the system based on available signals.

Let the observer’s representation be

where

Represents the observer’s mediation process.

The observer’s understanding of the system is therefore constructed through the representation

Rather than through direct access to the environmental state .

In this sense, the observer is not external to the system but part of a larger cognitive system that includes both the observed process and the observing agent.

Mediated Intelligence Systems

Mediated intelligence systems introduce an additional structural layer between environmental states and observer representations.

Instead of the observer directly constructing a representation from environmental signals, the system now produces its own mediated representation:

The observer then perceives the system through a second mediation layer:

The complete perceptual chain therefore becomes

where

represents the AI system’s internal representation of the environment,
represents the observer’s representation of the system.

Because each stage may discard information, the conditional entropy increases along the chain:

The observer therefore operates with even less information about the environment than the system itself.

The Operator Layer

In practical mediated intelligence systems, a third role emerges: the operator.

The operator is responsible for governing the behavior of the AI system. This governance includes defining operational constraints, policies, and performance objectives.

Let the operator’s governance function be

which acts on the AI system by adjusting operational parameters

The operator therefore modifies system behavior through

where

is the observer’s representation of the system,
represents observed outcomes.

The operator therefore governs the system using representations that are themselves mediated.

Three Interacting Behaviors

In mediated intelligence systems, system behavior emerges from the interaction of three distinct behavioral layers:

System Behavior The operational dynamics of the AI system.
Observer Behavior The interpretation and representation of system behavior.
Operator Behavior The governance decisions applied to the system.

Formally, these behaviors can be expressed as

System behavior:

Observer representation:

Operator governance:

Each layer operates on representations produced by the previous layer.

Governance vs. Outcome

This structure reveals an important distinction between two forms of governance.

System governance focuses on controlling the behavior of the AI system itself:

This corresponds to governing the AI system.

Outcome governance, in contrast, evaluates the effects produced by the system:

This corresponds to governing the outcomes produced by AI.

In mediated intelligence systems these two objectives do not necessarily coincide.

A system may satisfy governance rules governing its internal behavior while still producing outcomes that diverge from environmental reality.

The Missing Element

Second-order cybernetics correctly recognized that observers construct representations of systems. However, mediated intelligence systems introduce an additional structural feature that second-order cybernetics did not fully anticipate: large-scale computational mediation.

In these systems, representations are not produced solely by human observers but by layered computational processes that filter, rank, compress, and synthesize information.

As a result, both the system and the observer operate on representations that have already been transformed by mediation.

The missing element in classical second-order cybernetics is therefore explicit governance of mediation itself.

Without reflection mechanisms capable of examining and correcting the mediation process, governance decisions operate on representations that may already contain structural distortions.

Reflection Across the Observer Boundary

This observation connects directly to the Compute–Reason–Reflect architecture introduced earlier.

Compute and Reason operate within the system’s mediated representation:

Reflection must therefore operate across the entire observational chain:

Reflection must examine not only operational outcomes but also the mediation processes that generate both system and observer representations.

Only under these conditions can the system maintain alignment between environmental states, mediated representations, and governance decisions.

Implications for AI Governance

The observer problem reveals a structural limitation of many contemporary AI governance frameworks.

Most governance approaches focus on either:

Regulating system behavior, or
Evaluating system outcomes.

However, mediated intelligence systems require governance that also addresses the processes that shape perception itself.

Without such mechanisms, governance operates on representations that may already have diverged from the underlying environment.

The result is a system that appears compliant, transparent, and accountable while remaining structurally misaligned with the reality it is intended to represent.

Toward Observer-Aware Mediated Systems

Recognizing the observer concern suggests a broader architectural requirement for mediated intelligence systems.

Stability requires feedback loops that connect:

environmental states,
mediated representations,
observer interpretations,
operator governance decisions.

This requirement extends the Compute–Reason–Reflect architecture beyond the operational layer to include observer and governance processes as integral components of the system.

In this sense, mediated intelligence systems must treat the observer not as an external evaluator but as a participant within the system’s entropy regulation process.

Multi-System Mediation and Environmental Entropy

The paper discussed and analyzed mediated intelligence systems primarily as individual feedback structures consisting of system behavior, observer interpretation, and operator governance, however in practice modern environments rarely contain a single mediated intelligence system. Instead, many such systems coexist and interact within the same environment.

Each deployed AI system forms its own mediated intelligence loop consisting of three behavioral layers:

System behavior — the operational dynamics of the AI system
Observer behavior — the interpretation and representation of system behavior
Operator behavior — the governance decisions applied to the system

For system (i), these components can be expressed as

System representation

Observer representation

Operational behavior

Governance update

Although these mediated loops are internally coupled, they all interact with the same environmental state

Divergent Representations of Shared Environments

Because each system possesses its mediation functions

and

The representations constructed by different systems will generally differ:

even though both are derived from the same environment:

Consequently, the observer representations will also differ:

These differences arise even when the underlying environment remains unchanged.

Thus, multiple mediated systems operating in the same environment produce parallel but non-identical representations of reality.

Inter-System Entropy

The divergence between system representations can be measured as the entropy across the ensemble of mediated representations.

Let the set of system representations be

Define inter-system representational entropy as

This quantity measures the dispersion of representations across systems that observe the same environment.

As the number of mediated systems increases, the probability that their representations diverge also increases.

Governance Interaction

Governance decisions applied to one system may also affect other systems operating in the same environment.

For example, an operator adjusting system (i) modifies the policy

which alters the actions

These actions influence the shared environment:

Thus, changes introduced by one system propagate through the environment and influence the observations available to other systems.

This creates a coupled system of mediated loops.

Entropy Amplification Through Interaction

Because each system interprets environmental signals through its mediation functions, environmental changes caused by other systems may be interpreted differently by each observer.

Thus, system interaction introduces additional entropy beyond that generated within individual mediation loops.

The total entropy of the mediated environment can therefore be approximated as

where

represents entropy generated within individual mediated loops
represents entropy arising from divergence across systems

As the number of mediated systems increases, the inter-system entropy term becomes increasingly significant.

Multi-Actor Cybernetic Complexity

This dynamic introduces a structural complexity that classical cybernetics did not fully address. Traditional cybernetic models typically analyze a single feedback loop between system and environment.

In mediated intelligence environments, however, many such loops operate simultaneously and influence each other through the shared environment.

Each loop contains its system, observer, and operator behaviors. These behaviors interact not only with the environment but also indirectly with other mediated systems.

The resulting structure is therefore not a single feedback loop but a network of interacting mediated feedback loops.

Entropy Growth in Multi-System Environments

As the number of mediated systems increases, the number of interactions between systems grows approximately as

Where (n) is the number of systems.

Each interaction introduces additional opportunities for representational divergence.

Consequently, the entropy generated by multi-system environments may grow faster than the entropy generated within any single mediated system.

Implications

This analysis implies that entropy in mediated intelligence environments arises from two distinct sources:

Internal mediation entropy, generated within individual AI systems.
Interaction entropy, generated by divergence among multiple mediated systems operating in the same environment.

Even if each individual system maintains internal stability through reflection, the interaction of many such systems may still produce global representational divergence.

This phenomenon suggests that stability in mediated intelligence environments cannot be achieved solely by stabilizing individual systems. It also requires mechanisms capable of regulating the interactions between multiple mediated systems operating within a shared environment.

Optimization and Synthetic Realities

The emergence of synthetic realities in mediated intelligence ecosystems can be understood through the optimization dynamics of modern AI systems. Recent work has proposed that large language models and related systems may be interpreted as optimizers operating over internal representations rather than direct predictors of environmental states.

In such systems, the optimization process operates primarily on mediated representations produced during training rather than on direct observations of the environment.

Let the mediated representation be

where

denotes latent environmental states,
denotes the mediation process through which representations are constructed.

The operational behavior of the system is then generated through

Where represents the decision or generation function learned by the system?

Because optimization operates over the mediated representation rather than the underlying environment, system performance is evaluated through a loss function defined over representations:

Rather than directly over environmental states.

Optimization therefore seeks to minimize

This process can produce highly coherent internal representations even when those representations diverge from the environment that generated the original data.

Representation Optimization

When multiple mediated intelligence systems operate within the same environment, each system optimizes its own mediated representation.

For system (i),

and optimization proceeds through

Because the mediation functions (m_i) differ across systems, the resulting optimized representations will generally diverge:

Even when derived from the same underlying environment.

Each system therefore converges toward a locally optimized representation of reality.

Interaction of Optimized Systems

When multiple optimized systems interact, their actions collectively modify the environment:

where

Each system therefore observes an environment partially shaped by the optimized actions of other systems.

This interaction progressively transforms the environment into a product of mediated optimization rather than a direct reflection of environmental dynamics.

Emergence of Synthetic Reality

The observed environment can therefore be decomposed into two components:

where

represents the underlying environmental state,
represents system-generated modifications.

As the number of interacting optimized systems increases,

and the environment becomes increasingly dominated by system-generated effects.

At this stage, mediated systems are effectively optimizing their behavior relative to a reality that is itself produced by mediated optimization.

Irrecoverability of the Original Environment

Because mediation processes compress environmental information, multiple environmental states may map to identical mediated representations:

Once system interactions modify the environment, it becomes impossible to uniquely recover the original environmental state from mediated observations.

Formally,

This implies that the original environment cannot be reconstructed from the synthetic environment produced by interacting mediated systems.

Consequence for Mediated Intelligence

This analysis reveals a structural transition in mediated intelligence ecosystems. At a small scale, AI systems operate as observers and decision-makers within an environment. At a large scale, interacting AI systems collectively reshape the environment they observe.

The system therefore transitions from modeling reality to constructing reality.

Under these conditions, the environment itself becomes an emergent property of interacting mediated intelligence systems.

Absence of Stabilization and the Loss of Collective Trajectory

The preceding analysis established that mediated intelligence systems generate representational divergence when mediation scales faster than reflection. Without stabilizing reflection mechanisms, mediated systems increasingly operate on representations that progressively diverge from the environmental states they intend to model.

At the level of individual systems, this divergence manifests as operational misalignment between representation and environment. However, the consequences become more significant when mediated systems operate at a societal scale.

Modern societies increasingly rely on mediated intelligence systems to filter information, construct narratives, support decision-making, and govern complex infrastructures. As these systems multiply and interact, the absence of stabilizing reflection mechanisms propagates beyond individual systems and begins to affect collective decision structures.

Under these conditions, the absence of stabilization becomes not merely a property of individual systems but a structural property of the broader socio-technical environment.

From Event Instability to Reality Instability

Instability within mediated systems initially manifests as instability in events.

An event generated or interpreted through mediated systems may be amplified, suppressed, or reinterpreted depending on the mediation processes involved. As described earlier, mediation determines which signals become visible and which remain absent.

When such mediation occurs at large scale, instability extends beyond isolated events.

The mediated environment itself becomes unstable, as different actors operate on divergent representations of reality. This produces a condition in which disagreement no longer arises from differing interpretations of shared observations but from fundamentally different perceived realities.

The environment therefore transitions from event instability to reality instability.

Societal Consequences

Societies function through coordination across shared expectations about the future. These expectations guide economic decisions, political processes, institutional planning, and technological development.

Such coordination implicitly assumes that actors operate within sufficiently overlapping representations of reality.

When mediated systems introduce persistent divergence across representations, this assumption weakens. Different actors increasingly operate within different informational environments, leading to divergent interpretations of both present conditions and future possibilities.

Let the collective societal trajectory be represented as

where

represents the environmental state
Represents the actions of actors operating on mediated representations.

When representations diverge across actors, the actions taken by those actors diverge as well.

The resulting trajectory therefore becomes unstable.

The Absence of Destiny

Historically, societies maintained a degree of shared trajectory because actors operated within overlapping event streams and shared informational constraints.

In mediated environments characterized by high representational divergence, this shared trajectory becomes increasingly difficult to sustain.

The system no longer converges toward a coherent long-term path. Instead, it produces competing trajectories driven by locally optimized representations.

This condition can be described as the absence of collective destiny.

In this context, destiny does not imply inevitability or determinism. Rather, it refers to the existence of sufficiently stable shared expectations that allow societies to coordinate actions toward long-term outcomes.

When representational entropy becomes sufficiently large, such coordination becomes structurally difficult.

Structural Interpretation

The absence of stabilization therefore operates at multiple levels:

Event level — individual signals become unstable through mediation.
Reality level — actors operate on divergent representations of the environment.
Trajectory level — collective action fails to converge toward a shared future.

This progression reflects the amplification of representational entropy across scales.

Without stabilizing reflection mechanisms capable of reconciling representations with outcomes, mediated intelligence systems introduce structural divergence that propagates through the environments and societies that depend on them.

Implication

The challenge introduced by mediated intelligence systems is therefore not limited to system accuracy or algorithmic bias.

The deeper challenge concerns the preservation of coherence across representations at a societal scale.

When stabilization mechanisms fail, mediated intelligence systems may produce environments in which coordination around shared trajectories becomes increasingly difficult.

Preventing this outcome requires designing intelligence architectures in which reflection functions as a stabilizing force capable of reconnecting mediated representations with observed outcomes across time.

The Entropy of Destiny

The progression from mediated perception to synthetic environments introduces a final structural consideration: the stability of long-term collective trajectories. The analysis presented in this paper does not imply inevitable societal decline or systemic failure. Rather, it highlights a structural dynamic that may emerge when mediated intelligence systems scale without corresponding stabilization mechanisms.

In this context, the term entropy of destiny refers to the progressive loss of shared trajectory in environments characterized by high representational divergence.

Historically, societies have coordinated large-scale actions through sufficiently overlapping representations of the present and shared expectations about the future. Economic planning, institutional governance, technological development, and international cooperation all rely on the existence of a reasonably stable collective understanding of environmental conditions and probable future states.

These shared expectations implicitly constrain the space of possible societal trajectories.

Representational Entropy and Collective Trajectory

Let the societal trajectory be represented as

where

represents the environmental state
Represents actions taken by individual actors or systems.

Each actor bases decisions on mediated representations

When the mediation processes differ significantly across actors, the resulting representations diverge:

Decision processes therefore produce divergent actions

Which in turn influences the evolving trajectory of the system.

As representational divergence increases, the ability of actors to coordinate long-term decisions weakens.

Entropy of Future States

The uncertainty associated with collective trajectory can be expressed as the entropy of possible future states.

Let

Represent the probability of trajectory (T_k).

The entropy of societal trajectory can then be expressed as

When actors operate on overlapping representations of reality, the distribution of trajectories is constrained. Coordination reduces uncertainty about the future.

However, as mediated representations diverge across actors, the distribution of possible trajectories broadens. The system increasingly explores a wider set of potential future states.

This increase in trajectory entropy does not necessarily imply disorder or collapse. Rather, it reflects the reduced ability of actors to converge on a shared long-term path.

Destiny as Coordinated Trajectory

In this paper, the term destiny is not used in a metaphysical or deterministic sense. Instead, it refers to the capacity of a society to maintain a sufficiently coherent collective trajectory over time.

Destiny in this sense emerges when actors operating within a shared environment coordinate their actions around common expectations about future conditions.

When representational entropy grows, this coordination becomes progressively more difficult.

Different actors increasingly pursue locally coherent strategies based on representations that may diverge significantly from one another.

The result is not necessarily conflict or collapse but trajectory fragmentation.

A Cautionary Narrative

The entropy of destiny should therefore be understood not as a prediction but as a cautionary narrative. It highlights the potential consequences of scaling mediated intelligence systems without designing corresponding mechanisms capable of stabilizing shared representations.

The emergence of synthetic realities and interacting mediated systems introduces structural forces that increase representational entropy across societies.

Without reflection mechanisms capable of reconnecting mediated representations with environmental outcomes, these forces may gradually erode the informational foundations upon which coordinated long-term planning depends.

The Role of Reflection

The analysis presented throughout this paper suggests that maintaining coherent trajectories in mediated environments requires more than improved prediction or optimization.

It requires architectures capable of stabilizing representations through reflection.

Reflection introduces the possibility of reconciling mediated representations with observed outcomes across time. When embedded within system design, reflection can act as a structural counterbalance to the entropy introduced by mediation and interaction among systems.

In this sense, the challenge posed by mediated intelligence systems is not whether synthetic environments will emerge, but whether societies can design reflexive architectures capable of maintaining coherence within them.

Toward Stabilization in Synthetic Environments

The analysis presented throughout this paper has explored the structural consequences of mediated intelligence systems operating at scale. When perception is increasingly mediated by artificial systems and when those systems interact within shared environments, a new class of informational dynamics emerges.

These dynamics are characterized by several structural features:

mediation determines what is perceived
representations diverge across systems
optimization occurs over partial representations
system interactions transform the environment itself
collective trajectories become increasingly difficult to stabilize

These conditions do not arise from failure or malicious design. They emerge naturally from the scaling of mediated intelligence within complex environments.

As artificial systems increasingly participate in perception, reasoning, and decision-making, the environments in which they operate become progressively shaped by their own interactions. The result is the emergence of synthetic realities in which the distinction between environmental signals and system-generated signals becomes increasingly difficult to maintain.

Under these conditions, stability cannot be achieved solely through improvements in prediction accuracy, optimization techniques, or policy enforcement.

Stability becomes an architectural question.

Limits of Current Governance Approaches

Most current governance approaches focus on constraining the behavior of individual systems. These approaches assume that the environment against which system behavior is evaluated remains stable and external to the system itself.

However, the analysis presented in this paper suggests that this assumption becomes increasingly fragile as mediated systems multiply and interact.

Governance mechanisms designed to regulate individual systems cannot fully address the entropy introduced by interactions among many systems operating on divergent representations of shared environments.

Similarly, policy frameworks that evaluate outcomes alone may overlook the mediation processes that shape perception before reasoning occurs.

Stability in mediated environments therefore requires more than regulation of individual systems. It requires mechanisms capable of maintaining coherence across representations, systems, and environments simultaneously.

The Need for a Stabilizing Substrate

The structural dynamics described in this paper suggest the need for an underlying stabilizing layer capable of reconnecting mediated representations with environmental outcomes.

Such a layer would not function as a supervisory authority imposed on individual systems. Instead, it would provide a shared informational structure through which representations, actions, and outcomes can remain mutually interpretable across systems.

In this sense, stabilization requires something deeper than governance mechanisms applied after decisions are made.

It requires a substrate capable of supporting coherent mediation, reasoning, and reflection across the entire mediated ecosystem.

A Glimpse of the Next Step

This paper has focused primarily on identifying the structural conditions under which entropy emerges in mediated intelligence systems. It has shown how the scaling of mediation, optimization over partial representations, and interaction among systems can produce environments increasingly shaped by the systems themselves.

The question that naturally follows is how such environments might be stabilized.

Addressing that question requires moving beyond the analysis of individual systems toward the design of architectures capable of maintaining coherence across mediated ecosystems.

The concept of a cognitive substrate provides one possible direction for such an architecture.

Rather than treating intelligence systems as isolated agents interacting with an external environment, a substrate-based approach considers the possibility of shared representational structures through which perception, reasoning, and reflection remain anchored to observable outcomes.

Exploring what such a substrate might look like, how it might function, and how it might support stabilization in synthetic environments will be the focus of the next stage of this work.

Closing Reflection

The emergence of mediated intelligence systems marks a transition in the relationship between cognition and environment.

Artificial systems no longer merely observe the world; they increasingly participate in shaping the informational environments within which perception and decision-making occur.

Recognizing this transition is the first step toward designing systems capable of maintaining coherence within these new environments.

The challenge ahead is not simply to build more capable intelligence systems, but to understand the structures that allow intelligence—human and artificial—to remain grounded within realities that are increasingly mediated by their own activity.

The emergence of mediated intelligence systems marks a transition in how reality is perceived, interpreted, and acted upon. As synthetic environments increasingly shape the signals upon which decisions depend, stability can no longer be assumed to arise naturally from computation or governance alone. Maintaining coherence in such environments requires structures capable of anchoring representation, action, and outcome within a shared informational foundation.

The next paper will discuss what such a stabilizing substrate might be.