Read the full article at: jameshorsley.me/articles/semiodynamics

---

TL;DR

Semiodynamics is a framework for modeling how cognitive systems exchange meaning through signals and perform cognitive work to achieve their goals.

Cognitive systems are networks of cognitive nodes that exchange signals and perform cognitive work. When a signal is sent, rather than emitted, how much of the sender's semantic intent is encoded into the signal is based on the sender's encoding efficiency. Signals have a receiver-relative semantic content which is narrowed into semantic availability when viewed through a specific interface. Receivers use their internal models and available cognitive networks to generate a semantic yield based on their extraction efficiency which, along with their cognitive exchange rate, dictates the work the receiver pays. Semantic loss is any phase of the system where meaning is lost, but sender loss and receiver loss are distinct types of loss. Then, cognitive utility is the work performed along the system's path of least cognition towards a particular goal, and cognitive efficiency is the path of least cognition compared against the total work the system performed.

Semiodynamics introduces four principles 1) meaning and utility of signals are relative, 2) predictable structure aids efficiency, 3) motivated agents leverage predictable structure, and 4) real-world success requires networks of specialized nodes. It also introduces seven initial mathematical relations for 1) cognitive utility, 2) semantic extraction, 3) semantic encoding, 4) cognitive exchange rate, 5) receiver loss, 6) sender loss, and 7) cognitive efficiency.

This is just a beginning for semiodynamics, and there are many future directions to explore such as introducing more system dynamics over time, how the substrate affects the relationship between work and energy cost, the impact of feedback loops, more concrete understanding of units, and different treatments for the path of least cognition.

Introduction

The modern world has become almost completely dependent on information processing systems. This started in the mid-20th century with the rise of the information age and has grown to a point where almost every aspect of society relies on computers in some way. This integration wasn't a random happenstance; our dependence on computer-based technologies is a story of innovation, economics, and our ability to extract utility from those technologies. While this adoption hasn't been a smooth growth curve, with hype-cycles, economic bubbles, and disappointments, the overarching trend upwards has been undeniable.

Within the broader spaces of information technologies, the field of artificial intelligence (AI) emerged in the mid-20th century. AI showed initial great promise, but has had an extremely uneven growth trajectory towards our current AI systems. Up until the 2010s, most commercial use of AI has involved applying machine learning (ML) techniques to specialized, narrow task domains, typically for classification and recommendation tasks. Even though they were specialized tasks, their integration into our worldwide information systems meant they have had a huge societal impact. The public awareness of how the information we see online is directed by "the algorithms"¹⁰ is a primary example. But it wasn't until the late 2010s that the state of AI science, the hardware lottery⁹, and available training data coalesced into the current worldwide explosion in AI adoption. The rapid emergence of large language models (LLMs), and generative AI (GenAI) more broadly, has unlocked a dramatic, step-function level change in the general purpose capabilities of AI-backed information systems. As is the case with so many disruptive technologies throughout history, we are still in the early phases of figuring out how far we can push the technology, how best to sustain its power demands, where we can utilize it, and the societal implications of doing so.

This early phase of AI has similarities to the rise of heat engines in the 18th to 19th centuries. In the early 19th century Sadi Carnot, a French military engineer and physicist, was trying to better understand how steam engines worked and was reflecting upon their impact on society. Carnot saw the revolutionary impact that steam engines could have on society, as general-purpose sources of power to perform mechanical work. For example, the (translated) quote below from Carnot could easily be transposed into a modern framing of AI becoming a universal motor for cognitive work that humans want to delegate:

The study of these engines is of the greatest interest, their importance is enormous, their use is continually increasing, and they seem destined to produce a great revolution in the civilized world. Already the steam-engine works our mines, impels our ships, excavates our ports and our rivers, forges iron, fashions wood, grinds grains, spins and weaves our cloths, transports the heaviest burdens, etc. It appears that it must some day serve as a universal motor, and be substituted for animal power, waterfalls, and air currents. Over the first of these motors it has the advantage of economy, over the two others the inestimable advantage that it can be used at all times and places without interruption. If, some day, the steam-engine shall be so perfected that it can be set up and supplied with fuel at small cost, it will combine all desirable qualities, and will afford to the industrial arts a range the extent of which can scarcely be predicted. Source: Carnot, Sadi. Reflections on the Motive Power of Fire: And Other Papers on the Second Law of Thermodynamics (Dover Books on Physics) (pp. 31-32)

In addition to observing and anticipating the societal trends, Carnot also saw that we lacked a thorough scientific understanding of how heat engines extracted work from heat, both in terms of a theoretical model and the underlying engineering principles. Thankfully for us, Carnot didn't just share his observations; he wrote down his own ideas and reflections in a single, relatively short, book Reflections on the Motive Power of Fire¹². Carnot died at the age of 36 (1832) and his book received relatively little attention in his lifetime; however, his book went on to lay the foundation for modern thermodynamics, thanks to scientists like William Thomson (a.k.a. Lord Kelvin¹³) and Rudolf Clausius who expanded on Carnot's work to establish thermodynamics, in papers such as An Account of Carnot's Theory of the Motive Power of Heat¹⁴ and On the Moving Force of Heat¹⁵.

Carnot's book was focused on two primary questions:

1. Is there a universal limit to the amount of work that can be extracted from a heat source, or can it go on indefinitely?
2. Is there a substance that can improve the efficiency of heat engines over the use of steam?

Carnot explored these questions mostly using reasoning from principles, without even using much math (which would be added later by Clausius). One of Carnot's primary tools of exploration was an idealized, simplified heat engine that we now call a Carnot Engine, which drives a cycle of heat transference that we now call a Carnot Cycle. Carnot's approach of using an idealized system to reason about real-world systems that we don't fully understand is one that's had repeated success throughout the history of science, and is an approach this article will lean on.

As heat engines were to mechanical power, AI holds the promise of becoming a universal cognitive engine that humans can delegate cognitive work to. But, similar to Carnot's time, we lack a sufficiently rich theoretical understanding of what cognition is, how it works, and what its fundamental constraints are. We also lack a deep enough understanding of how the various substrates that cognition arises from, and the resources it operates on, affect its efficiency at performing cognitive work. While we don't have a strong theoretical model for cognition in AI, what we do know is that we can create information processing systems that, when built in certain ways, can perform cognitive tasks. This is similar to the engineers and scientists of the 18th-19th century who were building and using heat engines based on experimentation and phenomenological modeling. We can also study and observe intelligent systems in nature, which provide inspiration and experimental evidence that cognitive systems can operate more efficiently than our current AI. Given AI's potential to revolutionize how humans outsource cognitive work, it's important that we continue searching for a theoretical understanding, particularly given AI's current extreme power and training data demands, along with its contradictory behavior of simultaneously being superhumanly smart, while also being clueless and unreliable in ways we wouldn't tolerate from another human that we delegated work to.

The current AI wave that the industry is riding is based on the architectural paradigm of deep neural networks (DNNs), in particular LLMs. The success of DNNs is rooted in applying relatively simple techniques, at vast scales, to produce emergent behavior, as famously framed by Richard Sutton in The Bitter Lesson²³. LLMs have taken this lesson of scaling through computation to such an extreme that they are butting up against worldwide limits in compute capacity and availability of training data. But while scaling has been wildly successful, if we overly focus on scale, we are waving a metaphorical white flag towards a deeper understanding of what we're building that might provide new breakthroughs. Whether The Bitter Lesson is correct or not, Sutton's point is specifically that "we want AI agents that can discover like we can, not which contain what we have discovered"²³, not that we shouldn't seek new approaches and a deeper theoretical understanding of what we're building.

While theories are desirable, it doesn't mean we have to start by trying to find a grand unified theory. We can be pragmatically guided by real-world challenges when working with AI. There are many ongoing attempts to establish theoretical foundations for AI, such as mechanistic interpretability, geometric deep learning, and active inference. A specific challenge I've been focused on in recent years is the efficiency with which AI can model, message, and manipulate knowledge, which I group under a general heading of semantic efficiency. LLMs are deeply sample inefficient when learning capabilities, particularly in terms of generalization, requiring vast amounts of training data, and almost no ability to continually learn on-the-go, beyond in-context learning. Humans are an existence proof that AI hasn't reached a universal limit for sample and energy efficiency; while we don't know what the actual upper limit is, it's clear we're not there yet. It seems likely that an agent's semantic efficiency is coupled with its architecture, but it also seems like there are general principles that can provide insight into how we build semantically efficient systems. There is interesting research happening in the semantic analysis of signals, such as epiplexity¹⁶ which provides a computationally bounded measurement of structure that can be learned from a signal. However, in this article, we will zoom out to explore more general principles and relations for the relationship between signals, how they convey knowledge, and what utility they provide to a receiver.

A natural starting question someone might ask is "isn't this what information theory already provides us?" My answer is no; information theory is a necessary pillar but ultimately insufficient for the kind of semantic analysis we need theories for. Information theory helps us analyze the information content of a signal, but it's focused on the bits, not what the bits mean. Shannon explicitly stated this in his seminal paper A Mathematical Theory of Communication¹¹ where he wrote "these semantic aspects of communication are irrelevant to the engineering problem." In 1948 this assumption about the irrelevance of semantics was a useful and sensible simplification, given the types of engineering problems Shannon was solving, and the incredible results from Shannon's assumption speak for themselves. Shannon's work became, and remains, foundational to most of the technological advancement in the 20th and 21st centuries, in ways that brought us to our current moment. One of the reasons we've been able to conflate information and semantic content when building information and communication technologies (ICTs) for most of the information age⁵ is because our technologies have been manually crafted, algorithmic systems with narrow, pre-determined functional purposes. They have been forms of crystallized intelligence that have, at most, narrow fluid intelligence-backed capabilities, such as recommendation or personalization. This lack of fluid intelligence in our tools meant that the signals they received had to contain a certain level of structure and specificity for the ICT to reliably perform useful actions. Wherever signals were overly vague or ambiguous, such that interpretation was needed, our tools have required humans-in-the-loop to reduce ambiguity and uncertainty.

This has changed dramatically in the last few years. Modern AI has enabled our ICTs to process more weakly constrained and less specified data, which has made the "semantic aspects of communication" an extremely relevant engineering challenge. We need to look past the "bits" and create AI systems that more efficiently understand what data means and what should be done in response. To help explore this space, this article introduces a conceptual framework called semiodynamics to help model and analyze questions such as "how efficiently can an agent encode meaning into a signal?" or "how much utility can an agent extract from a signal?" Utility is being used broadly here, whether it's the affordances⁷ a signal provides or how a signal helps an agent achieve its goals. Semiodynamics frames the combination of agents, their tools, and the signals they exchange as a cognitive system that is employed to achieve a goal. It then introduces principles and relations to help analyze how meaning flows through the cognitive system and any work the system performs on the way to achieving its goal.

Core concepts

Before getting into the details of semiodynamics, the article will introduce some useful related concepts that will be referenced throughout this article. Most are established, existing terms and concepts, although we'll adapt some with a specific lens for our specific purposes.

Semiotics

Semiotics⁸ is the study of signs, in particular, how signs are used to communicate meaning. It is an umbrella field that spans syntactics, which is about how signs are structured and composed, semantics, which is about what signs mean and represent across many people, and pragmatics, which covers how individuals interpret signs such that signs have personalized, contextual meaning. Semiotic signs can be anything that represents something else, whether sounds, pictures, written words, spoken words, or gestures. Semiodynamics builds upon semiotics by focusing on how signs are transmitted as signals and then what utility the signal provides to the sender and receivers.

Fluid and Crystallized Intelligence

First is the concept of fluid and crystallized intelligence²¹ which are wonderfully self-descriptive terms, introduced by Raymond Cattell^{2 22}, that capture the difference between when you're figuring things out in the moment (fluid) and when you're using knowledge that you've already learned (crystallized). This distinction is something that everyone intuitively understands. We've all picked up a new skill or hobby, started out feeling like there's just so much information to process, constantly re-checking the instructions or remeasuring things to make sure they're right, which is heavily in the fluid intelligence phase. But then, gradually, we develop a "feel" for the task to where it becomes second nature, to the point where maybe the bigger challenge is staying focused rather than being on autopilot; this is where we've moved into crystallized intelligence.

Cognitive artifacts

Another useful concept, which is closely related to crystallized intelligence, is to think of certain tools as being "precomputed" cognitive artifacts^{3 4}. Cognitive artifacts were defined by Don Norman, who said they are "those artificial devices that maintain, display, or operate upon information in order to serve a representational function and that affect human cognitive performance." Per Norman's framing, not all tools are cognitive artifacts since it's not clear that a screwdriver operates on information in the way Norman meant it. However, I'm taking a broader stance in two ways: 1. any tools that were intentionally crafted are a form of cognitive artifact, since they represent a precomputed amount of cognitive work that is crystallized as a tool 2. cognitive artifacts can be both externalized tools in the agent's environment and internal to the agent, such as optimizations within the agent's internal model.

Some other important characteristics of cognitive artifacts are that they enable cognitive effort to be distributed across time and people. In Norman's own words:

The fact that the preparation of the list is done prior to the action has an important impact upon performance because it allows the cognitive effort to be distributed across time and people. This preparatory task, which Hutchins calls "precomputation" (E. Hutchins, 1989, personal communication), can be done whenever convenient, when there are no time pressures or other stresses, and even by a different person than the individual who performs the actions. In fact, precomputation can take place years before the actual event and one precomputation can serve many applications. Source: Cognitive Artifacts, Norman 1991

Norman's original article on cognitive artifacts provides a richer mental model and is packed with insights and valuable concepts. I highly recommend reading the original article³, or a dedicated overview⁴, as I won't capture everything he introduced in this paper. But another key aspect of cognitive artifacts worth bringing in is the system view and personal view of artifacts. Norman distinguishes the system view of an artifact, which is looking at the objective role the artifact performs in the work being done, from the personal view, which captures the subjective experience of the person using the artifact. Norman uses the example of a personal todo list to describe the difference. From the system view, a todo list is a memory-enhancing artifact that externalizes and crystallizes the upfront, precomputed cognitive effort of creating the task list. The personal view of the todo list captures the subjective experience of the person who created, maintains, and has to use the list. To that person, interacting with the todo list over time is itself a task and so it is deeply important how easy to use the todo list is. Additionally, while the todo list framing can just be for an individual, both views scale to many people, such as a shared todo list that many people are updating and reading from.

Norman's views connect to the framing in this work in important ways: the system view naturally maps to our overarching framing of there being a cognitive system that we're trying to model and measure, particularly in terms of cognitive work being performed ahead-of-time and then reused when performing tasks. The personal view maps to our framing that using tools requires cognitive work; how much cognitive work a user performs is influenced by the user-to-tool interface which, in turn, is dependent on the user's individual alignment to the tool's interface. The user-to-tool alignment has many dimensions that have varying levels of flexibility built in. Humans are remarkably adaptable and are continually learning new skills, as we discussed earlier with fluid and crystallized intelligence. Also, many interfaces themselves can be adapted to the needs of the individual user, either through manual configuration or through automated personalization. However, neither we nor the interface are infinitely adaptable, which, for example, is why we have accessibility standards for interfaces.

Structure, Patterns, Constraints, & Models

Structure, like so many of the concepts we're discussing, has an intuitive, everyday framing that we deal with all the time, but it gets slippery when you try to pin down what it actually is. Whether subatomic, galactic, biological, or sociological, structure seems intrinsic to our universe as, without it, everything would just be noise. In this work, we will loosely use the term structure in a couple of ways: first is "the structure of a system" which maps to how the parts of a system are arranged, and second, "how structured a system is" which captures the amount of order (i.e. regularity) in the system's structure. Patterns are the regularities we observe in a system, which may or may not be connected to the system's actual structure. In more technical terms structure is ontic ("what is"), and patterns are epistemic ("what we perceive and know"). This distinction means that 1) structure might exist without us knowing about it, 2) we can incorrectly see patterns that aren't really there, and 3) we can create abstract patterns that aren't necessarily in the structure of the system itself but are useful to us anyway, such as how we think of certain animals as pets when their "petness" isn't part of the animal itself. This is a deep topic that stretches back to antiquity, particularly Plato with his theory of forms⁶. We'll try to stay practical in this article but will touch upon generalization throughout the article.

Wherever structure exists in a signal, we can, in principle, construct constraints that model and describe the patterns we observe. These constraints exist on a spectrum of specificity, with weak constraints permitting more permutations of the information in the signal that satisfy the constraints, and strong constraints producing fewer. The scientific process itself is a process of discovering constraints that help us reliably make useful predictions about the future and to better understand what happened in the past. We discover constraints both through experimentally gathering data and constructing theories about why nature is the way it is. Constraints don't just have to be about physical systems as mathematics similarly defines constraints; it just defines them in an abstract space that we are often then able to apply to the real world around us. Constraints can be thought of from both the objective and subjective perspectives. Objective constraints are how the system actually behaves and should describe the actual system structure, whereas subjective constraints are what we understand about how the system behaves and should describe the patterns we observe. An interesting aspect of constraints is that they can somewhat be thought of as being part of a system, without necessarily being "in" the system the same way the system parts they constrain are. For example, the rules of chess are constraints on how any given chess game is played, but the rules aren't part of the game structure the same way the location of pieces is. Another example is our theory of gravity, which represents constraints about how the universe behaves; but our theory of gravity isn't in the physical universe the same way the physical stuff it describes is (putting aside philosophical questions about physicalism and idealism).

Observing patterns and discovering constraints is just half of the problem; we want to use this knowledge to simulate and predict system behavior in ways that help and guide us. We do this by creating models that are useful approximations of a system, or at least the parts of the system we care about. Models are a broad concept that comes in many varieties, whether static representations of theories, or online, generative, continually updated representations of a dynamical system. In this work we'll focus on models that agents create internally to help them understand and predict their environment, which they perceive through signals, such that they can perform useful actions. These kinds of models aren't just static knowledge of constraints; they map the model onto something in their environment, then use the constraints to simulate and make predictions based on observed patterns. I'm using "in their environment" in a broad sense, where the thing being mapped to can either be a physical "thing" or an abstract idea in the agent's mind. As such, models have both a representational part that relates the model to "things" in the agent's environment and encodes the agent's knowledge of constraints, plus a dynamical part which involves active simulation and prediction about what an actual system will do.

Factorization

Just because information is structured, it doesn't mean it's easy for an agent to process. Whether an agent is receiving, storing internally, or actively utilizing information, the specific shape of the structure affects how well the agent can work with it. For this article, we are more specifically focused on the structure of the knowledge contained within the information, rather than the lower-level structure of the underlying bits, although the lower-level representation clearly impacts the higher-level knowledge representation. We will use the framing of knowledge being well-factored vs. entangled; a framing wonderfully articulated by Kumar, Clune, Lehman & Stanley in their paper The Fractured Entangled Representation Hypothesis²⁰. In the paper, the authors explore how AI model performance connects to their internal representations, and observe that, "in large models, fractured entangled representations (FER) may be degrading core model capacities like generalization, creativity, and (continual) learning."

In the article we will consider knowledge being "well-factored" as a relative quality, where better factorization is relative to the agent working with the knowledge and the task they're applying it to. This idea of knowledge representations being factored towards the agent using it and what they're using it for is a widespread phenomenon. Humans like working with numbers in base-10 whereas computers prefer binary. Technical diagrams that aim to convey how something works are drawn differently from images of the same object that are used to understand how it aesthetically fits into a room¹⁷. And, it is a well-established pattern in software engineering to create multiple representations of the same underlying data that are structured in ways that are optimized for the data access patterns of the system itself, such as with caching, materialized views, and CQRS.

As an example, natural language sits at an interesting point in this structural spectrum. In the software field we often call natural language "unstructured data" even though it is clearly structured across multiple layers of abstraction. Even a simple plain text file has multiple layers of well-factored structure across 1) how the bits are arranged into predefined character encodings (UTF-8) which are aimed to be interpreted by computers, 2) the syntax, grammar, and language structure the characters participate in, and 3) the semantic coherence of what the words mean, even if syntactically correct, which has to be interpreted by the reader. What we mean by natural language being unstructured is about that third layer, the complexity and openness of the constraints involved in interpreting what the language means. But if you present the same UTF-8 text to someone in binary, it won't feel well-factored and efficient to consume at all, even though binary is an easy and efficient format for programmatic, rule-based systems to process. This points to how factorization being relative is deeply connected to the efficiency with which a system can process a signal. The alignment between the geometry of the knowledge in the signal, the architecture of the cognitive system processing it, and the task it's being applied to is a core driver of efficiency. This means we need to ask about factorization as a relative property across the entire cognitive system, not just the signal itself.

Factorization in Nature

Factorization being relative to the signal, agent, and their objective highlights a remarkable property of biological systems that distinguishes them from our current information technologies, including AI. Living organisms are continually adapting themselves to their environment through a myriad of mechanisms, many of which we're still discovering. This adaptation occurs throughout the life of an organism (ontogeny), such as through epigenetic changes and mental processes, and across generations through evolution (phylogeny). The adaptation is in response to the signals that organisms are continually receiving from their environment and, per natural selection, the organisms that can most efficiently and effectively utilize those signals have an advantage. At a cursory glance, it doesn't seem like a gigantic leap for modern AI systems to adapt to their environment, such as via memory and in-context learning, particularly if we can make strides in continual learning. However, what is remarkable about biological life is the extent to which it "rewires" itself over time.

Biological life is able to evolve new sensory, mobility, and energy apparatus to become more factored and aligned to the signals and resources of its environment. Without this quality we wouldn't exist, and it's why life can be found in even some of the harshest environments. Additionally, we usually focus on the adaptation of the higher-level organisms, such as plants, animals, and insects. But those organisms are abstractions on top of deeply nested, interconnected systems of smaller organisms, that themselves individually explore and adapt to their own environments. Whether you zoom in or zoom out, these systems of organisms form ecosystems so complex they are typically beyond our cognitive horizon to fully comprehend, and yet can still find pockets of mutually beneficial adaptation. A key point for this article is that these biological systems are still rooted in the same principles we're discussing for ICTs, such as signals, constraints, structure, and utility. As such, while the focus of this article is ICTs, the success of biological life can be a guide that informs how we evaluate, explore, and scale artificial agents.

Relevant insights include:

1. The concepts we're describing in this article are not just applicable to AI and information systems
2. Biological life has succeeded by forming nested systems of interconnected agents, operating at many different layers of abstraction and specialized, factored to the specific utility they extract from their environment
3. Biological life is continually exploring its environment for useful crystallized tools that are well-factored to its environment
4. Biological life evolves its internal architecture, both at an individual and system level, to be more factored towards extracting utility from its environment

Semiodynamics

Overview

We're all receiving signals continually, all day, every day, from our environment, from each other, and even internal signals from our minds and bodies. But not all signals are equally useful; what is it about each signal that affects how much utility we can extract to help us with our goals? It seems intuitively true that the utility of a signal relates to how much the receiver can learn from it. How much the receiver can learn from the signal then depends on the receiver's internal cognitive state, since different agents will be able to extract different meaning from the same signal. Hearing a sound you can identify as an ambulance siren in your quiet neighborhood tells you a lot more than if you didn't know what the sound was. A chess grandmaster who glances at a chess board sees much more than just the raw information about where the pieces are. There's a whole genre of fiction dedicated to detectives solving cases through "aha!" moments when a single piece of new evidence brings their whole investigation to some fascinating outcome. Then, even for popular online games like Wordle, how useful the feedback signals you get from each new word attempt are completely relative to the player's understanding of the game and language at large. This simple but pervasive observation tells us that we can't just analyze a signal in isolation to evaluate its meaning and utility; we have to assess the signal and the receiver together, as a subjective property of the receiving cognitive system.

But how do you model the subjective meaning and utility of a signal? The answer, at least for the foreseeable future, is that we can't. Internal models for intelligent agents are private, opaque, and inherently complex. We will need to find higher-order "macroscopic" properties of cognitive systems that represent the underlying flow of meaning through the network. My 2025 paper¹ introduces semantic efficiency as one such macroscopic property, measured by how efficiently one agent communicates with another as part of delegating a task to them. The measure has been a useful lens, but semantic efficiency is just one property of the cognitive system it emerges from. The cognitive system itself needs modeling in a way that can be applied to many, ideally any, situations, such that the flow of knowledge, modeling of constraints, cognitive work, and overall utility can be evaluated. Semiodynamics aims to provide this common framework by introducing terms, principles, and relations that can be used to model how cognitive systems send, receive, process, and utilize signals as part of achieving goals. It helps explain trade-offs such as how to optimize the flow of meaning through a system, how the arrangement of the system affects cognitive work, and how work contributes to goals.

In semiodynamics, a cognitive system is a network of cognitive nodes that each receive signals, perform cognitive work, and send signals in response. Cognitive work represents the quantity of cognition performed rather than energy expended, making it more akin to computational complexity¹⁹ than energy-based work in physics. Nodes exist on a spectrum of agency, where an agent is an originator of intent and objectives, with intermediary nodes taking action to support the agent's goal, both of which use cognitive artifacts as crystallized functionality for specific capabilities. Cognitive systems are nested networks that span layers of abstraction, where each node is itself almost always a smaller cognitive system that can be zoomed in on and viewed in isolation. Similar to thermodynamics, cognitive systems are considered closed or open, with closed cognitive systems having no additional meaning entering but still allowing cognitive work to transfer.

Signals are either sent or emitted. A signal is sent if the sender formed it with semantic intent, which only agents or nodes acting on an agent's behalf can do. A signal is emitted when there is no intent behind its formation, which typically means it is a byproduct of another process, such as an environmental signal. Signals enter receiving nodes through interfaces which project signals for the receiver to interact with. Each node has an internal model that captures both what it knows and how it processes signals, in particular knowledge of constraints about how signals should be processed. Nodes vary dramatically in what they know and how they process signals, enabling them to extract different kinds of cognitive utility from the same signal. Cognitive utility is cognitive work performed by a node that moves the system towards an objective, along the path of least cognition.

A received signal has semantic content which represents everything this specific node could learn from this signal with perfect access and unlimited time, but only using the currently available, closed cognitive system. Interfaces are lossy boundaries for cognitive nodes, so semantic availability is a subset of semantic content that represents everything this node could learn from the signal when viewed through the interface. Resources, including time, are finite, so semantic yield represents what the node actually learned from the signal, including use of their cognitive system. Semantic loss represents how much meaning is lost in different phases of the process, including from whose perspective the loss is happening. Sender loss is the amount of the sender's semantic intent that didn't make it into the receiver's semantic yield. Receiver loss is the amount of semantic content that didn't make it into their semantic yield. It is important to note that sender and receiver loss are fundamentally different, subjective views on the same process. A sender's view of loss is bound by what they wanted to communicate, a receiver's loss is bounded by what they could have learned, which is not at all bound by the sender's intent.

When signals are sent, the sender has to encode their intent into the signal. Encoding takes cognitive work, and how much work is affected by the sender's encoding efficiency. Receivers expend cognitive work to extract meaning from a signal, which is similarly affected by the receiver's extraction efficiency. Emitted signals pay no encoding cost as there is no intent behind them, but a receiver expends cognitive work to extract from the signal, whether there was intent or not. In real-world cognitive systems, the difference between sent and emitted signals is more than just a numerical facet; it is an important factor in how dynamical interactions unfold. A sender is likely to observe and adapt based on how their intent was received; emitted signals just don't have this feedback loop built in.

These terms, and more, are defined and collected in the glossary at the end of the article.

See the cognitive system figure in the full article →

Real-world cognitive systems are deeply complex, nested systems that can be viewed at many levels of abstraction. A zoomed-out perspective might capture the top-level interacting nodes, but, if you zoom in, each node will typically be its own distinct cognitive system. You can keep zooming in through these nested systems until you eventually reach a point where it's not clear that the node counts as information processing anymore, with the physical universe being the ground level. Even the framing of nesting is lossy as signals and nodes interact across layers; a "bombshell" news article might have a direct impact on readers, but then ripple throughout the economy or global geopolitics. While cognitive systems "in the wild" are incomprehensibly complex, the ability to utilize higher-order signals is a pervasive, durable pattern we see in both naturally evolved and engineered cognitive systems. Semiodynamics embraces this to try and find useful macroscopic properties that help us better evaluate and engineer cognitive systems. The hope is for these macroscopic properties to remain useful, whether scaled up or down.

At the level of culture, human society can be seen as an ever-growing cognitive system that, through collective cognition and scientific progress, has been expanding its understanding of signals from the universe, where we're becoming increasingly efficient at extracting utility from those signals through technology, although we remain bounded at any given point in history. Similarly, scaled down to fully deterministic, engineered artifacts that are nodes in our broader cognitive system, the more that the node's crystallized, rule-based internal model encodes the signal structure, the more efficiently it can process the signal. This is a common technique in software to improve performance, for example, the more that code behind an API knows about a strict schema that a JSON payload will adhere to, the more optimally the code can process the payload.

See the nested cognitive systems figure in the full article →

Semiodynamic Principles

Semiodynamics adopts the following four principles. They are largely derived from observation, intuition, and reasoning, rather than being claimed as empirically proven facts or laws. The rest of this article and future ones will aim to make the case that they are both broadly correct and, more importantly, a useful way to model cognitive systems, while recognizing the famous adage that "all models are wrong, but some are useful."¹⁸

First Principle

The meaning and utility of a signal are relative to the receiving agent, their objective, and surrounding context. They are not intrinsic to the signal alone.

What a signal means and the utility it provides aren't a property of the signal alone. A signal's meaning and utility come from the relationship between the information in the signal, the receiving agent's model (including situational context), and the agent's objective. The same signal can be hugely useful to one agent and noise to another, or even provide different utility to the same agent at different points in time, depending on the agent's current knowledge and goal. Cognitive utility is about the point-in-time fit between the signal, agent, and goal.

Second Principle

Predictable structure in signals increases the reliability and efficiency with which agents can extract task-specific utility by creating reusable cognitive artifacts. The reliability and efficiency gains are bounded by the amount of structure in the signal, the agent's knowledge of the constraints that describe that structure, and how well-factored or aligned the interfaces between the signal, artifact, and agent are.

When an agent receives signals that contain predictable structure, they can use knowledge of the structure's constraints to create cognitive artifacts that crystallize processing the signals on the agent's behalf. The more constrained structure the signals contain, the more reliable and efficient the artifact can be, as long as the constraints exist within the agent's cognitive horizon. But, the more crystallized the artifact is, the more fixed and specialized the utility it will extract. How much reliability and efficiency an agent gains is then bounded by the amount of structure in the signal, the agent's understanding of those constraints, and how aligned the agent and artifact's interfaces are. Interfaces are important as misunderstanding and misalignment add cognitive friction, increase cognitive work for the agent, and potentially introduce semantic loss. In practice, cognitive artifacts can be internal or external to the agent, whether optimized paths in their internal model, outside their model but still physically internal, or fully externalized into the agent's environment.

Third Principle

Motivated agents leverage the Second Principle to more effectively extract task-specific utility from their environments. How much an agent can gain is bounded by the amount of predictable structure in the environment, the cognitive carrying capacity of the environment, and the cognitive horizon of the agent.

Agents become more efficient by leveraging the Second Principle to create cognitive artifacts that help them more effectively act in their environment. Agents then employ these artifacts in cognitive systems to reduce the cognitive work they would otherwise need to perform. Creating artifacts takes effort, so agents need intrinsic or extrinsic motivators which dictate whether the created artifacts are helpful cognitive capital or a liability. Selective pressure is a broad overarching motivator that can manifest as agents creating artifacts for short-term selective reward, long-term exploratory reward, or even just for fun, play, and curiosity. Additionally, agents often prioritize the collective good across agents, so the motivation might be collective gain at individual cost. Beyond the bounds placed by the Second Principle, how much an agent can gain is also bounded by the cognitive carrying capacity of the environment, which limits how much the agent can externalize cognition into the environment. Agents are also bound by their own cognitive horizon and internal plasticity.

Fourth Principle

Agents most efficiently scale the utility they can extract from their environment by creating layered networks of cognitive artifacts to enact the top-level agent's intent. Each layer can be viewed as a distinct cognitive system that 1) operates at a particular level of abstraction, 2) exchanges signals with its neighbors via interfaces at their boundaries, and 3) supplies cognitive capital to the layer above.

The more agents leverage the Third Principle to scale themselves across different kinds of tasks, the more they run into scaling trade-offs and constraints with all the artifacts they create. There are resource constraints such as materials and storage, cognitive constraints in knowing how and where to use each artifact, and upkeep costs, just to name a few. But real-world goals are usually large and repetitive enough that they can be decomposed into a series of smaller subtasks that repeat and overlap over time. This process of decomposing tasks into subtasks can be recursively applied to find even more repetition and overlap. Per the Second Principle and Third Principle, the more tasks subdivide and repeat, the more benefit there is in assembling artifacts into layered networks of cognitive nodes that form a cognitive system. Nodes in the system communicate via signals, mediated through interfaces at their boundaries, to convey intent and extract meaning with child nodes providing utility to parents. In efficient systems, the layers typically operate across different levels of abstraction, with many different artifacts within a layer focusing on a particular domain of utility. The top-level agent provides the initial intent and objective, which then cascades through the network, with each activated layer providing cognitive capital to the layer above.

---

This email stops at the principles. The seven mathematical relations, the symbol table and glossary, the worked example, and the future directions are in the full article, which also has hover definitions for every symbol: jameshorsley.me/articles/semiodynamics

Read the full article at: jameshorsley.me/articles/semiodynamics