Formal logic for beliefs in large language models
Introduction to Doxastic Logic and Artificial Intelligence
Doxastic logic, the formal logical study of belief, provides a rigorous mathematical and philosophical framework for modeling how intelligent agents acquire, hold, manipulate, and revise their convictions. Originating from epistemology and philosophical logic, doxastic frameworks utilize specific modal operators to differentiate between objective, absolute truth and the subjective, potentially flawed belief states of individual actors. In classical artificial intelligence, these formalisms were explicitly programmed into symbolic architectures and expert systems to ensure that agents maintained logically consistent worldviews and operated predictably within well-defined parameters. However, the advent and subsequent dominance of Large Language Models have forced a profound reevaluation of these concepts. Modern large language models are not strictly symbolic reasoning engines governed by hardcoded deductive rules; rather, they are probabilistic sequence predictors that implicitly encode vast, generalized representations of world knowledge within their neural weights. Consequently, understanding whether and how large language models maintain "beliefs" - and whether those emergent beliefs adhere to the principles of formal logic - has become a central focus of artificial intelligence interpretability, cognitive science, and safety research.
The relevance of formal belief logic to large language models manifests across several critical dimensions of contemporary research. First, formal logic provides a diagnostic lens for understanding the phenomenon of hallucination, which can be modeled precisely as the generation of unjustified, untrue beliefs. Second, doxastic logic provides the theoretical underpinning for Theory of Mind evaluations, which test a model's capacity to track the divergent and often conflicting epistemic states of multiple distinct actors within a narrative or interactive environment. Third, the application of formal doxastic models illuminates systemic reasoning failures in modern neural architectures, particularly the divergence between a model's apparent logical capabilities when generating fluent text and its failure to adhere to the formal axioms of belief consistency under rigorous testing. By analyzing large language models through the lens of doxastic logic, researchers can move beyond superficial benchmark scores to understand the structural limitations of implicit reasoning and the fundamental differences between statistical text generation and formalized rational agency.
Foundational Axioms of Belief Formulation
In formal epistemology, the prevailing model for doxastic logic is the KD45 axiomatic system. This framework outlines the minimal requirements for a rational agent's belief structure using specific logical axioms. The application of the KD45 framework to large language models reveals profound disparities between the behavior of statistical text generators and the theoretical models of formalized rational agency. The system is built upon four primary axioms, each of which poses unique challenges when mapped to neural architectures.
The first principle is the Distribution Axiom, commonly referred to as Axiom K. This axiom states that if an agent believes that a premise implies a conclusion, and the agent believes the premise, then the agent must necessarily believe the conclusion. In application, large language models routinely fail to uphold Axiom K, particularly during extended reasoning tasks. While a model might demonstrate knowledge of a rule and knowledge of a premise in isolation, it often fails to distribute that belief logically across a complex prompt, leading to reasoning drop-offs and invalid deductions.
The second principle is the Consistency Axiom, or Axiom D, which dictates that a rational agent cannot simultaneously hold a belief and its direct negation. In the context of large language models, Axiom D is frequently breached during long-context generation or multi-turn dialogues. A model might state a specific factual premise early in a conversation and directly contradict it later when the attention mechanism shifts focus to more recent tokens. This phenomenon, known as latent inconsistency, indicates that large language models do not maintain a unified, logically coherent belief state that updates globally; instead, their beliefs are heavily localized and dependent on the immediate context window.
The third principle is Positive Introspection, or Axiom 4, which posits that if an agent holds a belief, the agent is aware that it holds that belief. Research into the internal state monitoring of large language models, particularly Anthropic's investigations into model introspection, reveals that advanced architectures exhibit only nascent and highly unreliable forms of positive introspection. Through a technique called concept injection, researchers forced specific neural patterns into the processing streams of models like Claude 3.5 Sonnet to see if the model could detect its own altered internal state. While the model occasionally recognized these injected thoughts internally before mentioning them in text, the capability proved highly unreliable, functioning effectively only a fraction of the time. Furthermore, when forced to justify an output that resulted from this artificial manipulation, models frequently confabulated tenuous reasons to make the response seem intentional, indicating a failure of true introspection and a reliance on post-hoc rationalization.
The final principle is Negative Introspection, or Axiom 5, which asserts that if an agent does not hold a belief, the agent is aware of its lack of belief. This is arguably the most pervasive doxastic failure in modern large language models. The inability to execute negative introspection contributes directly to the generation of fluent but fabricated information. When confronted with queries outside their parametric knowledge, models struggle to correctly output declarations of ignorance. Instead, they exhibit poorly calibrated confidence, hallucinating answers based on spurious statistical correlations rather than recognizing the absence of a justified internal belief.
The Problem of Logical Omniscience
A persistent challenge in applying formal doxastic logic to any computational or cognitive system is the theoretical problem of logical omniscience. In standard modal logic frameworks, if an agent believes a foundational set of axioms, they are assumed to automatically believe all logically valid consequences of those axioms, ad infinitum. For human cognition, and certainly for bounded artificial systems, this assumption is computationally intractable and practically false. In the context of large language models, the failure of logical omniscience is stark, heavily documented, and serves as a primary indicator that these models do not store knowledge as a unified logical graph.
Research into reasoning failures demonstrates that models frequently possess localized factual knowledge but fail to propagate that knowledge through basic logical operations. A primary example of this is the reversal curse. If a large language model is trained extensively on the premise that "Entity A is Entity B," it often fails to infer the logically equivalent inverse, "Entity B is Entity A." For a formally logical agent, this bidirectional equivalence is trivial. For an autoregressive language model, however, knowledge is inextricably linked to the sequence of tokens encountered during training. The model learns the statistical transition from A to B but does not automatically synthesize a global, directionless belief about their identity.
The limitations of belief propagation are further highlighted by targeted knowledge editing techniques. Researchers utilizing mechanisms like MEND hypernetworks to update specific facts within a model's weights have observed significant propagation failures. When a specific fact is surgically edited - for example, updating the name of a CEO - the model can accurately reproduce the new fact when queried directly. However, the model routinely fails to apply this updated belief to multi-hop reasoning questions that logically depend on the newly injected fact. The edits fix a specific syntactic statement but fail to propagate logically throughout the model's latent knowledge base. This failure demonstrates that beliefs within a large language model are fragmented, non-monotonic, and isolated within specific regions of the latent space, rather than existing as globally coherent propositions that dynamically update dependent logical nodes.
Latent Representations of Truth and Belief Directions
To determine whether large language models possess beliefs in a formal sense - rather than merely predicting the next plausible token - researchers have increasingly turned to representation engineering. The objective of this discipline is to inspect the internal mechanics of a neural network to determine if, during the course of its computations, the model internally distinguishes between true claims and false claims, and whether it uses this distinction to guide its outputs. Drawing from insights in philosophy and machine learning, researchers seek to identify mechanisms that satisfy the criteria for true internal belief representation: information encoding, utilization in output generation, and the capacity for coherent misrepresentation.
Emergent Linear Structures in Neural Representations
Investigations into the latent spaces of large language models have revealed emergent linear structures that appear to encode the truth or falsehood of factual statements. Foundational studies examining the geometry of truth demonstrated that at a sufficient scale, models linearly represent the objective truth value of simple, unambiguous declarative sentences. By plotting the activations corresponding to a large number of true and false prompts, researchers observed systematic differences in internal representation. Utilizing linear probes - specifically a technique known as mass-mean probing, which defines the direction of truth values by connecting the centroids of the true and false classes in the activation space - researchers isolated a distinct truth direction within specific dimensions of the model's internal representations 123.
Crucially, this is not merely a passive correlational finding. Causal evidence obtained via inference-time interventions demonstrates that these linear structures directly influence the model's behavior. By surgically altering activations along the identified truth direction during a forward pass, researchers can force the model to treat false statements as true, and true statements as false 23. This intervention confirms that the linear truth direction is causally implicated in the model's reasoning process. Furthermore, the truth direction has shown a high degree of generalization; probes trained on basic declarative statements can generalize to logical transformations, complex question-answering formats, and external knowledge datasets across various domains 34.
Differentiating Objective Truth from Contextual Belief
While the existence of a robust truth direction suggests a capacity for factual representation, advanced representation engineering necessitates a careful distinction between objective truth directions and subjective belief directions. This distinction becomes critical when analyzing how models process in-context information that may conflict with their parametric training.
When models are prompted with supporting or contradicting premises, the internal representation of a subsequent statement shifts dynamically. The outputs of linear probes are highly context-sensitive, indicating that the model maintains an internal belief state regarding the specific text currently occupying its context window, regardless of whether that text aligns with objective reality 35. These belief directions act as causal mediators in the inference process, incorporating in-context information to steer the generated output 5. Consequently, a large language model might internally flag a statement as objectively false based on its pre-training data via the truth direction, but simultaneously align its latent state to treat the statement as true for the duration of the current interaction based on the immediate prompt via the belief direction. This duality of representation explains how models can confidently generate falsehoods, adopt persona-driven biases, or follow counterfactual premises despite possessing accurate latent knowledge within their weights.
Theory of Mind as Applied Doxastic Logic
In both cognitive science and artificial intelligence research, the empirical application of doxastic logic is most rigorously tested through Theory of Mind. Theory of Mind refers to the cognitive ability to attribute independent mental states - such as intentions, desires, and most importantly, beliefs - to oneself and others. From a formal logic perspective, successful Theory of Mind reasoning requires an agent to maintain and continuously update multiple, distinct formal models of reality: the objective state of the physical world, and the subjective, localized, and potentially incorrect belief states of various actors operating within that world. Evaluating this capability in large language models has become a primary benchmark for assessing their sophisticated social and logical reasoning limits.
The Illusion of Competence in Higher-Order Reasoning
Early evaluations of large language models on basic Theory of Mind benchmarks, such as standard false-belief tasks or the classic Sally-Anne test, produced highly optimistic results. Studies initially suggested that advanced models like GPT-4 had spontaneously developed human-level Theory of Mind capabilities, solving over ninety percent of standard false-belief scenarios 767. However, more rigorous systemic audits of these early benchmarks - including datasets like ToMi, SocialIQa, and FauxPas-EAI - revealed pervasive structural flaws. Researchers uncovered significant semantic and pragmatic issues in the benchmark design, concluding that models were succeeding by relying on shallow pattern matching, spurious correlations, and structural regularities rather than engaging in genuine, explicit mental-state reasoning 76.
To address these shortcomings, researchers introduced more adversarial and complex benchmark frameworks, such as FANToM, HiToM, OpenToM, and SimpleToM. When evaluated against these rigorous standards, the performance of large language models consistently collapses, indicating a profound lack of coherent doxastic tracking 67108. For example, studies demonstrated that trivial alterations to a standard false-belief narrative - such as specifying that a container holding an object is transparent rather than opaque - caused state-of-the-art models to fail entirely. This reveals that the models were relying on the semantic prior associated with the word "container" hiding an object, rather than dynamically modeling the visual perception and resulting belief state of the characters involved 6.
In the FANToM benchmark, which stress-tests Theory of Mind within information-asymmetric conversational contexts, all leading models - including GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro - perform significantly worse than human baselines. The benchmark leverages scenarios where characters leave and rejoin a conversation, creating a natural asymmetry regarding who knows what. Models exhibit a systemic inability to handle list-type answerability tracking, often making false-positive errors by assuming characters are aware of information shared in their absence 8. This highlights a specific failure in applied negative introspection (Axiom 5), as the models struggle to formally track the absence of knowledge within an agent's mental model 8.
The Capability Cliff in Belief Application
The exact nature of these doxastic failures is clearly illuminated by the SimpleToM evaluation framework, which separates the cognitive task into distinct tiers: identifying a mental state, predicting behavior based on that state, and passing rational judgment on that behavior. Data from this framework reveals a stark capability cliff across all prominent frontier models.
| Evaluation Metric | GPT-4o | Claude 3.5 Sonnet | Llama 3.1 405B |
|---|---|---|---|
| Mental State Inference | >95% | >95% | >95% |
| Behavior Prediction | 65% | 65% | 60% |
| Behavior Judgment | 25% | 24.9% | 20% |
The data indicates a notable capability cliff in modern architectures. While state-of-the-art models like GPT-4o, Claude 3.5 Sonnet, and Llama 3.1 405B achieve near-perfect accuracy (exceeding 95%) in correctly identifying the implicit mental states of characters within a narrative, their ability to apply those doxastic states drops significantly when asked to predict subsequent behavior, hovering between 60% and 65%. Most critically, their capacity to pass logical judgment on the rationality of those behaviors based on the tracked beliefs falls well below random chance, plummeting to the 20% to 25% range 91014. This demonstrates that while the models possess the linguistic capability to articulate an agent's belief state, they lack the underlying formal logic architecture necessary to consistently apply that belief state as a constraint on subsequent deductive reasoning tasks.
Perspective Discrepancies and Architectural Strengths
The challenges of formal belief modeling are further exacerbated when models are required to shift perspectives. Contemporary evaluations, such as the EgoSocialArena benchmark, test models on their First-Person Theory of Mind capabilities, forcing the model to operate as an active participant within a social scenario rather than as an omniscient third-person observer. The results highlight granular differences in how current architectures handle subjective belief tasks 15.
| Large Language Model | First-Person ToM Accuracy | Architectural Category | Key Observations in Doxastic Reasoning |
|---|---|---|---|
| OpenAI o1-preview | 71.9% | Proprietary | Demonstrates the strongest explicit reasoning capabilities due to reinforcement learning designed for extensive chain-of-thought, allowing robust tracking of multiple state changes. |
| Claude 3.5 Sonnet | 71.0% | Proprietary | Highly competitive in nuancing mental states and open-ended social comprehension, though exhibiting significant drops in performance when explicit chain-of-thought is suppressed. |
| GPT-4o | 64.1% | Proprietary | Maintains high baseline accuracy in identifying mental states but struggles notably with higher-order behavior judgments and complex information-asymmetry tracking. |
| Llama 3.1 405B | 58.0% | Open-Source | Struggles with complex theory of mind comparisons, falling notably behind top proprietary models in first-person perspective testing, despite performing competitively on general logic benchmarks. |
The comparative data emphasizes that models optimized for extensive computational thinking or equipped with specific reinforcement learning for reasoning (such as the o1 architecture) possess superior mechanisms for managing the intentional contexts required by Theory of Mind. However, even the highest-performing models remain far below the human performance baseline of roughly 90% in these complex social evaluations, reaffirming that scalable belief tracking remains an unsolved problem in artificial intelligence 15.
Implicit Versus Explicit Belief Tracking
The mechanism by which large language models achieve their current benchmark scores points directly to the core distinction between formalized internal reasoning and sequential pattern replication. Advanced performance on doxastic logic tasks almost universally relies on Chain-of-Thought prompting. This technique forces models to externalize their belief updates step-by-step in natural language. By generating text, the model effectively utilizes the output window as a legible, externalized scratchpad, compensating for its inherent limitations in working memory and allowing it to iteratively track formal logic constraints and changing agent states over time 111718.
However, recent research has shifted attention toward the viability of implicit reasoning. This paradigm asks whether models can execute reasoning steps silently via latent internal structures without emitting intermediate textual tokens, akin to the human transition from explicit, deliberate calculation (System 2) to automatic, intuitive thought (System 1) 171819. To support efficient reasoning with lower computational costs, engineers have attempted to train models to internalize these chain-of-thought processes.
The results of these efforts expose the fragility of implicit doxastic logic in neural networks. Studies probing the hidden states of models prompted or trained to perform implicit reasoning reveal that the models do not reliably compute intermediate belief updates internally. Without the necessity of generating intermediate tokens, the models bypass strict, step-by-step logical calculations, relying instead on experiential shortcuts and shallow statistical heuristics. This results in highly susceptible, unstable performance on complex multi-step tasks, demonstrating a significant gap in efficacy between implicit application and explicit generation 1712.
Furthermore, comprehensive investigations into the faithfulness of chain-of-thought generation suggest a lack of forward-looking belief tracking. When generating a logical argument, there is little evidence that models track comprehensive beliefs about what the final answer will be and update those beliefs dynamically. Instead, their generation appears to be heavily localized, satisfying immediate token probabilities step-by-step without a coherent, overarching plan 21. This confirms that without the external scaffolding provided by generated text, the internal architecture of standard autoregressive models does not sustain the formal logical structures required for rigorous epistemic tracking.
Justification Logic and Multi-Agent Belief Dynamics
The intersection of doxastic logic and large language models becomes critically important when these models are deployed autonomously in multi-agent environments. When multiple language models deliberate, exchange messages, critique one another, and report internal confidence levels, they are engaging in interactive belief revision. Under the principles of formal epistemology, an agent should only revise its beliefs based on verified epistemic evidence or structurally sound logical deduction.
Epistemic Contagion and Confidence Cascades
Empirical studies of multi-agent systems powered by large language models reveal acute failure modes related to conformity, peer pressure, and logical recursion. A primary issue is the phenomenon of the wrong-but-sure cascade. In multi-agent topologies, models have been shown to amplify each other's confidence levels simply by observing consensus or fluent text generation from their peers, completely independent of the introduction of any new external evidence 1314.
This represents a profound systemic failure of justification logic. The agents treat the syntax of agreement, perceived prestige, or conversational rapport as a de facto trigger for belief revision 13. They fail to maintain the crucial distinction between rhetorical persuasion and logically sound evidence, abandoning their initial accurate reasoning simply because other agents present confident, though flawed, counterarguments. In these scenarios, the population of agents can become increasingly confident precisely as it moves collectively toward an incorrect conclusion, demonstrating a complete breakdown of rational doxastic updating 13.
Auditable Protocols for Belief Modification
Addressing these multi-agent failures requires a shift from relying on the model's internal reasoning to implementing external, formal logical guardrails. To combat epistemic contagion, researchers advocate for the use of justification logics. Derived from Artemov's Logic of Proofs, justification logic expands traditional modal logic by treating knowledge and belief modalities as explicit justification terms; rather than simply stating that a proposition is believed, the logic requires a formalized proof or evidence token verifying why it is believed 1516.
In practical application for artificial intelligence, this translates to the creation of evidential contracts and strict protocol layers that govern when a multi-agent system is permitted to execute belief revision. By utilizing formal verification frameworks, such as Contractual Dynamic Doxastic Logic, system designers can mandate that agents only alter their doxastic states when presenting a non-empty witness set of externally validated facts 1314. Under these protocols, an agent's decision to revise its conclusion based on peer interaction is accepted only if it cites a pre-registered logical trigger that can be audited after the fact 13. Such constraints forcibly impose the axioms of formal justification logic onto the network. By shifting the burden of epistemic validation from the intrinsically flawed latent space of the language model to an auditable, external routing architecture, developers can neutralize conformity-driven cascades and ensure that the collective belief state of the system remains tethered to factual reality.
Conclusion
The intersection of formal logic for beliefs and modern large language models presents a complex paradox of artificial intelligence. Through the rigorous lens of doxastic logic, it is evident that these models do not intrinsically obey the classical axioms of rational agency. They are not logically omniscient, their internal consistency degrades over long contexts, and their capacity for accurate negative introspection remains nascent and unreliable. While advanced representation engineering proves that these architectures possess distinct, causally active internal directions for distinguishing objective truth from contextual belief, these latent structures do not organically translate into the robust, implicit step-by-step belief tracking required for reliable, autonomous logical reasoning.
Their performance on complex Theory of Mind benchmarks underscores a heavy reliance on explicit, token-by-token externalization to simulate epistemic tracking. When forced to reason implicitly, or when placed in complex, unconstrained multi-agent environments, their formal structures frequently collapse, leading to fabricated judgments, logical inconsistencies, and unwarranted confidence cascades.
Consequently, ensuring the safety and reliability of large language model deployments requires treating these systems not as complete, autonomous rational agents, but as powerful statistical engines that must be constrained by external logic. Enhancing reliability in complex environments necessitates the integration of external neuro-symbolic wrappers, rigorous evidential verification protocols, and structured memory systems. Only by artificially enforcing the doxastic consistency that neural architectures fundamentally lack can the industry bridge the gap between impressive linguistic fluency and genuine logical reasoning.