first pass on search-based overview of Axon. needs better integration with existing.

Author: rcoreilly

content/axon.md

@@ -15,33 +15,63 @@ A fundamental challenge for any theoretical understanding of something as comple
 
 In this context, a primary goal in generating the extensive content here on [compcogneuro.org](https://compcogneuro.org) is to document the full extent to which the current version of Axon captures the existing scientific findings in neuroscience and cognitive psychology, so that the interested reader may form their own opinion about the extent to which the model provides an accurate picture of brain function. Feedback on any and all such issues is encouraged, using the [github discussions](https://github.com/compcogneuro/web/discussions) forum. The [github issues](https://github.com/compcogneuro/web/issues) can be used to report typos or other such "bugs", and pull requests for suggested fixes or other contributions are always welcome, and provide a way to document contributions.
 
-Ultimately, the definitive step in the scientific method is through direct empirical tests of the specific predictions from the model, of which there have been a large number over the years, as documented in the relevant places herein. Perhaps the most central such test is reported in [[Jiang et al 2025]], which directly tests the [[temporal derivative]] form of [[synaptic plasticity]] that drives learning in the Axon model.
+Ultimately, the definitive step in the scientific method is direct empirical tests of the specific predictions from the model, of which there have been a large number over the years, as documented in the relevant places herein. Perhaps the most central such test is reported in [[Jiang et al 2025]], which directly tests the [[temporal derivative]] form of [[synaptic plasticity]] that drives learning in the Axon model.
+
+## Computational motivation
+
+The computational-level process of [[search]] provides an overarching framework for motivating the neural mechanisms in Axon, from basic neural processing elements including spiking and synaptic weights, to the critical role of the goal-driven [[Rubicon]] system.
+
+The central insight is that every computational problem or process can be formulated in terms of searching through a space of alternatives, to find a state that satisfies various criteria. For example, the game of chess can be understood in terms of searching through the very large space of possible board configurations for sequences of such configurations that lead to winning the game. Indeed, this is precisely what the [Deep Blue](https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)) chess computer did to beat grandmaster Gary Kasparov, via an optimized "brute force" search process.
+
+The fundamental computational challenge for any search process is the [[curse of dimensionality]] as the size (dimensionality) of the space being searched increases. Each additional dimension adds an _exponential_ factor to the size of the space. For example, a binary space with 8 dimensions has $2^8 = 256$ states, but one with twice the number of dimensions (16) already has $2^{16} = 65,536$ elements. Any "real world" space almost certainly has an intractably large number of elements. See the [[search]] page for more details on how this relates to the computational complexity literature and the space of _NP-complete_ problems.
+
+From this search-based perspective, the reason [[artificial intelligence]] (AI) used to be generally ineffective is because it relied on fundamentally _serial_, symbolic mechanisms to search through the relevant spaces, so it just broke down when confronting the complexity of the real world. And the reason AI is now successful is because [[abstract neural network|neural networks]] can perform efficient search in high-dimensional spaces, via _stochastic gradient descent_ operating on networks of _dedicated-parallel_ elements with weighted synaptic connections. There are two versions of this process, at different time-scales:
+
+* [[Error backpropagation]] learning computes the _partial derivatives_ (i.e., gradients) across a network of interconnected neuron-like processing elements to efficiently search the space of possible representations, to find those that enable specific problems to be solved. 
+
+* [[Constraint satisfaction]] processing in [[bidirectional connectivity|bidirectionally connected]] networks also uses gradients to efficiently search the space of possible representations to find those that are consistent with both the constraints imposed by external inputs to the network, and constraints from the learned synaptic weights (i.e., the _knowledge_ in the network).
+
+Thus, at this computational level, a major hypothesis behind the Axon framework is that these gradient-based search mechanisms using dedicated-parallel neural hardware are _essential_ for successful computation in the face of the complexity of the real world. Critically, most extant AI models, including the [[large language models]] powering ChatGPT and the like, depend on the power of error backpropagation learning, but do not have bidirectional connectivity and only have more limited forms of constraint satisfaction processing.
+
+By contrast, the Axon framework is based foundationally on the bidirectional connectivity present in the [[neocortex]], which drives both the specific biologically-based form of error backpropagation it uses, and a full network-wide constraint satisfaction process at every step of neural processing. Conceptually, this means that an Axon network is computing with [[optimized representations]] at every step, which has important implications for how it behaves in challenging task environments. In effect, it can efficiently search a massive space of possible representations within roughly 200 ms, to find a way of solving problems that might otherwise take an intractable amount of time to find.
+
+The specific problem of serial trial-and-error behavior and learning, which is the domain of [[reinforcement learning]] (RL), poses a particularly critical challenge from the search perspective, because it is _inevitably_ serial. You can't take multiple different strategies to solve a problem at the same time, and you can't carry out multiple different action sequences at the same time. This _serial bottleneck_ of action ([[@Pashler94]]) makes it absolutely essential to employ highly optimized search processes wherever possible, to ensure that each such sequential step is as effective as possible. Otherwise, the curse of dimensionality will require an intractably large number of serial action steps to search spaces of even modest sizes.
+
+Indeed, current RL algorithms require mind-boggling numbers of learning trials to learn even relatively simple video games, and much of the research is focused on ways of managing and distributing this massive search process. This has given RL a bad reputation among learning algorithms, but it is essential to appreciate that it is not that RL algorithms are inherently stupid: it is just that the RL domain is inherently intractable due to is serial-search nature.
+
+In addition to the use of dedicated-parallel, gradient-based constraint-satisfaction processing at each time step, another critical way to mitigate the problem of serial action is to have an evolutionarily-optimized system of neural mechanisms that carefully guards the allocation of this precious serial search resource. This is the [[Rubicon]] goal-driven processing system in the Axon framework, which continuously learns about the benefit vs. cost tradeoffs associated with actions taken, and applies this knowledge to carefully evaluate the selection of new goals to pursue going forward.
+
+Thus, you can blame this goal-driven part of your brain when you end up procrastinating instead of sinking a lot of time into a challenging or boring task. This is just your very rational, evolutionarily-optimized brain saying: "don't do something that takes a lot of effort, especially if it is not urgently necessary right now!" Instead, it makes much more sense to take on shorter, simpler tasks that you know you can do with high confidence, and will give you a decent little kick of [[dopamine]] when you knock them off the list.
+
+Given that any organism living in the real world has to somehow solve the challenge of choosing sensible action plans based on an extremely limited ability to search the relevant problem space, understanding the full complement of neural mechanisms that make this work efficiently is thus a very high priority. This is reflected here in the significant effort invested to understand the often bewildering complexity of the extensive subcortical and cortical networks that implement the goal-driven action selection and engagement process.
 
 ## Axon mechanisms
 
-These are the central elements of Axon in terms of neural and computational mechanisms, most of which are well established properties of the mammalian [[neocortex]]:
+The central elements of Axon in terms of neural and computational mechanisms are as follows (most of which are well established properties of the mammalian [[neocortex]]):
 
-* [[neuron|Spiking neurons]] with relatively long-lasting [[neuron channels#NMDA]] and [[neuron channels#GABA-B]] channels that support [[stable activation]] states over the course of a roughly 200 msec [[theta rhythm|theta cycle]], which is essential for establishing a coherent representation of the current input state. This stability is necessary to drive effective learning
+* [[neuron|Spiking neurons]] with relatively long-lasting [[neuron channels#NMDA]] and [[neuron channels#GABA-B]] channels that support [[stable activation]] states over the course of a roughly 200 msec [[theta rhythm|theta cycle]], which is essential for establishing a coherent representation of the current input state. This stability is necessary to drive effective learning using the temporally-extended [[temporal derivative]] mechanism.
 
     The Axon neuron model is fairly conventional from a computational neuroscience perspective, featuring two compartments (soma and [[neuron dendrites|dendrite]]) using a range of conductance-based electrophysiologically-accurate [[neuron channels|channels]] and the widely-used _AdEx_ adaptive exponential approximation to the full Hodgkin-Huxley spiking dynamics ([[@BretteGerstner05]]).
 
-    The discrete spiking behavior of these neurons enables effective graded information integration over time in a way that continuous [[rate code activation]] communication does not, by allowing many different signals to be communicated over time, competing for the overall control of the network activation state as a function of the collective integration of spikes within the neurons in the network. As a result, Axon models are overall much more robust and well-behaved overall compared to their [[Leabra]] rate-code based counterparts.
+    The discrete spiking behavior of these neurons enables effective graded information integration over time in a way that continuous [[rate code activation]] communication does not, by allowing many different signals to be communicated over time, competing for the overall control of the network activation state as a function of the collective integration of spikes within the neurons in the network. As a result, Axon models are overall much more robust and well-behaved overall compared to their [[Leabra]] rate-code based counterparts, especially with respect to [[constraint satisfaction]] computation.
 
 * [[Error-driven learning]] based on errors computed via a [[temporal derivative]] that naturally supports [[predictive learning]], as the difference over time of network activity states representing the prediction followed by the outcome. Local [[synaptic plasticity]] based on the competition between kinases updating at different rates, i.e., the [[kinase algorithm]], naturally computes the error gradient via the temporal derivative dynamic. The result is a fully biologically plausible form of the computationally powerful [[error backpropagation]] algorithm, as shown by the [[GeneRec]] algorithm. Initial empirical support for this mechanism is reported in [[Jiang et al 2025]], in electrophysiological measurements of synaptic plasticity in a rodent preparation.
 
-    The combination of robust error-driven learning and biologically-detailed spiking neurons in Axon enables these neurons to learn to perform arbitrary computational and cognitive tasks. Furthermore, the availability of a clear computational measure of performance in terms of overall learning capability across a wide range of tasks has enabled the optimization of all the biological parameters to maximize learning performance. There is a consistent set of such parameters that generally works best across all the tasks investigated to date, and thus the additional degrees of freedom associated with these parameters are generally eliminated from consideration in constructing new models, greatly reducing the effective complexity of the model.
+    The combination of robust error-driven learning and biologically-detailed spiking neurons in Axon enables these neurons to learn to perform arbitrary computational and cognitive tasks. Furthermore, the availability of a clear computational measure of performance in terms of overall learning capability across a wide range of tasks has enabled the optimization of all the biological parameters to maximize learning performance. There is a consistent set of such parameters that generally works best across all the tasks investigated to date, and thus the additional degrees of freedom associated with these parameters are generally eliminated from consideration in constructing new models, greatly reducing the effective degrees of freedom of the model.
 
-* [[Bidirectional connectivity]] among excitatory neurons, which is necessary for propagating error signals throughout the network, and pooled [[inhibition]] which is necessary for controlling the effects of bidirectional excitatory connectivity, while also having beneficial computational effects in terms of [[attention]] and competition. Bidirectional connectivity also supports [[constraint satisfaction]] dynamics that can efficiently [[search]] through large high-dimensional knowledge spaces to find (and synthesize) the most relevant information given the current bottom-up (sensory) and top-down (goals) constraints. This results in the use of [[optimized representations]] at each step of processing, which has significant computational advantages.
+* [[Bidirectional connectivity]] among excitatory neurons, which is necessary for propagating error signals throughout the network, and pooled [[inhibition]] which is necessary for controlling the effects of bidirectional excitatory connectivity, while also having beneficial computational effects in terms of [[attention]] and competition. As noted above, bidirectional connectivity supports [[constraint satisfaction]] dynamics that can efficiently [[search]] through large high-dimensional knowledge spaces to find (and synthesize) the most relevant information given the current bottom-up (sensory) and top-down (goals) constraints. This results in the use of [[optimized representations]] at each step of processing, which has significant computational advantages.
 
     Perhaps most importantly, this bidirectional connectivity is widely thought to be essential for [[conscious awareness]] ([[@Lamme06]]), which is likely critical for the system to access its own internal state of knowledge. This ability is notably absent in current [[abstract neural network]] models that drive the widely-used [[large language models]] (LLMs) for example, which are notorious for their inability to accurately evaluate their own knowledge states, resulting in significant _confabulation_. Most experts do not think these models are conscious, which is consistent with the fact that they are based exclusively on feedforward connectivity.
 
     The central role of bidirectional connectivity in Axon represents one of the most important points of divergence relative to the vast majority of existing neural network models (along with the combination of biologically-detailed spiking dynamics and error-driven learning), and testing the functional importance of this property is a major overarching goal of this research.
 
-These basic neural mechanisms are sufficient to learn well-established functions of the posterior [[neocortex]], including spatially invariant [[object recognition]], spatial processing, and generally developing powerful internal predictive models of the world. See the discussion on [[combinatorial vs. conjunctive]] representations for a summary of the computational-level properties of these learned representations, and how they support effective [[generalization]] to novel situations.
+These neural mechanisms provide the dedicated-parallel, gradient-based foundation for the learning and processing associated with the posterior [[neocortex]], which learns powerful representations for encoding, predicting, and inferring states of the world around us. Specific models illustrating these abilities are discussed in the context of spatially invariant [[object recognition]], [[spatial attention]], and [[language]]. See the discussion on [[combinatorial vs. conjunctive]] representations for a summary of the computational-level properties of these learned representations, and how they support effective [[generalization]] to novel situations.
 
 ## General intelligence and motivation
 
-If Axon truly captures the essential computational capabilities of the human neocortex, then in principle it should be able to provide a novel and powerful framework for [[artificial intelligence#artificial general intelligence]] (AGI), because humans remain the only undisputed entities capable of fully general intelligence. However, based on what we know about the cognitive neuroscience of human intelligence, it is clear that many other brain areas with specialized functions relative to the posterior neocortex play critical roles as part of a larger overall set of interacting systems required to support our general intelligence abilities.
+The parallel, distributed processing of the posterior neocortex is only part of the overall picture for understanding the full scope of mammalian intelligent behavior. As noted above, there are substantial evolutionarily optimized subcortical and cortical mechanisms that carefully guard the serial aspects of cognition and behavior, which emerge out of the parallel brain mechanisms over time. These serial modes, while inefficient relative to dedicated-parallel search, have the unique advantage of being significantly more _flexible_.
+
+Indeed the seemingly-magical universal computational capacity of the [[Turing machine]] derives precisely from the intrinsic flexibility of serial processing, where arbitrary sequences of basic information processing operations can be combined to accomplish any task. Thus, in order to understand the full power of human intelligence (e.g., to create a true [[artificial intelligence#artificial general intelligence]] or  AGI), we also need to understand how this serial processing emerges from the parallel neural substrates.
 
 Computationally, these systems support the functionality associated with [[reinforcement learning#model-based]] reinforcement learning, which is necessary for organizing behavior over time to accomplish goals. Without these mechanisms, the neocortex alone would be a largely passive learning and perceiving system without the ability to organize behavior effectively over time (basically what you experience while dreaming, when the [[prefrontal cortex]] is selectively deactivated).