Hebbian learning

Dosidicus implements a simplified, computational version of Hebbian learning to simulate emergent squid behaviors and personality traits, drawing inspiration from biological principles but abstracting them for gameplay and interpretability.

The neural network in Dosidicus does not use traditional backpropagation for training. Instead, it employs a form of Hebbian learning, a biologically-inspired principle summarized as "neurons that fire together, wire together." This method allows the network to learn associations and patterns organically based on the squid's concurrent states, without requiring a separate training phase. The entire process is managed within the perform_hebbian_learning and update_connection methods.

1. The Learning Cycle

Learning is not continuous but occurs in discrete cycles to ensure stability and reduce computational load.

• Timed Trigger: The learning cycle is initiated by a timer. The interval for this timer is configurable in config.ini under the [Hebbian] section's learning_interval parameter (default every 30000 milliseconds).
• Pre-Pruning: Before each learning cycle begins, a pruning function is called to remove extremely weak and old connections from the network. This helps maintain network efficiency by clearing out irrelevant pathways before new learning occurs.

2. The Core Learning Process

The perform_hebbian_learning method executes a precise sequence of steps to update the network's weights.

1. Identify Active Neurons: The system first scans all neurons in the brain. Any neuron whose activation value is above the active_threshold defined in the configuration is considered "active" for this learning cycle. System-level neurons (e.g., is_eating, direction) are excluded from this process.
2. Random Pair Sampling: If fewer than two neurons are active, the learning cycle is aborted. If there are enough active neurons, the system does not update all possible pairs. Instead, it randomly samples a small number of pairs (e.g., two) from the pool of active neurons. This stochastic approach introduces variability and prevents the network from over-stabilizing into rigid patterns.
3. Update Connection Weight: For each selected pair, the update_connection method is called. This is where the core weight calculation happens:
   • Base Hebbian Rule: The fundamental change in weight is calculated by multiplying the two neurons' normalized activation values by a learning rate. This reinforces the connection between them.
   • Dynamic Learning Rate: The learning rate is not static. If one of the neurons in the pair was recently created via neurogenesis, the learning rate is temporarily boosted (e.g., by a factor of 2.0). This allows new, specialized neurons to integrate into the network more quickly and form meaningful connections.
   • Weight Decay: To prevent runaway weight growth and to help the network "forget" insignificant associations, a small decay factor is applied during the update. This factor, configured via weight_decay in config.ini, slightly reduces the magnitude of the connection's weight during each update.
   • Clamping: The final calculated weight is always clamped to a range of [-1.0, 1.0] to keep it normalized and prevent extreme values from destabilizing the network.

3. Visual Feedback

The learning process is tied directly to the application's user interface to provide clear, real-time feedback.

• Activity Log: In the "Learning" tab of the Brain Tool, a log entry is created for each learning event, explicitly stating which neuron pair had its connection strengthened or weakened and by how much.
• Network Animation: In the "Network" tab, the connection line between the learning pair will briefly glow or pulse. The color of the pulse indicates whether the weight increased (positive reinforcement) or decreased (negative reinforcement), providing an immediate visual cue of the learning event.