Malcolm A. MacIver

Consciousness arose through natural selection. Yet, few accounts suggest how it garnered selective benefit. Fleming & Michel provide a testable thesis: separating the real from the hypothetical, triggered by the profligate generativity of land for possible futures, is essential for a land animal to survive. Unexplained in the account is conscious planning, important for humans and perhaps other animals.

Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death'' for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.

Outside of the laboratory, animals behave in spaces where they can transition between open areas and coverage as they interact with others. Replicating these conditions in the laboratory can be difficult to control and record. This has led to a dominance of relatively simple, static behavioral paradigms that reduce the ethological relevance of behaviors and may alter the engagement of cognitive processes such as planning and decision-making. Therefore, we developed a method for controllable, repeatable interactions with others in a reconfigurable space. Mice navigate a large honeycomb lattice of adjustable obstacles as they interact with an autonomous robot coupled to their actions. We illustrate the system using the robot as a pseudo-predator, delivering airpuffs to the mice. The combination of obstacles and a mobile threat elicits a diverse set of behaviors, such as increased path diversity, peeking, and baiting, providing a method to explore ethologically relevant behaviors in the laboratory.

Navigation requires steering and propulsion, but how spinal circuits contribute to direction control during ongoing locomotion is not well understood. Here, we use drifting vertical gratings to evoke directed “fictive” swimming in intact but immobilized larval zebrafish while performing electrophysiological recordings from spinal neurons. We find that directed swimming involves unilateral changes in the duration of motor output and increased recruitment of motor neurons, without impacting the timing of spiking across or along the body. Voltage-clamp recordings from motor neurons reveal increases in phasic excitation and inhibition on the side of the turn. Current-clamp recordings from premotor interneurons that provide phasic excitation or inhibition reveal two types of recruitment patterns. A direction-agnostic pattern with balanced recruitment on the turning and nonturning sides is primarily observed in excitatory V2a neurons with ipsilateral descending axons, while a direction-sensitive pattern with preferential recruitment on the turning side is dominated by V2a neurons with ipsilateral bifurcating axons. Inhibitory V1 neurons are also divided into direction-sensitive and direction-agnostic subsets, although there is no detectable morphologic distinction. Our findings support the modular control of steering and propulsion by spinal premotor circuits, where recruitment of distinct subsets of excitatory and inhibitory interneurons provide adjustments in direction while on the move. SIGNIFICANCE STATEMENT Spinal circuits play an essential role in coordinating movements during locomotion. However, it is unclear how they participate in adjustments in direction that do not interfere with coordination. Here we have developed a system using larval zebrafish that allows us to directly record electrical signals from spinal neurons during “fictive” swimming guided by visual cues. We find there are subsets of spinal interneurons for coordination and others that drive unilateral asymmetries in motor neuron recruitment for direction control. Our findings suggest a modular organization of spinal premotor circuits that enables uninterrupted adjustments in direction during ongoing locomotion.

From wolves to ants, many animals are known to be able to hunt as a team. This strategy may yield several advantages: going after bigger preys together, for example, can often result in individuals spending less energy and accessing larger food portions than when hunting alone. However, it remains unclear whether this behavior relies on complex cognitive processes, such as the ability for an animal to represent and anticipate the actions of its teammates. It is often thought that ‘collaborative hunting’ may require such skills, as this form of group hunting involves animals taking on distinct, tightly coordinated roles – as opposed to simply engaging in the same actions simultaneously. To better understand whether high-level cognitive skills are required for collaborative hunting, Tsutsui et al. used a type of artificial intelligence known as deep reinforcement learning. This allowed them to develop a computational model in which a small number of ‘agents’ had the opportunity to ‘learn’ whether and how to work together to catch a ‘prey’ under various conditions. To do so, the agents were only equipped with the ability to link distinct stimuli together, such as an event and a reward; this is similar to associative learning, a cognitive process which is widespread amongst animal species. The model showed that the challenge of capturing the prey when hunting alone, and the reward of sharing food after a successful hunt drove the agents to learn how to work together, with previous experiences shaping decisions made during subsequent hunts. Importantly, the predators started to exhibit the ability to take on distinct, complementary roles reminiscent of those observed during collaborative hunting, such as one agent chasing the prey while another ambushes it. Overall, the work by Tsutsui et al. challenges the traditional view that only organisms equipped with high-level cognitive processes can show refined collaborative approaches to hunting, opening the possibility that these behaviors may be more widespread than originally thought – including between animals of different species.

From wolves to ants, many animals are known to be able to hunt as a team. This strategy may yield several advantages: going after bigger preys together, for example, can often result in individuals spending less energy and accessing larger food portions than when hunting alone. However, it remains unclear whether this behavior relies on complex cognitive processes, such as the ability for an animal to represent and anticipate the actions of its teammates. It is often thought that ‘collaborative hunting’ may require such skills, as this form of group hunting involves animals taking on distinct, tightly coordinated roles – as opposed to simply engaging in the same actions simultaneously. To better understand whether high-level cognitive skills are required for collaborative hunting, Tsutsui et al. used a type of artificial intelligence known as deep reinforcement learning. This allowed them to develop a computational model in which a small number of ‘agents’ had the opportunity to ‘learn’ whether and how to work together to catch a ‘prey’ under various conditions. To do so, the agents were only equipped with the ability to link distinct stimuli together, such as an event and a reward; this is similar to associative learning, a cognitive process which is widespread amongst animal species. The model showed that the challenge of capturing the prey when hunting alone, and the reward of sharing food after a successful hunt drove the agents to learn how to work together, with previous experiences shaping decisions made during subsequent hunts. Importantly, the predators started to exhibit the ability to take on distinct, complementary roles reminiscent of those observed during collaborative hunting, such as one agent chasing the prey while another ambushes it. Overall, the work by Tsutsui et al. challenges the traditional view that only organisms equipped with high-level cognitive processes can show refined collaborative approaches to hunting, opening the possibility that these behaviors may be more widespread than originally thought – including between animals of different species.