Cognitive flexibility can enhance the ability to adjust to changing environments. Here, we use learning simulations to investigate the possible advantages of flexible learning in volatile (changing) environments. We compare two established learning mechanisms, one with constant learning rates and one with rates that adjust to volatility. We study an ecologically relevant case of volatility, based on observations of developing cleaner fish Labroides dimidiatus that experience a transition from a simpler to a more complex foraging environment. There are other similar transitions in nature, such as migrating to a new and different habitat. We also examine two traditional approaches to volatile environments in experimental psychology and behavioral ecology: reversal learning, and learning set formation (consisting of a sequence of different discrimination tasks). These provide experimental measures of cognitive flexibility. Concerning transitions to a complex world, we show that both constant and flexible learning rates perform well, losing only a small proportion of available rewards in the period after a transition, but flexible rates perform better than constant rates. For reversal learning, flexible rates improve the performance with each successive reversal because of increasing learning rates, but this does not happen for constant rates. For learning set formation, we find no improvement in performance with successive shifts to new stimuli to discriminate for either flexible or constant learning rates. Flexible learning rates might thus explain increasing performance in reversal learning but not in learning set formation, and this can shed light on the nature of cognitive flexibility in a given system. Animals need to adjust to changes that occur in their environment, such as new food types becoming available or old food types becoming unsuitable. Learning about these changes could be essential for success, in particular, if the environment is complex, with many things to learn about. When changes happen, it might be advantageous to quickly learn about new things. We use computer simulations of learning to investigate how big the advantage might be.

Empiricists often struggle to apply game theory models to real-life cases of animal cooperation. One reason is that many examples of cooperation occur in stable groups, where individuals form social bonds that influence exchanges of help in ways that are not well described by previous models, including the extent of reciprocity and how relationships are initiated. We present a game theory model exploring the conditions under which social bonds between group members promote cooperation. In the model, bonds build up from exchanges of help in a similar way as the strength of association increases in learning, as in the Rescorla–Wagner rule. The bonds in turn affect partner choice and influence helping amounts. The model has a mechanism of reciprocity for bonded pairs, which can evolve toward either loose or strict reciprocation. Several aspects of the model are inspired by observations of food sharing in vampire bats. We find that small social neighborhoods are required for the evolutionary stability of helping, either as small group sizes, or if bonded members of larger groups can form temporary (daily) smaller groupings. The costs of helping need to be fairly low, while the benefits can be substantial. The form of reciprocity that evolves is neither immediate nor very strict. Individuals in need request help based on bond strength, but there is also an evolved preference for initiating bonds with new group members. In contrast, if different groups come into temporary contact, the evolved tendency is to avoid forming bonds between groups.

Learning is a taxonomically widespread process by which animals change their behavioural responses to stimuli as a result of experience. In this way, it plays a crucial role in the development of individual behaviour and underpins substantial phenotypic variation within populations. Nevertheless, the impact of learning in social contexts on evolutionary change is not well understood. Here, we develop game theoretical models of competition for resources in small groups (e.g. producer-scrounger and hawk-dove games) in which actions are controlled by reinforcement learning and show that biases in the subjective valuation of different actions readily evolve. Moreover, in many cases, the convergence stable levels of bias exist at fitness minima and therefore lead to disruptive selection on learning rules and, potentially, to the evolution of genetic polymorphisms. Thus, we show how reinforcement learning in social contexts can be a driver of evolutionary diversification. In addition, we consider the evolution of ability in our games, showing that learning can also drive disruptive selection on the ability to perform a task.

Game theory in biology gained prominence 50 years ago, when Maynard Smith & Price formulated the concept of an evolutionarily stable strategy (ESS). Their aim was to explain why conflicts between animals of the same species usually are of a ‘limited war’ type, not causing serious injury. They emphasized that game theory is an alternative to previous ideas about group selection, which were used by ethologists to explain limited aggression. Subsequently, the ESS concept was applied to many phenomena with frequency dependence in the evolutionary success of strategies, including sex allocation, alternative mating types, contest behaviour and signalling, cooperation, and parental care. Both the analyses of signalling and cooperation were inspired by similar problems in economics and attracted much attention in biology. Here we give a perspective on which of the ambitions in the field have been achieved, with a focus on contest behaviour and cooperation. We evaluate whether the game-theoretical study of the evolution of cooperation has measured up to expectations in explaining the behaviour of non-human animals. We also point to potentially fruitful directions for the field, and emphasize the importance of incorporating realistic behavioural mechanisms into models.

Variation in stress responses has been investigated in relation to environmental factors, species ecology, life history and fitness. Moreover, mechanistic studies have unravelled molecular mechanisms of how acute and chronic stress responses cause physiological impacts (‘damage’), and how this damage can be repaired. However, it is not yet understood how the fitness effects of damage and repair influence stress response evolution. Here we study the evolution of hormone levels as a function of stressor occurrence, damage and the efficiency of repair. We hypothesise that the evolution of stress responses depends on the fitness consequences of damage and the ability to repair that damage. To obtain some general insights, we model a simplified scenario in which an organism repeatedly encounters a stressor with a certain frequency and predictability (temporal autocorrelation). The organism can defend itself by mounting a stress response (elevated hormone level), but this causes damage that takes time to repair. We identify optimal strategies in this scenario and then investigate how those strategies respond to acute and chronic exposures to the stressor. We find that for higher repair rates, baseline and peak hormone levels are higher. This typically means that the organism experiences higher levels of damage, which it can afford because that damage is repaired more quickly, but for very high repair rates the damage does not build up. With increasing predictability of the stressor, stress responses are sustained for longer, because the animal expects the stressor to persist, and thus damage builds up. This can result in very high (and potentially fatal) levels of damage when organisms are exposed to chronic stressors to which they are not evolutionarily adapted. Overall, our results highlight that at least three factors need to be considered jointly to advance our understanding of how stress physiology has evolved: (i) temporal dynamics of stressor occurrence; (ii) relative mortality risk imposed by the stressor itself versus damage caused by the stress response; and (iii) the efficiency of repair mechanisms.

Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer–scrounger, caller–satellite, or hawk–dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action–value learning and actor–critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action–value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.

Social hierarchies are often found in group-living animals. The hierarchy position can influence reproductive success (RS), with a skew towards high-ranking individuals. The amount of aggression in social dominance varies greatly, both between species and between males and females within species. Using game theory we study this variation by taking into account the degree to which reproductive competition in a social group is mainly local to the group, emphasizing within-group relative RS, or global to a larger population, emphasizing an individual's absolute RS. Our model is similar to recent approaches in that reinforcement learning is used as a behavioural mechanism allowing social-hierarchy formation. We test two hypotheses. The first is that local competition should favour the evolution of mating or foraging interference, and thus of reproductive skew. Second, decreases in reproductive output caused by an individual's accumulated fighting damage, such as reduced parenting ability, will favour less intense aggression but should have little influence on reproductive skew. From individual-based simulations of the evolution of social dominance and interference, we find support for both hypotheses. We discuss to what extent our results can explain observed sex differences in reproductive skew and social dominance behaviour.

1. Social hierarchies are often found in group-living animals and can be formed through pairwise aggressive interactions. The dominance rank can influence reproductive success (RS) with a skew towards high-ranking individuals.
2. Using game theory, we investigate how the opportunity for differently ranked individuals to achieve RS influences the costs of hierarchy formation and the strength of winner and loser effects.
3. In our model, individuals adjust their aggressive and submissive behaviour towards others through reinforcement learning. The learning is based on rewards and penalties, which depend on relative fighting ability. From individual-based simulations, we determine evolutionary equilibria of traits such as learning rates. We examine situations that differ in the extent of monopolisation of contested RS by dominants and in the proportion of total RS that is contested.
4. The model implements two kinds of fighting costs: a decrease in effective fighting ability from damage (loss of condition) and a risk of mortality that increases with the total accumulated damage. Either of these costs can limit the amount of fighting.
5. We find that individuals form stable dominance hierarchies, with a positive correlation between dominance position and fighting ability. The accumulated costs differ between dominance positions, with the highest costs paid by low or intermediately ranked individuals. Costs tend to be higher in high-skew situations.
6. We identify a ‘stay-in, opt-out’ syndrome, comprising a range from weaker (stay-in) to stronger (opt-out) winner–loser effects. We interpret the opt-out phenotype to be favoured by selection on lower ranked individuals to opt out of contests over social dominance, because it is more pronounced when more of the total RS is uncontested.
7. We discuss our results in relation to field and experimental observations and argue that there is a need for empirical investigation of the behaviour and reproductive success of lower ranked individuals.

The ability to dominate conspecifics and thereby gain access to resources depends on a number of traits and skills. Experience of dominance relationships during development is a potential source of learning such skills. We here study the importance of social experience, aggressiveness and morphological traits for competitiveness in social interactions (contest success) in male domestic fowl (Gallus gallus domesticus). We let males grow up either as a single (dominant) male or as an intermediately ranked male in a group of males, and measured their success in duels against different opponents. We found that single-raised males had lower contest success than group-raised males, and that aggression and comb size correlated positively with contest success. This indicates that experience of dominance interactions with other males increases future success in duels. We similarly studied the consequences of growing up as a dominant or subordinate in a pair of males, finding no statistically significant effect of the dominance position on contest success. Finally, we found that males were consistent over time in contest success. We conclude that social experience increases contest success in male domestic fowl, but that certain behavioural and morphological characteristics have an equal or even stronger covariation with contest success.

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple ‘unbiased’ reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual’s behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is ‘rational’.