Despite the common assumption that the brain is malleable to surrounding conditions mainly during ontogeny, plastic neural changes can occur also in adulthood. One of the driving forces responsible for alterations in brain morphology is increasing environmental complexity that may demand enhanced cognitive abilities (e.g. attention, memory and learning). However, studies looking at the relationship between brain morphology and learning are scarce. Here, we tested the effects of both learning and environmental enrichment on neural plasticity in guppies (Poecilia reticulata), by means of either a reversal-learning test or a spatial-learning test. Given considerable evidence supporting environmentally induced plastic alterations, two separate control groups that were not subjected to any cognitive test were included to account for potential changes induced by the experimental setup alone. We did not find any effect of learning on any of our brain measurements. However, we found strong evidence for an environmental effect, where fish given access to the spatial-learning environment had larger relative brain size and optic tectum size in relation to those exposed to the reversal-learning environment. Our results demonstrate the plasticity of the adult brain to respond adaptively mainly to environmental conditions, providing support for the environmental enhancement theory.

The vertebrate brain displays enormous morphological variation and the quest to understand the evolutionary causes and consequences of this variation has spurred over a century of research. The mosaic brain evolution hypothesis, stating that brain regions can evolve relatively independently, is a highly influential idea in this research field. Here we provide the first experimental support for this hypothesis through an artificial selection experiment in the guppy (Poecilia reticulata). After three generations of selection on relative telencephalon volume in replicated up-selected, down-selected and control selection lines, we found substantial overall changes in relative telencephalon size (i.e. relative to brain size), but no changes in other brain regions. The differences were not evident at birth but present at the time of sexual maturation. There was a non-significant trend towards asymmetry in the response to selection in both sexes, with larger changes occurring during upwards selection as opposed to downwards selection. Our results demonstrate that independent evolutionary changes in specific brain regions can be an important mechanism during cognitive evolution.

It has long been acknowledged that individuals consistently differ in their behaviour over time and across contexts, often defined as animal personality. Much of the existing research into personality traits in animals have focussed on the bold-shy continuum, given its implications in fitness and relatively straight-forward quantification. Recently, a substantial research effort has focussed on how brain morphology affects the level of boldness. Using recently established artificial selection lines with known differences in telencephalon size, a brain region important for decision making, we tested this potential link with a test battery for boldness (emergence test, open field test and novel object test). We did not detect an effect of telencephalon size on overall level of boldness. However, supporting previous findings we found a strong sex-difference in behavioural patterns whereby male guppies were generally more active and more likely to emerge from the start compartment, presumably due to different life-history strategies. Our results thus do not support any direct link between the size of the telencephalon and boldness at the intraspecific level.

The telencephalon is a brain region believed to have played an essential role during cognitive evolution in vertebrates. However, till now, all the evidence on the evolutionary association between telencephalon size and cognition stems from comparative studies. To investigate the potential evolutionary association between cognitive abilities and telencephalon size experimentally, we used male guppies artificially selected for large and small telencephalon relative to the rest of the brain. We tested a functionally important aspect of executive cognitive ability, inhibitory control, in a detour task and found that males with larger telencephalon outperformed males with smaller telencephalon. Theyshowed faster improvement in performance during detour training and weremore successful in reaching the food reward without touching the transparent barrier. Together, our findings provide the first experimental evidence showing that evolutionary enlargements of relative telencephalon size confer cognitive benefits, supporting an important role for mosaic brain evolution during cognitive evolution.