The striking variation in brain morphology across the animal kingdom and the link to behaviour has fascinated scientists for centuries. Several factors coexist and interact during the evolution of brain morphology. Together with the complexity of brain function and morphology, evolutionary biologists have been challenged when seeking to identify general principles of how and why brains evolve. Extensive comparative research has identified patterns and formed hypotheses on the link between brain morphology and behaviour. Artificial selection on various aspects of brain morphology can complement comparative research and test such hypotheses on the relationship between brain morphology and behaviour.

In this thesis, I used an experimental approach to investigate two aspects of rapid changes in brain morphology and how such changes are linked to behaviour. The specific aims were to (i) examine the relationship between relative brain size and cognitive abilities, and (ii) brain region size and collective behaviours. First, we tested several aspects of learning in guppies artificially selected on relative brain size. We found that brain size improves cognitive abilities, but that cognitive divergence is mostly quantitative at the intraspecific level (paper 1). We also examined the effect of brain size on cognitive ageing. We found that while more fundamental aspects of cognitive abilities were maintained throughout the ecologically relevant lifespan in guppies, behavioural flexibility declined faster in large brain size selected guppies (paper II). Second, we assessed collective motion and collective decision-making in guppy shoals artificially selected for small or large telencephalon size. We found no effect of telencephalon size on collective motion when exploring an open arena (paper III). However, collective decision-making to avoid a model predator was faster in large telencephalon size selected guppy shoals (paper IV). This result suggests rapid mosaic changes in brain region size may be an important mechanism behind social behavioural variation with strong fitness implications. Taken together, this thesis strengthens the theory that increased investment in brain tissue can improve advanced cognitive abilities. While more fundamental cognitive abilities are unaffected by changes in brain morphology. It also indicates that mosaic brain evolution can be a highly cost-efficient driver of cognitive divergence.