The sixth mass extinction is well under way, with biodiversity disappearing at unprecedented rates in terms of species richness and biomass. At the same time, given the currentpace, we would need the next two centuries to complete the inventory of life on Earthand this is only one of the necessary steps toward monitoring and conservation of species. Clearly, there is an urgent need to accelerate the inventory and the taxonomic researchrequired to identify and describe the remaining species, a critical bottleneck. Arguably, leveraging recent technological innovations is our best chance to speed up taxonomic research. Given that taxonomy has been and still is notably visual, and the recent break-throughs in computer vision and machine learning, it seems that the time is ripe to exploreto what extent we can accelerate morphology-based taxonomy using these advances inartificial intelligence. Unfortunately, these so-called deep learning systems often requiresubstantial computational resources, large volumes of labeled training data and sophisticated technical support, which are rarely available to taxonomists. This thesis is devoted to addressing these challenges. In paper I and paper II, we focus on developing an easy-to-use (’off-the-shelf’) solution to automated image-based taxon identification, which is at the same time reliable, inexpensive, and generally applicable. This enables taxonomists to build their own automated identification systems without prohibitive investments in imaging and computation. Our proposed solution utilizes a technique called feature transfer, in which a pretrained convolutional neural network (CNN) is used to obtain image representations (”deep features”) for a taxonomic task of interest. Then, these features are used to train a simpler system, such as a linear support vector machine classifier. In paper I we optimized parameters for feature transfer on a range of challenging taxonomic tasks, from the identification of insects to higher groups --- even when they are likely to belong to subgroups that have not been seen previously --- to the identification of visually similar species that are difficult to separate for human experts. In paper II, we applied the optimal approach from paper I to a new set of tasks, including a task unsolvable by humans - separating specimens by sex from images of body parts that were not previously known to show any sexual dimorphism. Papers I and II demonstrate that off-the-shelf solutions often provide impressive identification performance while at the same time requiring minimal technical skills. In paper III, we show that phylogenetic information describing evolutionary relationships among organisms can be used to improve the performance of AI systems for taxon identification. Systems trained with phylogenetic information do as well as or better than standard systems in terms of common identification performance metrics. At the same time, the errors they make are less wrong in a biological sense, and thus more acceptable to humans. Finally, in paper IV we describe our experience from running a large-scale citizen science project organized in summer 2018, the Swedish Ladybird Project, to collect images for training automated identification systems for ladybird beetles. The project engaged more than 15,000 school children, who contributed over 5,000 images and over 15,000 hours of effort. The project demonstrates the potential of targeted citizen science efforts in collecting the required image sets for training automated taxonomic identification systems for new groups of organisms, while providing many positive educational and societal side effects.

Rapid and reliable identification of insects is important in many contexts, from the detection of disease vectors and invasive species to the sorting of material from biodiversity inventories. Because of the shortage of adequate expertise, there has long been an interest in developing automated systems for this task. Previous attempts have been based on laborious and complex handcrafted extraction of image features, but in recent years it has been shown that sophisticated convolutional neural networks (CNNs) can learn to extract relevant features automatically, without human intervention. Unfortunately, reaching expert-level accuracy in CNN identifications requires substantial computational power and huge training data sets, which are often not available for taxonomic tasks. This can be addressed using feature transfer: a CNN that has been pretrained on a generic image classification task is exposed to the taxonomic images of interest, and information about its perception of those images is used in training a simpler, dedicated identification system. Here, we develop an effective method of CNN feature transfer, which achieves expert-level accuracy in taxonomic identification of insects with training sets of 100 images or less per category, depending on the nature of data set. Specifically, we extract rich representations of intermediate to high-level image features from the CNN architecture VGG16 pretrained on the ImageNet data set. This information is submitted to a linear support vector machine classifier, which is trained on the target problem. We tested the performance of our approach on two types of challenging taxonomic tasks: 1) identifying insects to higher groups when they are likely to belong to subgroups that have not been seen previously and 2) identifying visually similar species that are difficult to separate even for experts. For the first task, our approach reached 92% accuracy on one data set (884 face images of 11 families of Diptera, all specimens representing unique species), and 96% accuracy on another (2936 dorsal habitus images of 14 families of Coleoptera, over 90% of specimens belonging to unique species). For the second task, our approach outperformed a leading taxonomic expert on one data set (339 images of three species of the Coleoptera genus Oxythyrea; 97% accuracy), and both humans and traditional automated identification systems on another data set (3845 images of nine species of Plecoptera larvae; 98.6 % accuracy). Reanalyzing several biological image identification tasks studied in the recent literature, we show that our approach is broadly applicable and provides significant improvements over previous methods, whether based on dedicated CNNs, CNN feature transfer, or more traditional techniques. Thus, our method, which is easy to apply, can be highly successful in developing automated taxonomic identification systems even when training data sets are small and computational budgets limited. We conclude by briefly discussing some promising CNN-based research directions in morphological systematics opened up by the success of these techniques in providing accurate diagnostic tools.

Computer vision has made dramatic progress in recent years on image classification tasks. There are now numerous success stories, where supervised learning of convolutional neural networks using large training sets has resulted in impressive classification performance. Nevertheless, these systems occasionally make grave errors that humans would never make. One reason for this may bethat the training algorithms are over-simplified. They typically use binary scores, 1 for the correct target and 0 otherwise. Recently it has been shown that label smoothing, that is, distributing a small portion of the score equally on all “wrong” categories, can improve performance in some cases. Both of these methods assume that image categories are equally distant, but we often have backgroundknowledge about the similarity relations between them, which can be useful in developing more sophisticated methods. Here, we explore the utility of phylogenetic information in training and evaluating CNNs for identification of biological species. Specifically we propose label smoothing based on taxonomic information (taxonomiclabel smoothing) or distances between species in a reference phylogeny (phylogenetic label smoothing). Using two empirical examples (38,000 images of 83 species of snakes, and 2,600 images of 153 species of butterflies and moths), we show that networks trained with phylogenetic information perform at least as well on common performance metrics as standard systems, while making errors that are more acceptable to humans and less wrong in an objective biological sense. We argue that this is likely to make the systems more robust and better at placing unseen categories or unusual images correctly. We demonstrate the potential power of the approach by showing that a phylogenetically informed system is better at identifying venomous snakes than a system trained using standard methods.

Automating taxonomic identification has been a dream for many insect systematists, as this could free up time for more challenging scientific endeavours, such as circumscribing and describing new species or inferring phylogeny. The last decades have seen several attempts to develop systems for automated identification, but it has required significant skills and effort to develop them, the systems have only been able to handle specific tasks, and they have rarely been able to compete in performance with human taxonomists. With recent advances in computer vision and machine learning, this situation is now rapidly changing. Here, we illustrate several practical perspectives on the current state-of-the-art in automated identification using a case study, the discrimination of ten species of the flower chafer beetle genus Oxythyrea, and an imagined conversation between an insect taxonomist and an expert in artificial intelligence. Modern automated identification is based on training convolutional neural networks (CNNs) using extremely large training sets, so-called deep learning. In previous work, we have shown that the size of the training sets required for insect identification can be reduced significantly by using CNNs pretrained on general image classification tasks. Here we demonstrate that high-resolution images are not needed; adequate training sets for insect identification can be collected cheaply and rapidly using smartphone cameras with an attachable zoom lens. Furthermore, we demonstrate that easy-to-use off-the-shelf solutions are often sufficient even for challenging identification tasks that are impossible for humans, such as separating the sexes from the dorsal habitus in these beetles. Finally, we demonstrate recent techniques that can highlight the morphological features used by the trained systems in discriminating species. This can be helpful in developing a better understanding of how the species differ. We hope that these results will encourage insect systematists to explore some of the many exciting opportunities that modern AI tools offer.

Citizen scientists contribute large volumes of critical biodiversity data but the taxonomiccoverage is limited and the species determinations are not always reliable. Both problems couldbe addressed, at least in part, by developing better identification tools. Thanks to the dramaticprogress in AI-based computer vision technology, the interest in automated speciesidentification systems has exploded in recent years. However, these techniques often requirelarge training sets of accurately labeled images, which are unavailable for most organismgroups. In the Swedish Ladybird Project, we engaged school children in collecting photos of oneof these groups, ladybird beetles (Coccinellidae), to improve AI identification tools. We estimatethat more than 15,000 school children participated in the project. Children were provided with anapp for submitting photos and macro lenses fitting onto mobile device cameras, while theirteachers received instructions and educational materials focused on the diversity and biology ofladybird beetles. Over the summer of 2018, participants collected more than 5,000 photos of 30species of coccinellids. The project added substantially to the openly-licensed ladybird imagesavailable previously from the GBIF portal, more than doubling the number of images for fourspecies. Adding the project images to the GBIF data improved AI identification accuracy for allbut the most common ladybird species. We conclude that citizen-science projects targetingteachers and school children can be an effective way of improving AI identification systems forbiodiversity while providing many positive educational and societal side effects.