In this study, we investigated whether patterns of gene expression in larvae feeding on different plants can explain important aspects of the evolution of insect-plant associations, such as phylogenetic conservatism of host use and re-colonization of ancestral hosts that have been lost from the host repertoire. To this end, we performed a phylogenetically informed study comparing the transcriptomes of 4 nymphalid butterfly species in Polygonia and the closely related genus Nymphalis. Larvae were reared on Urtica dioica, Salix spp., and Ribes spp. Plant-specific gene expression was found to be similar across butterfly species, even in the case of host plants that are no longer used by two of the butterfly species. These results suggest that plant-specific transcriptomes can be robust over evolutionary time. We propose that adaptations to particular larval food plants can profitably be understood as an evolved set of modules of co-expressed genes, promoting conservatism in host use and facilitating re-colonization. Moreover, we speculate that the degree of overlap between plant-specific transcriptomes may correlate with the strength of trade-offs between plants as resources and hence to the probability of colonizing hosts and complete host shifts.