Changes in behaviour may initiate shifts to new adaptive zones, with physical adaptations for novel environments evolving later. While new mutations are commonly considered engines of adaptive change, sensory evolution enabling access to new resources might also arise from standing genetic diversity, and even gene loss. We examine the relative contribution of molecular adaptations, measured by positive and relaxed selection, acting on eye expressed genes associated with shifts to new adaptive zones in ecologically diverse bats from the superfamily Noctilionoidea. Collectively, noctilionoids display remarkable ecological breadth, from highly divergent echolocation to flight strategies linked to specialized insectivory, the parallel evolution of diverse plant-based diets (e.g., nectar, pollen, and fruit) from ancestral insectivory, and –unusually for echolocating bats– often have large, well-developed eyes. We report contrasting levels of positive selection in genes associated with the development, maintenance, and scope of visual function, tracing back to the origins of noctilionoids and Phyllostomidae (the bat family with most dietary diversity), instead of during shifts to novel diets. Generalized plant visiting was not associated with exceptional molecular adaptation, and exploration of these novel niches took place in an ancestral phyllostomid genetic background. In contrast, evidence for positive selection in vision genes was found at subsequent shifts to either nectarivory or frugivory. Thus, neotropical noctilionoids that use visual cues for identifying food and roosts, as well as for orientation, were effectively preadapted, with subsequent molecular adaptations in nectar-feeding lineages and the Stenodermatinae subfamily of fig-eating bats fine-tuning pre-existing visual adaptations for specialized purposes.

Vestigial characters are common across the tree of life, but the underlying evolutionary processes shaping phenotypic loss are poorly understood. The mammalian vomeronasal system, which detects social chemical cues important to fitness, is an impressive example of a sensory system lost multiple times. Three times more losses are inferred among bats than in other mammalian orders. We characterized the relationship between amino acid substitutions in a gene tightly linked to vomeronasal function (Trpc2) and the accessory olfactory bulb, a brain region that processes the detection of these vomeronasal chemical cues. By applying a phylogenetic logistic regression, we found a strong negative relationship between the branch lengths representing rates of codon changes in the Trpc2 gene tree and the presence or absence of an accessory olfactory bulb. Longer branch lengths predict loss of the accessory olfactory bulb, suggesting selection has relaxed on the system as a whole. Based on this relationship, we predicted the absence of an accessory olfactory bulb in 19 bat species with unknown morphology. Several species with predicted losses have specialized skull morphology, suggesting a potential tradeoff between adaptation in skull shape and maintenance of the vomeronasal system. This study offers a new approach to relate genetic mechanisms and phenotypes at a macroevolutionary scale.

Comparative methods are often used to infer loss or gain of complex phenotypes, but few studies take advantage of genes tightly linked with complex traits to test for shifts in the strength of selection. In mammals vomerolfaction detects chemical cues mediating many social and reproductive behaviors and is highly conserved, but all bats exhibit degraded vomeronasal structures with the exception of two families (Phyllostomidae and Miniopteridae). These families either regained vomerolfaction after ancestral loss, or there were many independent losses after diversification from an ancestor with functional vomerolfaction. In this study, we use the Transient receptor potential cation channel 2 (Trpc2) as a molecular marker for testing the evolutionary mechanisms of loss and gain of the mammalian vomeronasal system. We sequenced Trpc2 exon 2 in over 100 bat species across 17 of 20 chiropteran families. Most families showed independent pseudogenizing mutations in Trpc2, but the reading frame was highly conserved in phyllostomids and miniopterids. Phylogeny-based simulations suggest loss of function occurred after bat families diverged, and purifying selection in two families has persisted since bats shared a common ancestor. As most bats still display pheromone-mediated behavior, they might detect pheromones through the main olfactory system without using the Trpc2 signaling mechanism.

The earliest record of plant visiting in bats dates to the Middle Miocene of La Venta, the world's most diverse tropical palaeocommunity. Palynephyllum antimaster is known from molars that indicate nectarivory. Skull length, an important indicator of key traits such as body size, bite force and trophic specialization, remains unknown. We developed Bayesian models to infer skull length based on dental measurements. These models account for variation within and between species, variation between clades, and phylogenetic error structure. Models relating skull length to trophic level for nectarivorous bats were then used to infer the diet of the fossil. The skull length estimate for Palynephyllum places it among the larger lonchophylline bats. The inferred diet suggests Palynephyllum fed on nectar and insects, similar to its living relatives. Omnivory has persisted since the mid-Miocene. This is the first study to corroborate with fossil data that highly specialized nectarivory in bats requires an omnivorous transition.