The role of pollinators in floral divergence has long attracted the attention of evolutionary biologists. Although abundant studies have reported the effect of pollinators on flower shape variation and plant speciation, the influence of pollinators on plant species differentiation during rapid radiations and the specific consequences of shifts among similar pollinators are not well understood. Here, we evaluate the association between pollinators and floral morphology in a closely related and recently diversifying clade of Linaria species (sect. Supinae subsect. Supinae). Our approach combined pollinator observations, functional floral morphometric measures and phylogenetic comparative analyses. The fauna visiting Linaria species was determined by extensive surveys and categorized by a modularity algorithm, while the size and shape of flowers were analyzed by means of standard and geometric morphometric measures. Standard measures failed to find relationships between the sizes of representative pollinators and flowers. However, discriminant-function analyses of geometric morphometric data revealed that pollination niches are finer predictors of flower morphologies in Linaria if compared with phylogenetic relationships. Species with the most restrictive flowers displayed the most slender spurs and were pollinated by bees with larger proboscides. These restrictive flower shapes likely appeared more than once during the evolutionary history of the study group. We show that floral variation can be driven by shifts between pollinators that have been traditionally included in a single functional group, and discuss the consequences of such transitions for plant species differentiation during rapid radiations.