Creating and evaluating predictions are considered important features in sensory perception. Little is known about processing differences between the senses and their cortical substrates. Here, we tested the hypothesis that olfaction, the sense of smell, would be highly dependent on (nonolfactory) object-predictive cues and involve distinct cortical processing features. We developed a novel paradigm to compare prediction error processing across senses. Participants listened to spoken word cues (e.g., “lilac”) and determined whether target stimuli (odors or pictures) matched the word cue or not. In two behavioral experiments (total n = 113; 72 female), the disparity between congruent and incongruent response times was exaggerated for olfactory relative to visual targets, indicating a greater dependency on predictive verbal cues to process olfactory targets. A preregistered fMRI study (n = 30; 19 female) revealed the anterior cingulate cortex (a region central for error detection) being more activated by incongruent olfactory targets, indicating a role for olfactory predictive error processing. Additionally, both the primary olfactory and visual cortices were significantly activated for incongruent olfactory targets, suggesting olfactory prediction errors are dependent on cross-sensory processing resources, whereas visual prediction errors are not. We propose that olfaction is characterized by a strong dependency on predictive (nonolfactory) cues and that odors are evaluated in the context of such predictions by a designated transmodal cortical network. Our results indicate differences in how predictive cues are used by different senses in rapid decision-making.