Motivations to avoid infectious disease seem to influence prejudice toward some groups, including groups not explicitly associated with infectious disease. The standard explanation for this phenomenon is based on signal detection theory and proposes that some prejudices partially arise from pathogen detection mechanisms that are biased toward making false alarms (false positives) in order to minimize misses (false negatives). Therefore, pathogen detection mechanisms arguably categorize a broad array of atypical features as indicative of infection, which gives rise to negative affect toward people with atypical features. We tested a key hypothesis derived from this explanation: specific appearance-based prejudices are associated with tendencies to make false alarms when estimating the presence of infectious disease. While this hypothesis is implicit in much work on the behavioral immune system and prejudice, direct tests of it are lacking and existing relevant work contains important limitations. To test the hypothesis, we conducted a cross-sectional study using a large U.S. sample (N = 1450). Using signal detection theory methods, we assessed tendencies to make false alarms when identifying infection threats. We further assessed prejudice toward multiple relevant social groups/categories. Results showed weak evidence for the key hypothesis: for only one of four tested target groups were tendencies to make false alarms in sickness detection significantly associated with prejudice. However, this relation was not significant when controlling for a potential confound. These results cast doubt on the notion that individual differences in appearance-based prejudices arise from individual differences in tendencies to make false alarms in assessing pathogen threats.