In this cross‐cohort study using three population‐based cohorts, we examined associations of multiple common pathogens with neuroimaging markers related to subclinical dementia—brain volume, hippocampal volume, and white matter lesions—and tested whether these associations were modified by APOE ε4 and ε2 carrier status. This evidence concerns the gene APOE and dementia.