These findings support our hypothesis that Aβ pathologies and vascular diseases play distinct roles in hippocampal atrophy and cognitive decline in non-demented elderly adults with and without APOE-ε4 allele. Here, APOE is linked to hippocampal atrophy.