This genetic association study analyzes data from 4 Alzheimer disease and cognitive aging cohorts to investigate whether sex and race modify associations of APOE ε4 and ε2 with cognition. Here, APOE is linked to Alzheimer disease.