We found no significant associations between non-modifiable risk factors (family history of dementia, APOE4) and midlife SVD burden. This evidence concerns the gene APOE and dementia.