Our results suggest that a visuomotor task may have potential for detecting the early effects of the APOE e4 allele on white matter microstructure and medial temporal lobe grey matter early in dementia progression, as the white matter and grey matter declines observed here were associated with visuomotor difficulties only in e4 carriers. Here, APOE is linked to dementia.