Using high-speed, live-cell imaging of polarized adult primary RPE monolayers, Abca4–/– mice, and human AMD donor tissues, we show that efficient cholesterol transport in RPE expressing the nonrisk ApoE3 or the protective ApoE4 isoforms preserves critical homeostatic functions such as autophagy despite cellular stress. Here, ABCA4 is linked to age-related macular degeneration.