Here we show that in breast cancer cells the BAF, but not the closely related PBAF complex, is required for progesterone induction of several target genes including MMTV, where it catalyzes localized displacement of histones H2A and H2B and subsequent NF1 binding. Here, BANF1 is linked to breast cancer.