Expression of ER and AR in breast cancer tissues was associated with a decreased infiltration of immune cells into the tumor microenvironment. Here, ESR1 is linked to breast cancer.