This review focuses on the roles of ER and AR signaling in both breast cancer cells and stromal cells, summarizing recent findings regarding the effects of estrogens and androgens on stromal cells within the breast cancer microenvironment. This evidence concerns the gene ESR1 and breast carcinoma.