This phase I/II trial studies the side effects and best dose of HCQ when given together with palbociclib and letrozole before surgery in treating participants with estrogen receptor positive, HER2 negative breast cancer. This evidence concerns the gene ESR1 and breast carcinoma.