Comparing lobular and non‐lobular ER+ HER2− PDXs, we identify a transcriptional program underlying the lobular‐specific molecular characteristics and show that extracellular matrix (ECM) modulation is a tumor cell‐intrinsic characteristic and targetable feature of ILCs. Here, ESR1 is linked to neoplasm.