In this pilot study, we introduce a machine learning framework to identify relationships between cancer tissue morphology and hormone receptor pathway activation in breast cancer pathology hematoxylin and eosin (H&E)-stained samples. Here, NR4A1 is linked to cancer.