Divergent roles for androgen receptor (AR) in breast cancer have been reported. Following aromatase inhibitor (AI) treatment, the conversion of circulating androgens into estrogens can be diminished by >99%. We wished to establish whether the steroid environment can dictate the role of AR and the implications of this for subsequent therapy. This study utilizes models of AI resistance to explore responsiveness to PI3K/mTOR and anti-AR therapy when cells are exposed to unconverted weak androgens. Transcriptomic alterations driven by androstenedione (4AD) were assessed by RNA-sequencing. AR and estrogen receptor (ER) recruitment to target gene promoters was evaluated using ChIP, and relevance to patient profiles was performed using publicly available data sets. Although BEZ235 showed decreased viability across AI-sensitive and -resistant cell lines, anti-AR treatment elicited a decrease in cell viability only in the AI-resistant model. Serum and glucocorticoid-regulated kinase 3 (SGK3) and cAMP-dependent protein kinase inhibitor β (PKIB) were confirmed to be regulated by 4AD and shown to be mediated by AR; crucially, reexposure to estradiol suppressed expression of these genes. Meta-analysis of transcript levels showed high expression of SGK3 and PKIB to be associated with poor response to endocrine therapy (HR = 2.551, P = 0.003). Furthermore, this study found levels of SGK3 to be sustained in patients who do not respond to AI therapy. This study highlights the importance of the tumor steroid environment. SGK3 and PKIB are associated with poor response to endocrine therapy and could have utility in tailoring therapeutic approaches.