Although increased awareness leading to early detection and prevention, as well as advancements in treatment strategies, have resulted in superior clinical outcomes, African American women with breast cancer continue to have greater mortality rates, compared to Caucasian American counterparts. Moreover, African American women are more likely to have breast cancer at a younger age and be diagnosed with aggressive tumor sub-types. Such racial disparities can be attributed to socioeconomic differences, but it is increasingly being recognized that these disparities may indeed be due to certain genetic and other non-genetic biological differences. Tumor microenvironment, which provides a favorable niche for the growth of tumor cells, is comprised of several types of stromal cells and the various proteins secreted as a consequence of bi-directional tumor-stromal cross-talk. Emerging evidence suggests inherent biological differences in the tumor microenvironment of breast cancer patients from different racial backgrounds. Tumor microenvironment components, affected by the genetic make-up of the tumor cells as well as other non-tumor-associated factors, may also render patients more susceptible to the development of aggressive tumors and faster progression of disease resulting in early onset, thus adversely affecting patients’ survival. This review provides an overview of breast cancer racial disparity and discusses the existence of race-associated differential tumor microenvironment and its underlying genetic and non-genetic causal factors. A better understanding of these aspects would help further research on effective cancer management and improved approaches for reducing the racial disparities gaps in breast cancer patients.