Chronic myeloid leukemia (CML) originates in a hematopoietic stem cell (HSC) transformed by the breakpoint cluster region (BCR)-abelson (ABL) oncogene and is effectively treated with tyrosine kinase inhibitors (TKIs). TKIs do not eliminate disease-propagating leukemic stem cells (LSCs), suggesting a deeper understanding of niche-dependent regulation of CML LSCs is required to eradicate disease. Cxcl12 is expressed in bone marrow niches and controls HSC maintenance, and here, we show that targeted deletion of Cxcl12 from mesenchymal stromal cells (MSCs) reduces normal HSC numbers but promotes LSC expansion by increasing self-renewing cell divisions, possibly through enhanced Ezh2 activity. In contrast, endothelial cell-specific Cxcl12 deletion decreases LSC proliferation, suggesting niche-specific effects. During CML development, abnormal clusters of colocalized MSCs and LSCs form but disappear upon Cxcl12 deletion. Moreover, MSC-specific deletion of Cxcl12 increases LSC elimination by TKI treatment. These findings highlight a critical role of niche-specific effects of Cxcl12 expression in maintaining quiescence of TKI-resistant LSC populations.