Hematopoiesis is dynamically regulated by metabolic cues in homeostatic and stressed conditions; however, the cellular and molecular mechanisms mediating the metabolic sensing and regulation remain largely obscure. Bone marrow adipose tissue remodels in various metabolic conditions and has been recently proposed as a niche for hematopoietic stem cells after irradiation. Here, we investigated the role of marrow adipose tissue-derived hematopoietic cytokine stem cell factor in unperturbed hematopoiesis by selectively ablating the Kitl gene from adipocytes and bone marrow stroma cells using Adipoq-Cre and Osx1-Cre, respectively. We found that both Adipoq-Kitl knockout (KO) and Osx1-Kitl KO mice diminished hematopoietic stem and progenitor cells and myeloid progenitors in the bone marrow and developed macrocytic anemia at the steady-state. The composition and differentiation of hematopoietic progenitor cells in the bone marrow dynamically responded to metabolic challenges including high fat diet, β3-adrenergic activation, thermoneutrality, and aging. However, such responses, particularly within the myeloid compartment, were largely impaired in Adipoq-Kitl KO mice. Our data demonstrate that marrow adipose tissue provides stem cell factor essentially for hematopoiesis both at the steady state and upon metabolic stresses.