Hypoxia is a prominent feature in the maintenance of hematopoietic stem cell (HSC) quiescence and multipotency. Hypoxia-inducible factor (HIF) prolyl hydroxylase domain proteins (PHDs) serve as oxygen sensors and may therefore regulate this system. Here, we describe a mouse line with conditional loss of HIF prolyl hydroxylase 2 (PHD2) in very early hematopoietic precursors that results in self-renewal of multipotent progenitors under steady-state conditions in a HIF1α- and SMAD7-dependent manner. Competitive bone marrow (BM) transplantations show decreased peripheral and central chimerism of PHD2-deficient cells but not of the most primitive progenitors. Conversely, in whole BM transfer, PHD2-deficient HSCs replenish the entire hematopoietic system and display an enhanced self-renewal capacity reliant on HIF1α. Taken together, our results demonstrate that loss of PHD2 controls the maintenance of the HSC compartment under physiological conditions and causes the outcompetition of PHD2-deficient hematopoietic cells by their wild-type counterparts during stress while promoting the self-renewal of very early hematopoietic progenitors.