Mammalian chromosomes are partitioned into A/B compartments and topologically associated domains (TADs). The inactive X (Xi) chromosome, however, adopts a distinct conformation without evident compartments or TADs. Here, through exploration of an architectural protein, structural-maintenance-of-chromosomes hinge domain containing 1 (SMCHD1), we probe how the Xi is reconfigured during X chromosome inactivation. A/B compartments are first fused into “S1” and “S2” compartments, coinciding with Xist spreading into gene-rich domains. SMCHD1 then binds S1/S2 compartments and merges them to create a compartment-less architecture. Contrary to current views, TADs remain on the Xi but in an attenuated state. Ablating SMCHD1 results in a persistent S1/S2 organization and strengthening of TADs. Furthermore, loss of SMCHD1 causes regional defects in Xist spreading and erosion of heterochromatic silencing. We present a stepwise model for Xi folding, where SMCHD1 attenuates a hidden layer of Xi architecture to facilitate Xist spreading.