The primary unseptated heart tube undergoes extensive remodeling including septation at the atrial, atrioventricular, ventricular, and ventriculo-arterial level. Alignment and fusion of the septal components is required to ensure full septation of the heart. Deficiencies lead to septal defects at various levels. Addition of myocardium and mesenchymal tissues from the second heart field (SHF) to the primary heart tube, as well as a population of neural crest cells, provides the necessary cellular players. Surprisingly, the study of the molecular background of these defects does not show a great diversity of responsible transcription factors and downstream gene pathways. Epigenetic modulation and mutations high up in several transcription factor pathways (e.g. NODAL and GATA4) may lead to defects at all levels. Disturbance of modulating pathways, involving primarily the SHF-derived cell populations and the genes expressed therein, results at the arterial pole (e.g. TBX1) in a spectrum of ventricular septal defects located at the level of the outflow tract. At the venous pole (e.g. TBX5), it can explain a variety of atrial septal defects. The various defects can occur as isolated anomalies or within families. In this review developmental, morphological, genetic, as well as epigenetic aspects of septal defects are discussed.