Synaptic refinement via the elimination of inappropriate synapses and strengthening of appropriate ones is crucially important for the establishment of specific, topographic neural circuits. The mechanisms driving these processes are poorly understood, particularly concerning inhibitory projections. Here, we address the refinement of an inhibitory topographic projection in the auditory brainstem in functional and anatomical mapping studies involving patch-clamp recordings in combination with minimal and maximal stimulation, caged glutamate photolysis, and single axon tracing. We demonstrate a crucial dependency of the refinement on Ca_V1.3 calcium channels: Ca_V1.3^−/− mice displayed virtually no elimination of projections up to hearing onset. Furthermore, strengthening was strongly impaired, in line with a reduced number of axonal boutons. The mediolateral topography was less precise and the shift from a mixed GABA/glycinergic to a purely glycinergic transmission before hearing onset did not occur. Together, our findings provide evidence for a Ca_V1.3-dependent mechanism through which both inhibitory circuit formation and determination of the neurotransmitter phenotype are achieved.