In this study, we propose and evaluate a technique known as common average referencing (CAR) to generate a more ideal reference electrode for microelectrode recordings. CAR is a computationally simple technique, and therefore amenable to both on-chip and real-time applications. CAR is commonly used in EEG, where it is necessary to identify small signal sources in very noisy recordings. To study the efficacy of common average referencing, we compared CAR to both referencing with a stainless steel bone-screw and a single microelectrode site. Data consisted of in vivo chronic recordings in anesthetized Sprague-Dawley rats drawn from prior studies, as well as previously unpublished data. By combining the data from multiple studies, we generated and analyzed one of the more comprehensive chronic neural recording datasets to date. Reference types were compared in terms of noise level, signal-to-noise ratio, and number of neurons recorded across days. Common average referencing was found to drastically outperform standard types of electrical referencing, reducing noise by >30%. As a result of the reduced noise floor, arrays referenced to a CAR yielded almost 60% more discernible neural units than traditional methods of electrical referencing. CAR should impart similar benefits to other microelectrode recording technologies—for example, chemical sensing—where similar differential recording concepts apply. In addition, we provide a mathematical justification for CAR using Gauss-Markov theorem and therefore help place the application of CAR into a theoretical context.