Most individuals infected with SARS-CoV-2 develop neutralizing antibodies that target the viral spike protein. Here we quantify how levels of these antibodies change in the months following SARS-CoV-2 infection by examining longitudinal samples collected between {approx} 30 and 152 days post-symptom onset from a prospective cohort of 34 recovered individuals with asymptomatic, mild, or moderate-severe disease. Neutralizing antibody titers declined an average of about four-fold from one to four months post-symptom onset. Importantly, our data are consistent with the expected early immune response to viral infection, where an initial peak in antibody levels is followed by a decline to a lower plateau. Additional studies of long-lived B-cells and antibody titers over longer time frames are necessary to determine the durability of immunity to SARS-CoV-2.