Detailed knowledge about the dynamics of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is important for uncovering the viral and host factors that contribute to coronavirus disease 2019 (COVID-19) pathogenesis. Old-World nonhuman primates recapitulate mild to moderate cases of COVID-19, thereby serving as important pathogenesis models. We compared African green monkeys inoculated with infectious SARS-CoV-2 or irradiated, inactivated virus to study the dynamics of virus replication throughout the respiratory tract. Genomic RNA from the animals inoculated with the irradiated virus was found to be highly stable, whereas subgenomic RNA, an indicator of viral replication, was found to degrade quickly. We combined this information with single-cell RNA sequencing of cells isolated from the lung and lung-draining mediastinal lymph nodes and developed new analysis methods for unbiased targeting of important cells in the host response to SARS-CoV-2 infection. Through detection of reads to the viral genome, we were able to determine that replication of the virus in the lungs appeared to occur mainly in pneumocytes, whereas macrophages drove the inflammatory response. Monocyte-derived macrophages recruited to the lungs, rather than tissue-resident alveolar macrophages, were most likely to be responsible for phagocytosis of infected cells and cellular debris early in infection, with their roles switching during clearance of infection. Together, our dataset provides a detailed view of the dynamics of virus replication and host responses over the course of mild COVID-19 and serves as a valuable resource to identify therapeutic targets.