Coronaviruses are evolutionarily successful RNA viruses, common to multiple avian, amphibian and mammalian hosts. Despite their ubiquity and potential impact, knowledge of host immunity to coronaviruses remains incomplete, partly owing to the lack of overt pathogenicity of endemic human coronaviruses (HCoVs), which typically cause common colds. However, the need for deeper understanding became pressing with the zoonotic introduction of three novel coronaviruses in the past two decades, causing severe acute respiratory syndromes in humans, and the unfolding pandemic of coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This renewed interest not only triggered the discovery of two of the four HCoVs, but also uncovered substantial cellular and humoral cross-reactivity with shared or related coronaviral antigens. Here, we review the evidence for cross-reactive B cell memory elicited by HCoVs and its potential impact on the puzzlingly variable outcome of SARS-CoV-2 infection. The available data indicate targeting of highly conserved regions primarily in the S2 subunits of the spike glycoproteins of HCoVs and SARS-CoV-2 by cross-reactive B cells and antibodies. Rare monoclonal antibodies reactive with conserved S2 epitopes and with potent virus neutralising activity have been cloned, underscoring the potential functional relevance of cross-reactivity. We discuss B cell and antibody cross-reactivity in the broader context of heterologous humoral immunity to coronaviruses, as well as the limits of protective immune memory against homologous re-infection. Given the bidirectional nature of cross-reactivity, the unprecedented current vaccination campaign against SARS-CoV-2 is expected to impact HCoVs, as well as future zoonotic coronaviruses attempting to cross the species barrier. However, emerging SARS-CoV-2 variants with resistance to neutralisation by vaccine-induced antibodies highlight a need for targeting more constrained, less mutable parts of the spike. The delineation of such cross-reactive areas, which humoral immunity can be trained to attack, may offer the key to permanently shifting the balance of our interaction with current and future coronaviruses in our favour.