Human endogenous retroviruses (HERVs) are remnants of retroviral germ line infections of human ancestors and make up ~8% of the human genome. Under physiological conditions, these elements are frequently inactive or non-functional due to deactivating mutations and epigenetic control. However, they can be reactivated under certain pathological conditions and produce viral transcripts and proteins. Several disorders, like multiple sclerosis or amyotrophic lateral sclerosis are associated with increased HERV expression. Although their detailed contribution to individual diseases has yet to be elucidated, an increasing number of studies in vitro and in vivo suggest HERVs as potent modulators of the immune system. They are able to affect the transcription of other immune-related genes, interact with pattern recognition receptors, and influence the positive and negative selection of developing thymocytes. Interestingly, HERV envelope proteins can both stimulate and suppress immune responses based on different mechanisms. In the light of HERV proteins becoming an emerging drug target for autoimmune-related disorders and cancer, we will provide an overview on recent findings of the complex interactions between HERVs and the human immune system with a focus on autoimmunity.