Older adults (≥65 years of age) bear a significant burden of severe disease and mortality associated with influenza, despite relatively high annual vaccination coverage and substantial pre-existing immunity to influenza. To test the hypothesis that host factors, including age and sex, play a role in determining the effect of repeated vaccination and levels of pre-existing humoral immunity to influenza, we evaluated pre- and post-vaccination strain-specific hemagglutination inhibition (HAI) titers in adults over 75 years of age who received a high-dose influenza vaccine in at least four out of six influenza seasons. Pre-vaccination titers, rather than host factors and repeated vaccination were significantly associated with post-vaccination HAI titer outcomes, and displayed an age-by-sex interaction. Pre-vaccination titers to H1N1 remained constant with age. Titers to H3N2 and influenza B viruses decreased substantially with age in males, whereas titers in females remained constant with age. Our findings highlight the importance of pre-existing immunity in this highly vaccinated older adult population and suggest that older males are particularly vulnerable to reduced pre-existing humoral immunity to influenza.