Tissue-resident memory T cells (T_RM) reside in barrier tissues and provide local immediate protective immunity. Here, we show that the salivary gland (SG) most-effectively induces CD8⁺ and CD4⁺ T_RM cells against murine cytomegalovirus (MCMV), which persists in and spreads from this organ. T_RM generation depended on local antigen for CD4⁺, but not CD8⁺, T_RM cells, highlighting major differences in T cell subset-specific demands for T_RM development. CMV-specific CD8⁺ T cells fail to control virus replication upon primary infection in the SG due to CMV-induced MHC I downregulation in glandular epithelial cells. Using intraglandular infection, we challenge this notion and demonstrate that memory CD8⁺ T cells confer immediate protection against locally introduced MCMV despite active viral immune evasion, owing to early viral tropism to cells that largely withstand MHC I downregulation. Thus, we unravel a yet-unappreciated role for memory CD8⁺ T cells in protecting mucosal tissues against CMV infection.