Tumors are sustained by complex networks of interactions between malignant cells, stromal cells and tumor-infiltrating immune cells. These networks differ from patient to patient in terms of nature, composition and organization as well as with regard to the precise localization of tumor-infiltrating cells. Of note, the heterogeneity of the immunological component of the tumor microenvironment, as opposed to its mere abundance, has been shown to influence disease outcome. However, a key question remains: where does the activation of tumor-specific T cells take place? The recently described, tumor-associated lymph node-like entities termed “tertiary lymphoid structures” exhibit a structural organization that is reminiscent of secondary lymphoid organs, and thus may imprint the local immune contexture. Here, we discuss how cancer-associated tertiary lymphoid structures impact on the tumor micro-architecture, immune microenvironment, and ultimately, patient survival.