Effective immunotherapy, whether by checkpoint blockade or adoptive cell therapy, is limited in most patients by a key barrier: the immunosuppressive tumor microenvironment. Suppression of tumor-specific T cells is orchestrated by the activity of a variety of stromal myeloid and lymphoid cells. These often display inducible suppressive mechanisms that are triggered by the same anti-tumor inflammatory response that the immunotherapy intends to create. Therefore, a more comprehensive understanding of how the immunosuppressive milieu develops and persists is critical in order to harness the full power of immunotherapy of cancer.