Nivolumab and ipilimumab are associated with distinct immune landscape changes and response-associated immunophenotypes

, consistent with poor outcomes in the IPI > NIVO cohort of Checkmate-064. In further support of this interpretation, in clinical trials, the response rates to NIVO in patients progressing after IPI are lower than those in IPI-naive patients (28–32). However, these response rates are compared across different studies, and lower response rates to NIVO in IPI-refractory patients may result from selection of immunotherapy-resis-tant patients. Response rates in the NIVO > IPI cohort were similar to concurrent NIVO and IPI, suggesting that concomitant IPI does not negatively impact NIVO efficacy. Regardless, the data presented herein raise concerns about the negative impact of IPI treatment before NIVO and warrant consideration in patient treatment decisions. Future studies will investigate immunophenotypic changes occurring with concurrent treatment and associated patient outcomes. Furthermore, studies will need to address whether the immune landscape changes resulting from IPI are normalized over time and the duration that takes to occur. Future investigations will also need to investigate the function of and potential value as biomarkers of immune cell populations that are associated with treatment outcomes in this study


Introduction
Immunotherapies have had remarkable success in treating a variety of malignancies, including metastatic melanoma.However, many patients do not benefit from these treatments, and mechanisms differentiating patient outcomes remain elusive.The immune checkpoint antibodies ipilimumab (IPI) and nivolumab (NIVO) block ligation of the CTLA4 and PD1 inhibitory receptors, respectively, expressed on T cells.Consequently, proper T cell receptor (TCR) signaling and acquisition of effector function is restored to tumor-reactive T cells (1).These agents are given systemically, so they also exert effects on other, non-tumor-reactive T cell populations, likely altering the immune landscape.However, the impact of these agents on the immune landscape and its potential relation to patient outcomes remains underinvestigated.
Combination therapy with IPI and NIVO results in a 55% response rate and 52% overall survival at 5 years, superior to either single agent, albeit at the cost of increased immune-related toxicity (2,3).To reduce treatment-related toxicity, the sequential administration of NIVO and IPI for the treatment of metastatic melanoma was evaluated in the Checkmate-064 trial (4).In this trial, significant differences in patient outcomes were observed; patients receiving IPI for 12 weeks, followed by NIVO for 12 weeks (IPI > NIVO) with subsequent NIVO maintenance had significantly lower response rates and overall survival compared with the NIVO > IPI sequence.
BACKGROUND.The reshaping of the immune landscape by nivolumab (NIVO) and ipilimumab (IPI) and its relation to patient outcomes is not well described.

METHODS.
We used high-parameter flow cytometry and a computational platform, CytoBrute, to define immunophenotypes of up to 15 markers to assess peripheral blood samples from metastatic melanoma patients receiving sequential NIVO > IPI or IPI > NIVO (Checkmate-064).
RESULTS.The 2 treatments were associated with distinct immunophenotypic changes and had differing profiles associated with response.Only 2 immunophenotypes were shared but had opposing relationships to response/survival.To understand the impact of sequential treatment on response/survival, phenotypes that changed after the initial treatment and differentiated response in the other cohort were identified.Immunophenotypic changes occurring after NIVO were predominately associated with response to IPI > NIVO, but changes occurring after IPI were predominately associated with progression after NIVO > IPI.Among these changes, CD4 + CD38 + CD39 + CD127 -GARP -T cell subsets were increased after IPI treatment and were negatively associated with response/survival for the NIVO > IPI cohort.

CONCLUSION.
Collectively, these data suggest that the impact of IPI and NIVO on the immunophenotypic landscape of patients is distinct and that the impact of IPI may be associated with resistance to subsequent NIVO therapy, consistent with poor outcomes in the IPI > NIVO cohort of Checkmate-064.insight.jci.orghttps://doi.org/10.1172/jci.insight.137066

C L I N I C A L M E D I C I N E
Defining biomarkers of patient outcome for immunotherapy has been the focus of much recent effort.Several biomarkers differentiating patient response to checkpoint inhibition have been described, including tumor and infiltrating cell PDL1 expression, tumor mutation burden, expression of a tumor IFN-γinduced gene expression signature, increased levels of tumor-infiltrating CD8 + T lymphocytes, and others (5).Interestingly, there are few biomarkers associated with response to both αPD1 and αCTLA4.For some biomarkers, such as TCR diversity, T cell memory subsets, and frequency of circulating Tregs, there is a reciprocal association with response for the 2 therapies (6)(7)(8)(9).While useful for identifying rational targets for combination therapy, these biomarkers have failed to stratify patient responses well enough to have clinical utility for prediction of outcome.To accomplish this, multiple biomarkers that capture the complexity of individual immune responses will likely be required.Emerging high-dimension, single-cell technologies offer the ability to probe the immune response at an unprecedented depth and define potentially novel biomarkers of outcome.
High-parameter, single-cell technologies are powerful tools to address whether variations in the immune landscape may be associated with clinical benefit.Their utility in biomarker discovery has been highlighted by recent publications detailing signatures and complex immunophenotypes associated with metastatic melanoma patient outcomes.For example, using single-cell RNA sequencing (RNA-Seq), tumor transcriptional signatures associated with immunological exclusion and CD8 + T cell transcriptional states associated with checkpoint immunotherapy resistance have been shown (10).Mass cytometry, another high-dimension, single-cell technology, has also shown promise in discovering immune cell phenotypes associated with immunotherapy response/resistance.Using this technology, the frequency of monocyte populations has been shown to be associated with patient response to immunotherapy and overall survival (11).
For the work presented here, we used a potentially new, rapid computing platform for enumerating complex cell populations in single-cell data sets, described by combinatorial expression of up to 15 markers.Using this approach, we describe the effects of NIVO and IPI on the peripheral blood immune landscape of metastatic melanoma patients and determine the association of those effects with response to therapy and overall survival.We used high-parameter (22-to 27-color) flow cytometry to assess peripheral blood specimens from patients treated with sequential NIVO > IPI or IPI > NIVO.By using paired baseline and week-13 specimens, we were able to directly compare the immunophenotypic changes accompanying treatment with single-agent NIVO or IPI.We used a potentially novel computational platform, CytoBrute, to assess frequencies of cell types defined by combinatorics for CD4 + and CD8 + T cells and myeloid cells.This methodology allowed us to identify clusters of immunophenotypically related cells that were associated with each treatment and with patient outcomes.

High-dimension flow cytometry and combinatorics characterize immunophenotypes in metastatic melanoma patients.
In the Checkmate-064 trial, immunotherapy-naive metastatic melanoma patients received treatment with sequential NIVO > IPI or the reverse sequence IPI > NIVO (Figure 1A).To determine how NIVO and IPI shaped the immune landscape, we performed high-dimension flow cytometry on baseline and week-13 PBMC samples (i.e., after the first drug in each sequence).The markers assessed in 4 separate staining panels are shown in Supplemental Table 1 (supplemental material available online with this article; https:// doi.org/10.1172/jci.insight.137066DS1).
We first performed t-Distributed Stochastic Neighbor Embedding (tSNE) analyses to compare treatment regimens and time points (Supplemental Figure 1).While differences were noted between responders and nonresponders and between week 0 and week 13, the significance of the changes reported by the tSNE analysis was difficult to ascertain -particularly in identifying treatment-induced changes, given the nonpaired nature.It was also difficult to exhaustively identify all the changes depicted in a tSNE graphic using only visual approaches.We also noted that, for the highly heterogeneous and diverse cell populations studied in our flow cytometry panels, tSNE fails to produce clearly separated "islands" of cells, further complicating the identification of cell types that differ by patient group or time point.For the tSNE shown, the perplexity was set at 30 and the number of iterations was 1000.Several variations of the tSNE algorithm with variations in perplexity (of 30, 50, 70) and iterations (of 1000, 1750, 2500) were performed with similar results (data not shown).
To evaluate immunophenotypes in a more comprehensive manner, we used CytoBrute, a computational approach designed for analysis of high-parameter flow data sets.Briefly, each flow cytometry data file was gated on immune cell lineages (i.e., viable CD3 + CD4 + , CD3 + CD8 + , and leukocytes) of interest.Data were then analyzed using CytoBrute, which creates Boolean combinations for all possible phenotypes up to 15 markers and calculates the frequency of cells expressing each combination of markers.Figure 1B illustrates the basis of this approach, and Figure 1, C-D, shows how many phenotypes can be quantified.Although this approach can theoretically generate up to 14 million phenotypes (3 15 subsets, considering positive, negative, and neutral expression), many of these cell types are exceedingly rare or nonexistent, so we selected the top 1000 most frequent phenotypes per sample, for each antibody panel.As each sample had different immunophenotypes comprising the top 1000, nonoverlapping frequencies were also calculated, resulting in about 100,000 immunophenotypes evaluated.The frequencies of immunophenotypes observed in each panel are depicted as mosaic plots in Supplemental Figure 2.Each tile is sized to indicate the proportion of cells expressing a particular phenotype, and tiles are sorted from most frequent phenotypes (upper left corner) to least frequent (lower right corner).The tiles are color-coded by the number of markers defining the phenotype.The mosaic plot demonstrates the relevance of phenotypes defined by a high number of markers (i.e., phenotypes defined by 8-15 markers, green and red boxes, most easily identified for Supplemental Table 1), which are interspersed among the most frequently observed phenotypes.Notably, there is a small proportion of phenotypes that are rarely observed or absent (gray boxes, lower right corner of Supplemental Figure 2), demonstrating that the majority of potential immunophenotypes, both frequent and infrequent, are represented within the data.

C L I N I C A L M E D I C I N E
Changes in the peripheral blood immune landscape follow NIVO or IPI treatment.We assessed the effects of NIVO and IPI on peripheral blood immune cells derived from metastatic melanoma patients by assessing baseline and week-13 paired samples using Wilcoxon signed-rank tests.With NIVO treatment, 1744 immunophenotypes increased from baseline (pretreatment), while 2284 were decreased at P < 0.05 (Figure 2A).To dissect how the altered immunophenotypes were related, we projected significantly changed immunophenotypes into a 2-dimensional Uniform Manifold Approximation and Projection (UMAP) and then clustered the data using k-means clustering.As shown in Supplemental Figure 3A, 8 clusters were identified among the immunophenotypes increased after NIVO.The frequency for the top 15 markers represented in each of these clusters is shown in Supplemental Figure 3B.Orange bars represent markers that are expressed (+); gray bars represent markers that are not expressed (-).The length of the bar denotes the percentage of immunophenotypes in that cluster expressing the corresponding marker.For example, the most common marker in the immunophenotypes composing cluster 6 is CD38 + , expressed by nearly all of the immunophenotypes in that cluster.The second most common marker comprising this cluster is GITR -.Both CD4 + and CD8 + T cells are represented in this cluster.Cell types that increased after NIVO include T cells expressing the ectoenzymes CD38 and CD39 (clusters 1, 6, and 7) and CD73 (cluster 1), T cells with a naive-like phenotype (cluster 2), and cell types that are not well defined by the markers measured (clusters 3-5 and 8).Eight clusters were also identified for the immunophenotypes decreasing after NIVO, as shown in Supplemental Figure 3C.The phenotypic composition of these clusters is shown in Supplemental Figure 3D.Cell types that decreased with NIVO were predominately characterized by expression of markers associated with central memory T cells (e.g., CD45RO + , CCR7 + , CD127 + , CD95 + ).
In IPI-treated patients, 4498 immunophenotypes were increased and 2679 were reduced, relative to baseline, with a P < 0.05 (Figure 2B).For immunophenotypes increasing after IPI, 8 clusters were identified (Supplemental Figure 4A); these were defined by the markers listed in Supplemental Figure 4B.In contrast to NIVO-associated changes, many of these phenotypes were characterized by CD45RO + and CD95 + .Nine clusters were identified from the immunophenotypes decreasing after IPI (Supplemental Figure 4C), for which the phenotypic breakdown is shown in Supplemental Figure 4D.The majority of the cells were CD4 + and expressed CD127.Several clusters also contained GARP + immunophenotypes.
NIVO and IPI have distinct impacts on the peripheral immunophenotypic landscape.To better compare the immunophenotypic effects of the 2 drugs, we examined which changes were common to both drugs versus unique to only 1 drug.For the 2 treatments, 584 immunophenotypes that changed overlapped (~5%), as shown in Figure 2C. Figure 2D shows that, of the 584 immunophenotypic changes common across drugs, 281 increased and 244 decreased.However, 56 immunophenotypes had reciprocal changes; these cell populations were expanded after IPI but contracted after NIVO.Three additional phenotypes were increased after NIVO but decreased after IPI.Supplemental Figure 5A shows the immunophenotypes that increased with both treatments fell into 7 clusters.A CD4 + CD38 + T cell phenotype predominated in clusters 2, 3, 4, and 7 (Supplemental Figure 5B).The 244 immunophenotypes that decreased with IPI or NIVO grouped into 7 clusters (Supplemental Figure 5C), which included almost exclusively CD4 + immunophenotypes, with the exception of cluster 7 (Supplemental Figure 5D).Six clusters represented the immunophenotypes decreasing after NIVO but increasing after IPI (Supplemental Figure 5E).Clusters 1 and 4 were composed of CD4 + OX40 + Ki67 + T cells (Supplemental Figure 5F).
We next evaluated whether changes in circulating immunophenotypes could distinguish treatment regimens.To do so, the difference in frequency between week 13 and week 0 (delta value) was calculated for each immunophenotype.These delta values were then fed into an elastic net (EN) regularized regression model with repeated cross validation.Using paired changes of the 584 identified overlapping phenotypes from all patient samples, the model was able to predict whether paired patient samples were from those who received NIVO or IPI with an AUC of 0.867.The corresponding receiver operating characteristic (ROC) is shown by the black dotted line in the left panel of Figure 2E.The model values for individual paired samples are shown in the right panel of Figure 2E.Patient outcomes were added as a color dimension with responding patients (partial or complete response according to RECIST 1.1 criteria) in blue and progressing patients in red.Using the EN model determined by all patient samples, we then determined the ROC for responding and progressing patients separately.As shown in the left panel of Figure 2E, an AUC of 0.982 was achieved in responding patients (blue line) and an AUC of 0.714 in progressing patients, suggesting that patients who respond to therapy have more distinct immune changes than nonresponders.
Peripheral blood immunophenotypes at baseline and after treatment are associated with patient outcomes after NIVO > IPI sequential therapy.We next sought to determine if baseline and/or week 13 (after first agent) peripheral blood immunophenotypes were associated with patient outcomes.In baseline samples from NIVO > IPI-treated patients (left panel, Figure 3A), 260 signatures were associated with both response to therapy (P < 0.05, Mann-Whitney U test comparing responders and progressors) and overall survival (divided above and below median frequency, P < 0.05, Mantel-Cox test).Each dot represents a significant immunophenotype and is colored by the associated P value from the comparison of frequency differences between responders and progressors.The x coordinate is the median frequency of the immunophenotype in progressors and the y coordinate is the corresponding median frequency in responders.The 61 immunophenotypes significantly elevated at baseline in responding patients formed 5 clusters, as shown in Supplemental Figure 6A.Cluster 1 consisted of CD8 + CD95 + PD1 -CD25 -T cells, clusters 2 and 4 consisted of CD4 + CD45RA + CD127 + HELIOS -CD73 -CD49B -CD38 -T cells, and cluster 5 contained CD4 + or CD8 + T cells expressing LAG3 (Supplemental Figure 6B).The 199 immunophenotypes that were relatively lower in responding patients (and therefore higher in progressing patients) formed 8 clusters, as shown in Supplemental Figure 6C.Clusters 1, 3, and 7 were composed of CD4 + CD38 + CD39 + CD127 -GARP -T cells (Supplemental Figure 6D).Clusters 2, 4, 5, 6, and 8 were composed of CD95 + -expressing CD4 + T cells.
After NIVO (week 13), 662 immunophenotypes were found to be associated with patient outcomes in those receiving sequential NIVO > IPI treatment (right panel, Figure 3A).In contrast to baseline response- associated immunophenotypes, which were predominately CD4 + , nearly all the post-NIVO response-associated immunophenotypes were CD8 + -in particular, those found at higher frequencies in responders.Among these immunophenotypes, 561 were increased in frequency in responding relative to progressing patients.As shown in Supplemental Figure 6E, 10 clusters were formed from these immunophenotypes (Supplemental Figure 6F).Combinations of CCR7 + , CD127 + , CD45RO + , or CD95 + CD8 + T cells comprised clusters 2, 3, 4, 5, 6, 8, and 10.A total of 101 immunophenotypes forming 7 clusters were found to be decreased in responding relative to progressing patients (Figure 3A, right panel, and Supplemental Figure 6G).As shown in Supplemental Figure 6H, clusters 1, 2, and 6 were composed of CD8 + CD45RA + T cells.Collectively, a number of immunophenotypes that were associated with patient response and survival and were grouped into several immunophenotypically related clusters were identified.These included elevated levels of naive-like (CD45RA + , CD127 + ) T cells and CD8 + LAG3 + phenotypes at baseline and elevated levels of central memory-like (CD45RO + , CCR7 + , CD127 + , CD95 + ) T cells after NIVO.A population of CD4 + CD38 + CD39 + T cells at baseline was also associated with progression and shorter survival.
Peripheral blood immunophenotypes at baseline and after treatment are associated with patient outcomes after IPI > NIVO therapy.In IPI > NIVO-treated patients, 432 baseline immunophenotypes were associated with response to treatment and survival (Figure 3B, left panel).Of these, 376 were elevated in frequency

C L I N C A L M E D I C I N E
in responding relative to progressing patients.These 376 immunophenotypes formed 8 clusters and were a mixture of CD4 + and CD8 + T cell populations (Supplemental Figure 7, A and B).Clusters 5 and 8 were composed of CD4 + T cells expressing LAG3 + and GARP + .Fifty-six phenotypes, forming 5 clusters, were decreased in frequency in responding patients (Supplemental Figure 7, C and D).All clusters contained a predominance of CD45RO + and CD95 + immunophenotypes, while clusters 2, 4, and 5 also contained CCR7-expressing immunophenotypes.At week 13, after IPI treatment (Figure 3B, right panel), 668 immunophenotypes were associated with treatment response and survival, with 608 elevated and 60 at lower frequencies in responding compared with progressing patients (Figure 3B, right panel).As shown in Supplemental Figure 7E, immunophenotypes higher in relative frequency in responders formed 7 clusters.CD4 + and CD8 + T cell immunophenotypes with CCR7 + expression comprised clusters 1, 2, 3, 5, and 7 (Supplemental Figure 7F).The 60 immunophenotypes with lower frequencies in responders formed 6 clusters (Supplemental Figure 7G).These clusters were mixed populations of CD4 + and CD8 + T cells (Supplemental Figure 7H).Cluster 2 included T cells expressing CD38 and CD39.
Distinct immunophenotypes are associated with patient response in NIVO > IPI-and IPI > NIVO-treated patients.We next compared the immunophenotypes associated with outcome between NIVO > IPI and IPI > NIVO patients.As shown in Figure 3C, only 2 (related) immunophenotypes were associated with response in both cohorts.The associations were reciprocal between the 2 cohorts.These cells were CD14 + CD11C + C-D33 + CD15 -CD19 -CD66B -PDL1 -PDL2 + CD163 + GAL9 -CD80 -CD86 -41BBL + CD40 + OX40L + , and an identical immunophenotype in which CD66B was not measured.The frequency of this immunophenotype in responding and progressing patients for each treatment cohort is shown in Figure 3D.In NIVO > IPI-treated patients, higher frequencies of these immunophenotypes were associated with progression (P = 0.0164), while in IPI > NIVO-treated patients, higher frequencies were associated with patient response (P = 0.0246).This was reflected in the survival curves shown in Figure 3E.Frequencies of these phenotypes above the median were associated with shorter survival in NIVO > IPI-treated patients (P = 0.0050; HR 95% CI, 1.681-18.74)but with prolonged survival in IPI > NIVO treated patients (P = 0.0463; HR 95% CI, 1.019-9.038).
NIVO-associated immune landscape changes favor response-associated immunophenotypes in the IPI > NIVO cohort.In the Checkmate-064 trial, the NIVO > IPI treatment arm had greater rates of response and overall survival compared with the IPI > NIVO treatment arm (4).We hypothesized that the immunophenotypic impact of treatment with NIVO or IPI may have altered the immune landscape in a manner that influenced subsequent response or resistance to the second agent.We tested this hypothesis by determining the overlap between the immunophenotypes that changed significantly after the first treatment in the regimen and those associated with treatment response and survival in the opposing cohort.As depicted in Figure 4A, of the 3959 cell populations that were significantly altered after NIVO, 4 overlapped with those that were associated at baseline with response and survival in IPI > NIVO-treated patients.However, none of the markers measured were expressed by the cells in these 4 immunophenotypes (i.e., no positive markers; data not shown).An additional 95 overlapping immunophenotypes were found that associated with treatment response/patient survival at week 13 in the IPI > NIVO cohort and changed after NIVO treatment.A total of 99% (94 of 95) of these overlapping immunophenotypes were positively associated with response, with almost all (93 of 94) immunophenotypes increasing after NIVO, and they were elevated at week 13 in responders to IPI > NIVO.
We next evaluated the composition of the 95 immunophenotypes identified above associated with treatment response/patient survival at week 13 in the IPI > NIVO cohort (Supplemental Figure 8A).The 93 immunophenotypes that were increased at week 13 after NIVO and associated at baseline with response in IPI > NIVO (green dots in Supplemental Figure 8A) are described in Supplemental Figure 8B.These cells were all CD4 + and predominately CD45RO -and CCR7 + .A representative population of CD4 + C-D45RO -CCR7 + T cells is shown in Figure 5A.The left-most panel shows that most patients had an increase in the frequency of this population after NIVO treatment (P = 0.035).The second panel shows that higher frequencies of this population in IPI > NIVO-treated patients at week 13 were associated with response (P = 0.0046).The second to right panel shows that IPI > NIVO patients with greater than the median frequency of CD4 + CD45RO -CCR7 + T cells at week 13 have longer overall survival (P = 0.019; HR 95% CI, 1.343-27.42).To validate this result, we evaluated changes in the frequency of this immunophenotype in a separate cohort of metastatic melanoma patients treated with NIVO monotherapy (ClinicalTrials.govidentifier NCT01176461).Forty paired patient samples were assessed.The right-most panel of Figure 5A insight.jci.orghttps://doi.org/10.1172/jci.insight.137066

C L I N I C A L M D I C I N E
shows that, in this cohort of patients treated with NIVO, CD4 + CD45RO -CCR7 + T cells also increased in frequency after NIVO (P = 0.046).
IPI-associated immune landscape changes favor progression-related immunophenotypes in the NIVO > IPI cohort.In a similar assessment of the overlap between post-IPI changes and NIVO > IPI outcome-associated immunophenotypes, we identified 74 immunophenotypes that were altered by IPI treatment and whose frequency at

C L I N I C A M E D I C I N E
baseline in the NIVO > IPI treatment arm was associated with response/survival (Figure 4B).Similarly, we identified 36 immunophenotypes that were altered by IPI and whose presence at week 13 in the NIVO > IPI treatment arm was associated with response/survival (Figure 4B).Supplemental Figure 8, C-E, shows how these immunophenotypes cluster into groups of cell populations.Unlike the NIVO-associated changes, IPI-associated changes had a negative association with NIVO response.At baseline, 92% of the overlapping phenotypes (68 of 74; Figure 4B) were associated with progression; 67 of these immunophenotypes were increased after IPI and were found at relatively lower frequencies in NIVO > IPI responding patients.As shown in Supplemental Figure 8D, these immunophenotypes were all CD4 + CD38 + CD39 + GARP -CD127 -and varied with respect to other markers.A representative population from these 67 immunophenotypes, CD4 + CD38 + CD39 + CD127 -HELIOS -CD25 -LAG3 -CXCR3 - CCR4 -LAP -GARP -T cells, is shown in Figure 5B.The left-most panel shows that this population was increased after IPI (P = 0.039).The second panel shows that significantly higher frequencies of this population are seen in progressing patients in the NIVO > IPI cohort (P = 0.0079).The second to right panel shows that patients with greater than median frequency of this population also have significantly shorter survival (P < 0.0001; HR 95% CI, 4.399-69.23).To independently validate the association of this immunophenotype with patient response, we assessed the frequency of CD4 + CD38 + CD39 + CD127 -GARP -T cells in peripheral blood samples of metastatic melanoma patients treated with NIVO monotherapy.Twenty responding and 47 progressing patient samples were assessed.Shown in the right-most panel of Figure 5B, elevated levels of that immunophenotype were associated with progression of disease (P = 0.047).
At week 13, all (36 of 36) of the phenotypes altered by IPI were associated with poor treatment response/survival for NIVO > IPI patients.Of the 36 immunophenotypes identified, IPI increased the frequency of 30 of these cell populations, but patients responding to NIVO > IPI had significantly decreased frequencies of these cells.Supplemental Figure 8F shows how these immunophenotypes clustered, and Supplemental Figure 8, G and H, describes the marker composition of these clusters.The additional 6 overlapping immunophenotypes were decreased after IPI but had elevated frequencies at week 13 in NIVO > IPI responding patients.These 6 immunophenotypes included both CD4 + and CD8 + T cells and were predominately CD127 + , CD95 + , CD45RO + , and CCR7 + .A representative phenotype from these 6 is shown in Figure 5C.The left-most panel shows that this population was significantly decreased after IPI (P = 0.011).The middle panel shows that the frequency of this population at week 13 in NIVO > IPI patients is significantly higher in responders compared with progressors (P = 0.035).The second to right panel shows the survival benefit in patients with more than the median frequency of this population (P = 0.016; HR 95% CI, 1.322-15.49).However, shown in the right-most panel of Figure 5C, in a cohort of metastatic melanoma patients treated with NIVO, we were unable to confirm a relationship between reduced levels of these cells and disease progression (P = 0.530).
Changes in identified immunophenotypes correctly classify patient response in a cross-validated EN model.To determine if the changes in overlapping immunophenotypes described in Figure 4, A and B, were sufficient to predict patient outcomes, we used patient delta values for the 209 overlapping immunophenotypes and an EN model with leave-one-out (LOO) cross validation.As shown in Figure 4C, responding and progressing patients in the NIVO > IPI-treated cohort were accurately categorized with an AUC of 0.918.Figure 4D shows that patients in the IPI > NIVO-treated cohort were correctly categorized as responders or progressors, but with lesser sensitivity and specificity, resulting in an AUC of 0.786.These results further support the importance of immunophenotypic changes resulting from treatment and their relation to patient response.

Discussion
In the current study, we used a potentially novel and powerful approach to analyzing high-dimension flow cytometry data to assess the impact of the immune checkpoint inhibitors NIVO and IPI on the peripheral blood immune landscape.By assessing the frequencies of complex immunophenotypes in lieu of dimension-reduction analytical methods (e.g., tSNE), we were able to more precisely identify treatment-associated changes in the immunophenotypic landscape, identify response-associated immunophenotypes, and assess their relationships.Collectively, these data suggest that IPI and NIVO alter the peripheral blood immunophenotypic landscape of patients in distinct ways.Furthermore, IPI-associated alterations overlapped with immunophenotypes associated with progression of disease and shorter survival in the NIVO > IPI cohort.Although the overlap occurred with the patient responses to sequential NIVO > IPI noted

C L I N I A L M E D I C I N E
at week 25, overall responses deviated little from week-13 responses (after NIVO alone) (4).These results suggest that IPI-associated immune landscape changes may impair responses to subsequent NIVO and may explain the lower response rate and shorter survival seen in the IPI > NIVO cohort.
While the emergence of single-cell, high-dimension technologies has increased the ability to probe the antitumor immune response, approaches to analyze the complex data generated have failed to keep pace.Dimension-reduction techniques such as principal component analysis and tSNE are commonly used to visualize high-parameter data, but these techniques do not report the specific combination of markers expressed by cell types.A critical, unanswered question in high-parameter data analysis is whether clustering and dimension-reduction algorithms sufficiently capture all the cell types that differ across study groups.Classically, to exhaustively examine cell phenotypes in lower-parameter flow cytometry data sets, analysis has been performed by using combinatorics, a method that constructs all possible phenotypes from the markers measured.For example, using combinatorics, an n-color flow cytometry experiment would report the number of cells expressing each combination of expression (+) or lack of expression (-) for the n markers, resulting in 2 n phenotypes.The use of a neutral condition for each marker allows assessment of shorter and simpler phenotypes (12), resulting in 3 n phenotypes.The identification of simple phenotypes is critical for generating translatable discoveries from high-parameter technology.It would be unnecessarily complex and expensive to develop clinical tests to detect cell populations that are defined by much more than 3 markers, when simpler surrogates might readily be identified within a high-parameter data set.While combinatorics offers a means to precisely identify and quantify all cell subsets in a sample, it is computationally intensive.Even for a 10-parameter experiment measuring 1 million cells, combinatoric analysis using the R-based flowType algorithm requires 4 hours; in comparison, the same analysis is completed by the CytoBrute platform within 2 seconds.The improvement in computational time is attributable to the unique and proprietary distributive computing approach CytoBrute uses, and this approach is adaptable to other R-based algorithms and applications, including machine learning-based analysis of high-parameter data sets.
There are shortcomings in this study that impact the interpretation of the data.Rather than using traditional FDR approaches, we used a non-multiple comparison-adjusted P value of < 0.05 as a determination of significance.This approach was taken for several reasons.First, the number of samples available for assessment were limited, and in turn, the lower threshold of P values obtainable was limited.Second, the combinatoric nature of the CytoBrute approach creates nonindependent measurements, which would be highly overcorrected if using traditional multiple comparison corrections.However, given (a) the use of LOO cross validation in EN models, (b) the validation of important identified immunophenotypes in an independent cohort, and (c) our focus on demonstrating that the impact of IPI and NIVO are distinct and that the immune landscape changes induced by IPI are associated with lower response to NIVO and shorter survival in the IPI > NIVO cohort, the conclusions of this study are supported by the data.Many response-associated immunophenotypes not overlapping with treatment changes were also identified that have not yet been evaluated in a validation cohort.While beyond the scope of this study, these represent potentially important biomarkers and are the subject of future validation efforts.
Several thousand significant changes in peripheral blood immunophenotypes were observed after NIVO or IPI treatment, highlighting the systemic impact of these agents.These changes were largely distinct, with only < 5% overlap in significantly changed immunophenotypes.Of the few overlapping immunophenotypes, both treatments increased populations of CD4 + CD38 + CD39 + CD127 -GARP -T cells, which were found to be associated with poor outcomes.This population was also associated with progression in an independent cohort of NIVO-treated patients, validating the importance of this phenotype in patient response.To our knowledge, this population of T cells has not been previously described.CD39 is an ectonucleotidase that converts extracellular ATP into adenosine and has been the focus of many studies demonstrating its roles in generating an immunosuppressive tumor microenvironment, resulting in its emergence as a potential therapeutic target (13)(14)(15)(16)(17)(18).CD38 also functions as an ectoenzyme, both as a hydrolase and converting NAD + to cyclic ADP-ribose.It is being assessed as a target in combination with immunotherapy (19,20).In agreement with our data, CD38 expression was recently shown to be upregulated after immunotherapy and was found to be associated with negative outcomes in a murine checkpoint inhibition model and in human patients (21).Based on the current knowledge of the function of CD38 and CD39, and our observation that CD4 + CD38 + CD39 + immunophenotypic clusters are associated with poor patient outcomes, we hypothesize that this population generates an immunosuppressive microenvironment through the ectoenzymatic activity of CD38 and CD39.Investigations of mechanistic relationship to patient response, the function of this population, and the efficacy of targeting it are warranted by the data presented in this study and are ongoing.
In addition to the low level of overlap of immunophenotypic changes, no immunophenotypes associated with patient outcomes were found to be shared between the treatment sequences, with the exception of 2 related immunophenotypes.This further highlights the distinct impact of the 2 therapies.The 2 overlapping immunophenotypes were reciprocally associated with patient outcomes in the 2 cohorts.Based on the expression of CD14, CD11b, and CD33, these cells are likely of myeloid origin (22) and expressed a mixed inflammatory (e.g., 41BBL + , CD40 + ) (23,24) and suppressive phenotype (e.g., PDL2 + , CD80 -, CD86 -) (25-27).While beyond the scope of the present study, this cell population will be further interrogated in functional assays to determine potential mechanisms by which it could be associated with the disparate outcomes in the 2 treatment cohorts.
T cell phenotypes associated with memory subsets were consistently and significantly associated with patient outcomes in both treatment cohorts.In the NIVO > IPI cohort, responding patients had higher frequencies of T cells with a naive phenotype (CD4 + CD45RA + CD127 + ) at baseline and central memory phenotypes (CD4/8 + C-D45RO + CD127 + CD95 + CCR7 + ) after NIVO.Conversely, at baseline, progressing patients had more differentiated effector T cells (CD4 + CD45RO + CD95 + ) and higher frequencies of CD8 + CD45RA + T cells after NIVO.These data suggest that the formation of memory T cell phenotypes may be important for the efficacy of NIVO.In contrast, in the IPI > NIVO cohort, responding patients had increased frequencies of CD4/8 + CD45RO + CD95 + and CD4 + CD45RA + CD95 + T cells at baseline and increased frequencies of CD4 + CD45RO -CCR7 + T cells after IPI.Several other immunophenotypic clusters were found to be associated with patient response and survival, including CD4 + CD45RA + CD127 + , CD4 + CD45RO + CD95 + , and CD8 + LAG3 + T cell populations.Taken with data presented, showing a lack of overlap in treatment-associated changes and outcome-associated immunophenotypes between the 2 cohorts/treatments and published literature (6)(7)(8)(9), it is increasingly clear that the systemic immune impact and the mechanisms by which CTLA4 and PD1 blockade function are distinct.
Collectively, these data suggest that the impact of IPI and NIVO on the immunophenotypic landscape of patients is distinct and that the impact of IPI may be associated with resistance to subsequent NIVO therapy, consistent with poor outcomes in the IPI > NIVO cohort of Checkmate-064.In further support of this interpretation, in clinical trials, the response rates to NIVO in patients progressing after IPI are lower than those in IPI-naive patients (28)(29)(30)(31)(32).However, these response rates are compared across different studies, and lower response rates to NIVO in IPI-refractory patients may result from selection of immunotherapy-resistant patients.Response rates in the NIVO > IPI cohort were similar to concurrent NIVO and IPI, suggesting that concomitant IPI does not negatively impact NIVO efficacy.Regardless, the data presented herein raise concerns about the negative impact of IPI treatment before NIVO and warrant consideration in patient treatment decisions.Future studies will investigate immunophenotypic changes occurring with concurrent treatment and associated patient outcomes.Furthermore, studies will need to address whether the immune landscape changes resulting from IPI are normalized over time and the duration that takes to occur.Future investigations will also need to investigate the function of and potential value as biomarkers of immune cell populations that are associated with treatment outcomes in this study.

Methods
Patient samples and processing.Cryopreserved peripheral blood mononuclear cells (PBMC), obtained from samples collected at baseline and week 13 of the CA209-064 clinical trial (ClinicalTrials.govidentifier NCT01783938), were thawed, washed, and stained for flow cytometry in a single batch.Viability for all samples was > 85%.Samples were fixed in 0.5% paraformaldehyde.Baseline and week-13 PBMC obtained from patients treated with NIVO monotherapy as part of the CA209-006 clinical trial (ClinicalTrials.govidentifier NCT01176461) were similarly assessed in validation experiments.
Flow cytometry.Four, 24+ parameter antibody panels (Supplemental Table 1) were developed using ColorWheel software, an automated flow cytometry panel design tool that proposes candidate panels based on spillover-spreading error, dye brightness, and antigen density (33).Data were collected on a custom BD FACSymphony A5 30-parameter flow cytometer, using a 96-well plate reader.
Data analysis.After compensation for overlap in fluorescent spectra, removal of events representing fluorescence aggregates, and exclusion of cell doublets and dead cells, we identified fluorescence intensity thresholds (i.e., gates) that distinguished positive from negative expression of each marker studied.A general data cleanup and gating strategy is shown in Supplemental Figure 9. Fifteen markers were chosen from each panel for further analysis, based on the strength of antibody staining, variability across donors, and biological interest.CytoBrute (RocketML), an adaptation of the R-algorithm FlowType based on RocketML's rapid computing technology, was then used to measure the number of cells expressing every individual marker and every combination of 2-15 markers within the data file.CytoBrute reported the top 1000 most frequent combinatorial phenotypes in each data file, along with those found among the top 1000 in some patients but not the others.In total, approximately 80,000 cell populations were compared across patient groups and time points.
Many of these populations were not independent (i.e., one population shared markers with another); therefore, we developed approaches to better characterize and summarize the results of group-wise comparisons.We clustered cell populations using the UMAP R package with default settings to group similar

Figure 1 .
Figure 1.Overview of Approach.(A) PMBC samples were obtained from metastatic melanoma patients treated as part of the Checkmate-064 clinical trial.Patients were treated with sequential NIVO > IPI (Cohort A) or IPI > NIVO (Cohort B).Samples were collected before treatment (baseline) and after first agent (week 13).The indicated number of samples for each cohort and time point, broken down by patient response (responders in blue; nonresponders in red) are given.(B) A generalized illustration of how the data were assessed using combinatorics and CytoBrute is shown.Briefly, for each cell population (CD3 + CD8 + in the example illustration), cells were assessed for single markers and all combinations up to 15 markers in complexity.This was performed for 4 separate flow cytometry panels.(C) The number of theoretical immunophenotypes for 1-15 markers in complexity is shown by blue bars and the corresponding number of actual immunophenotypes measured in the data set are shown in red.(D) For the actual immunophenotypes measured, the number of positive markers measured at each increment of complexity is shown.

Figure 2 .
Figure 2. Nivolumab and ipilimumab differentially impact on peripheral blood immunophenotypes.(A) The median frequency at baseline on the x axis and the week-13 median frequency on the y axis is shown for significantly changed immunophenotypes (P < 0.05, Wilcoxon signed-rank test) in nivolumab-treated patient samples.Each dot represents an immunophenotype and is colored by P value.The purple dotted line with a slope of 1 corresponds to no change in median frequency.(B) Ipilimumab-treated patient samples are likewise shown.(C) A Venn diagram is shown with the number of significantly changed immunophenotypes in each group and the overlap.The 525 immunophenotypes are those overlapping with changes in the same direction in both NIVO-and IPI-treated patient samples.(D) The median relative change from baseline to week 13 in nivolumab-treated patient samples on the x axis and the relative change in ipilimumab-treated patient samples on the y axis is shown for the 584 overlapping immunophenotypes.The purple dotted lines correspond to no change in median frequency.(E) The delta values (week 13 minus baseline) of the 584 overlapping phenotypes were used in an elastic net regularized regression model to categorize whether a paired patient sample received nivolumab or ipilimumab treatment.The receiver operator characteristic (ROC) and resulting AUC for all paired samples is shown by the dotted black line in the left panel.The ROC and AUC for responding patient samples is shown in blue and for progressing patient samples in red.The model values for nivolumab and ipilimumab-treated paired patient samples are plotted in the right panel.Box plots show median ± quartiles with whiskers indicating range.

Figure 3 .
Figure 3. Immunophenotypes associated with patient response are distinct in nivolumab and ipilimumab sequentially treated patients.(A) The median frequency of immunophenotypes that are significantly different for both response (determined by Mann-Whitney U test) and overall survival (determined by Mantel-Cox test) in nonresponding patients are shown on the x axis and in responding patients on the y axis for nivolumab > ipilimumab-treated patient samples.Each dot represents an immunophenotype and is colored by the P value of the comparison between responders and nonresponders, for that cell population.The purple dotted line with a slope of 1 corresponds to no change in median frequency.Significantly different immunophenotypes in baseline patient samples are shown in the left panel, and significantly different immunophenotypes in week-13 patient samples are shown in the right panel.(B) Immunophenotypes for ipilimumab > nivolumab-treated patients are likewise shown.(C) A Venn diagram is shown with the number of immunophenotypes significantly different in each cohort and time point.(D) A graph of 1 of the 2 related significantly different immunophenotypes overlapping between nivolumab > ipilimumab-and ipilimumab > nivolumab-treated patients at baseline is shown.Frequencies of the populations shown are plotted by cohort and response.Each dot represents an individual patient sample.Box plots show median ± quartiles, with whiskers indicating range.(E) A survival plot for this immunophenotype is also shown.Patients were stratified based on median frequency of the immunophenotype.Nivolumab > ipilimumab-treated patients with greater than median frequencies are shown in blue and less than median frequency in green.Ipilimumab > nivolumab-treated patients with greater than median frequencies are shown in red and less than median are shown in purple.

Figure 4 .
Figure 4. Ipilimumab-induced immunophenotypic changes are associated with lack of response to sequential nivolumab-ipilimumab. (A) A Venn diagram is shown with the number of immunophenotypes significantly changed after nivolumab (purple circle) and immunophenotypes significantly different for survival and response in ipilimumab > nivolumab-treated patients at baseline (top orange circle) and after ipilimumab (bottom orange circle).The direction of change and relative representation in responding patients for overlapping immunophenotypes at baseline (top quadrant plot) and after ipilimumab (bottom quadrant plot) is plotted.The number of immunophenotypes significantly increased after nivolumab and lower in ipilimumab-nivolumab responding patients are shown in the top left quadrants.The number of immunophenotypes significantly increased after nivolumab and higher in ipilimumab > nivolumab responding patients are shown in the top right quadrants.The number of immunophenotypes significantly decreased after nivolumab and lower in ipilimumab > nivolumab responding patients are shown in the bottom left quadrants.The number of immunophenotypes significantly decreased after nivolumab and higher in ipilimumab > nivolumab responding patients are shown in the bottom right quadrants.(B) A similar Venn diagram, along with quadrant plots, is shown for significant ipilimumab changes and nivolumab-ipilimumab response/survival-associated immunophenotypes.(C and D) Delta values (weeks 13-0) for the 210 immunophenotypes significantly changed after nivolumab or after ipilimumab and associated with response/survival in ipilimumab > nivolumab-or nivolumab > ipilimumab-treated patients were used in an elastic net regularized regression model to categorize patient response.(C) The ROC and AUC for patients in the nivolumab > ipilimumab-treated cohort are shown.The model values for nonresponding and responding patients are plotted in the bottom right inlay.Box plots show median ± quartiles, with whiskers indicating range.(D) Results for ipilimumab > nivolumab-treated patients are likewise shown.

Figure 5 .
Figure 5. Ipilimumab-induced immunophenotypic changes are associated with lack of response to sequential nivolumab > ipilimumab.(A) A representative immunophenotype increased after nivolumab and having an increased frequency in responding patients is shown.The left-most graph shows the paired frequency changes in nivolumab-treated patients (significance determined using Wilcoxon signed-rank test).Responding patients are represented by blue lines, nonresponding patients with red lines, and nonevaluable patients in black.The second to left graph plots the week-13, postipilimumab frequencies of this population comparing responding and nonresponding patients.Box plots show median ± quartiles, with whiskers indicating range.Significance was determined by Mann-Whitney U test.The third panel is a survival plot for this immunophenotype.Patients were stratified based on median frequency of the immunophenotype at week 13.Patients with greater than median frequencies are shown in orange and less than median frequency in green.Significance was determined by Mantel-Cox test.The right-most graph shows the paired frequency changes in an independent validation cohort of nivolumab-treated patients.Significance was determined by Mann-Whitney U test.(B) A representative immunophenotype increasing after ipilimumab and having decreased frequency at baseline in nivolumab > ipilimumab responding patients is likewise shown.(C) A representative immunophenotype decreasing after ipilimumab and having increased frequency at baseline in nivolumab > ipilimumab responding patients is likewise shown.