<!-- page 1 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## **NATIONAL INSTITUTE FOR HEALTH AND CARE EXCELLENCE** 

## **Final appraisal determination** 

# **Nivolumab for treating locally advanced unresectable or metastatic urothelial cancer after platinum-containing chemotherapy** 

## **1 Recommendations** 

- 1.1 Nivolumab is not recommended, within its marketing authorisation, as an option for treating locally advanced unresectable or metastatic urothelial carcinoma in adults who have had platinum-containing therapy. 

- 1.2 This recommendation is not intended to affect treatment with nivolumab that was started in the NHS before this guidance was published. People having treatment outside this recommendation may continue without change to the funding arrangements in place for them before this guidance was published, until they and their NHS clinician consider it appropriate to stop. 

## **Why the committee made these recommendations** 

Treatment options for people with locally advanced unresectable or metastatic urothelial carcinoma who have had platinum-containing therapy are limited. They are usually offered docetaxel, paclitaxel and best supportive care. 

Nivolumab has been studied in a clinical trial, but it has not been directly compared with other treatments. So it is not clear how effective nivolumab is compared with current clinical practice. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 1 of 22 Issue date: May 2018 




<!-- page 2 -->

## **CONFIDENTIAL UNTIL PUBLISHED** 

Nivolumab meets NICE’s criteria to be considered a life-extending treatment at the end-of-life. The committee agreed that the assumptions incorporated in the ERGs revised base-case were mostly consistent with its preferred assumptions. The committee agreed that the most plausible ICERs were somewhere between the ERG’s estimates of £58,791 per QALY gained (versus paclitaxel) and £78,869 per QALY gained (versus docetaxel), above what NICE normally considers to be acceptable for end-of-life treatments. There was substantial uncertainty because the model used a simulated treatment comparison, so the ICER could be considerably higher. Therefore nivolumab could not be recommended for routine use in the NHS for locally advanced, unresectable or metastatic urothelial cancer after platinum-containing chemotherapy. 

Because neither data collection from clinical practice or the ongoing trials would resolve the identified uncertainty, nivolumab is not suitable for use within the Cancer Drugs Fund for people with unresectable or metastatic urothelial cancer after platinum-containing therapy. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 2 of 22 Issue date: May 2018 




<!-- page 3 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

**2 The technology** 

|**2**<br>**The technology**|**2**<br>**The technology**|
|---|---|
|**Nivolumab (Opdivo, Bristol-Myers Squibb)**||
|**Marketing authorisation**|Nivolumab as monotherapy is indicated for the<br>treatment of locally advanced unresectable or<br>metastatic urothelial carcinoma in adults after failure<br>of platinum-containing therapy.|
|**Recommended dose and**<br>**schedule**|3 mg/kg by intravenous infusion every 2 weeks.|
|**Price**|£439 per 40-mg vial or £1,097 per 100-mg vial<br>(excluding VAT; British national formulary online,<br>accessed September 2017).<br>The company has agreed a patient access scheme<br>with the Department of Health. If nivolumab had been<br>recommended, this scheme would provide a simple<br>discount to the list price of nivolumab with the<br>discount applied at the point of purchase or invoice.<br>The level of the discount is commercial in confidence.<br>The Department of Health considered that this patient<br>access scheme would not constitute an excessive<br>administrative burden on the NHS.|



## **3 Committee discussion** 

The appraisal committee (section 5) considered evidence submitted by Bristol-Myers Squibb and a review of this submission by the evidence review group (ERG). See the committee papers full details of the evidence. 

## _**The condition**_ 

## **Urothelial carcinoma substantially decreases quality of life** 

- 3.1 Urothelial carcinoma causes a number of symptoms, including haematuria (blood in the urine) and increased frequency, urgency and pain associated with urination. The patient experts commented that chemotherapy is associated with unpleasant adverse effects such as fatigue, nausea and vomiting and places people at more risk of infection. The committee was aware that many people with the disease are older and may have comorbidities, which can affect the choice of treatment. The committee recognised that locally advanced unresectable or metastatic urothelial carcinoma has a substantial effect on quality of life. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 3 of 22 Issue date: May 2018 




<!-- page 4 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## _**Clinical management**_ 

## **There is unmet need for effective treatment options** 

- 3.2 Initial treatment is usually with a cisplatin-containing chemotherapy regimen. Treatment options for people whose disease progresses after platinum-containing chemotherapy include docetaxel, paclitaxel or best supportive care. The clinical experts explained that none of these treatments offer lasting benefit and that prognosis is poor, even for people having their first therapy. The patient experts explained that the adverse effects of chemotherapy can have a large negative effect on quality of life and that regular hospital visits for treatment disrupt usual activities. The clinical experts noted that there have been no new treatments for locally advanced unresectable or metastatic urothelial carcinoma for a number of years and that, unlike for other cancers, there is no targeted or personalised treatment available after platinum-containing chemotherapy. The committee concluded that there is an unmet need for effective treatment options for people with locally advanced unresectable or metastatic urothelial carcinoma who have had platinum-containing chemotherapy. 

## _**Comparators**_ 

## **Paclitaxel, docetaxel and best supportive care are relevant comparators for people who have had platinum-containing chemotherapy** 

- 3.3 The company submitted clinical- and cost-effectiveness analyses comparing nivolumab with paclitaxel, docetaxel, and best supportive care. The committee understood that because nivolumab is an immunotherapy with a different adverse effect profile to taxanes (such as paclitaxel and docetaxel), there may be some people for whom nivolumab is suitable who would otherwise have best supportive care. It recognised that introducing immunotherapy may change clinical practice in the future, but that best supportive care is currently a treatment option for urothelial carcinoma and is therefore a relevant comparator. The committee 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 4 of 22 





<!-- page 5 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

understood that re-treatment with first-line chemotherapy was used before a standard second-line treatment option became available, and that now most clinicians would use a taxane. The clinical experts explained that retreatment with a first-line chemotherapy would most likely be for disease that had responded well, in people whose disease has not progressed for a long period of time after first-line treatment and who are fit enough to have re-treatment with platinum. Second-line treatment is with a taxane. The clinical experts explained that paclitaxel is used as current standard of care in the UK because of its availability and favourable adverse effect profile compared with docetaxel. Clinical experts in other ongoing immunotherapy appraisals for this population explained that both 

paclitaxel and docetaxel could be considered clinically equivalent and are both used in clinical practice. The committee concluded that docetaxel, paclitaxel, and best supportive care are appropriate comparators, but retreatment with first-line chemotherapy is not. 

## _**Clinical trial evidence**_ 

## **The CheckMate trials are broadly generalisable to UK clinical practice** 

3.4 The clinical-effectiveness evidence for nivolumab comes from 2 phase II, single-arm trials; CheckMate 275 and CheckMate 032. The trials included: 

- 270 patients with locally advanced unresectable or metastatic urothelial carcinoma with disease progression or recurrence after treatment with at least 1 platinum-containing agent (CheckMate 275), and 

- 78 patients with carcinoma of the renal pelvis, ureter, bladder, or urethra and disease progression after treatment with at least 1 platinum-containing chemotherapy (CheckMate 032). 

There is a lack of UK patients in the trials and more patients with an ECOG performance status of 0 compared with those seen in clinical practice, which might affect the generalisability of the results. The ERG stated that 23% of people in CheckMate 032 switched to nivolumab plus 

ipilimumab at disease progression. The committee noted that this would Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 5 of 22 Issue date: May 2018 




<!-- page 6 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

bias the efficacy results from this study, but acknowledged that when pooled with CheckMate 275 the proportion of patients switching treatment in the entire pooled population is low, and therefore the overall effect is minimal. It also noted a difference in the mean age of people in the trial (66 years) compared with clinical practice, which the experts suggested is around 75 years. The clinical experts explained that this could suggest a role for using nivolumab in younger people whose disease has newly progressed. The committee accepted that the patient populations in the CheckMate trials are broadly generalisable to those seen in UK practice. 

## **The CheckMate trials provide efficacy estimates for nivolumab but no randomised controlled trial evidence is available** 

- 3.5 Data from CheckMate 275 reported an objective response rate of 20.0% (95% confidence interval [CI] 15.4 to 25.3) and median overall survival of 8.57 months (95% CI 6.05 to 11.27). The data originally presented for CheckMate 032 reported an objective response rate of 24.4% (95% CI 15.3 to 35.4) and median overall survival of 6.51 months (95% CI 1.91 to not estimable). The company provided updated clinical-effectiveness data for both CheckMate 275 and CheckMate 032. The updated results are confidential and cannot be presented here. The company included the latest CheckMate 032 data in its economic analysis. The company stated that the updated results from CheckMate 275 confirm the original data presented at the first appraisal committee meeting, but did not include the updated figures in its economic analysis. The committee agreed that the availability of the latest CheckMate 275 data is not likely to have a substantial effect on clinical-effectiveness estimates. It was concerned that, without a trial directly comparing nivolumab with other treatments, it is difficult to reliably assess the relative treatment benefit of nivolumab. The committee also noted that the trial data are immature and based on small numbers of patients, and therefore there is considerable uncertainty in the results. The clinical experts highlighted that people whose disease responds to treatment with an immunotherapy such as nivolumab can 

have a lasting response, good quality of life and prolonged survival. They 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 6 of 22 





<!-- page 7 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

explained that the novel mechanism of action of immunotherapies such as nivolumab represent an important new treatment option in clinical practice. The committee concluded that it would be challenging to accurately assess the relative treatment benefit of nivolumab without any available randomised controlled trial evidence. 

## _**Indirect comparison**_ 

## **The results of the simulated treatment comparison need to be treated with caution because the analysis was unanchored** 

- 3.6 Nivolumab has only been studied in single-arm trials for previously treated urothelial cell carcinoma., To compare nivolumab with the relevant comparators the company did a simulated treatment comparison and network meta-analysis. This was an unanchored comparison because none of the evidence included in the analysis shared a common comparator. The committee was aware that bias is introduced into a simulated treatment comparison if all important prognostic factors are not accounted for. It considered that it is unlikely that all of the important prognostic factors had been accounted for in the simulated treatment comparison, therefore effecting the robustness of the results. The ERG explained that the way to test the external validity of the simulated treatment comparison is the out-of-sample method. This could be used to assess the presence of bias in the comparison model. The company explained that because of the limited availability of data, this validation method would not provide an accurate estimation of bias. The ERG noted that the effect of using alternative prognostic factors in the prediction model could have been assessed in a sensitivity analysis, and this should have been done to test the robustness of the comparison. The committee concluded that, because of the concerns about the robustness of the simulated treatment comparison, the results of the analysis need to be treated with caution. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 7 of 22 Issue date: May 2018 




<!-- page 8 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## **The network meta-analysis produced results which are inconsistent with current clinical expectations about the effectiveness of nivolumab** 

- 3.7 The company linked the results of the individual simulated treatment comparisons together through a network meta-analysis, using a fractional polynomial model. The committee noted that this is not a conventional modelling approach for a network meta-analysis. It was concerned that a lack of evidence in the network increased the reliance on the fractional polynomial model to estimate relative treatment benefit of nivolumab. The ERG explained that the fractional polynomial model is a highly flexible form of analysis. But, it expressed reservations about the robustness of the fractional polynomial model and noted that incremental costeffectiveness ratio (ICER) estimates were highly sensitive to the parameterisation of the model. The company stated that the evidence base for paclitaxel that informs the comparison with nivolumab is the most robust of the comparisons presented; it comes from a UK-only, randomised controlled trial. The committee acknowledged that although the evidence for paclitaxel came from a relevant randomised control trial, only the results for the paclitaxel arm were included in the indirect comparison. This removed the benefits of randomisation. The committee noted that including the results from a simulated treatment comparison which was not robust would affect the meta-analysis results, although it is unclear to what extent. The committee agreed that the results of the indirect comparison should be treated with caution. This is because the optimal parameterisation of the fractional polynomial is unknown and the network of evidence is sparse. The time varying hazard ratios used to infer relative treatment effect of nivolumab for overall survival and progression-free survival were estimated using the results of the network meta-analysis. The committee understood that results from the company’s network meta-analysis suggest that, compared with docetaxel and best supportive care, the relative effectiveness of nivolumab decreases with time. It agreed that this was inconsistent with clinical expectations about the effectiveness of nivolumab given that it is expected to have a long- 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 8 of 22 





<!-- page 9 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

lasting effect because of its novel mechanism of action. The committee concluded that the relative-effectiveness estimates inferred from the network meta-analysis are counterintuitive and associated with uncertainty, which needs to be accounted for in its decision-making. 

## _**Adverse events**_ 

## **Nivolumab is well tolerated** 

- 3.8 The clinical experts explained that in their experience of using nivolumab, it is well tolerated and has a preferable adverse effect profile compared with comparator chemotherapies. They stated that the rate of serious adverse effects from nivolumab are similar to those seen for chemotherapies. They noted that the mortality risk from current treatments has decreased as clinical understanding improves after more wide spread use of the treatment. The committee noted that a similar trend could happen if nivolumab was recommended, with treatment-related mortality dropping as clinical understanding improves. It noted that it is challenging to make a robust comparison of adverse events without randomised control trial evidence. The committee acknowledged that nivolumab is associated with some rare but unpleasant and potentially serious adverse events that are specific to immunotherapy. It concluded that nivolumab may be a tolerable alternative to chemotherapies as more experience is gained with this type of treatment. 

## _**Assumptions used in economic model**_ 

## **The use of standard parametric time-to-event survival analysis is preferred to a response-based approach** 

- 3.9 The company stated that standard parametric time-to-event models are unsuitable for modelling the possible sustained and long-term response to treatment expected with nivolumab. To account for this, the company modelled survival using a response-based analysis. This approach 

   - modelled survival for people until a pre-determined time point (landmark), 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 9 of 22 





<!-- page 10 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

when survival was individually assessed according to response to treatment. The company opted for an 8-week landmark point, based on the median time to response in the CheckMate trials. The ERG explained that alternative landmarks were not fully explored and therefore the effect on the ICER was not appropriately assessed. The company used the Kaplan–Meier estimates up until the landmark, at which point parametric distributions were fitted to the responder and non-responder curves to model progression-free survival and overall survival. The ERG stated that the company did not provide a mathematical justification to support their argument that a different response cannot be accurately described by standard parametric survival models. It explained that standard 

approaches are flexible enough to accurately model different responses, without needing to introduce unnecessary assumptions in to the analysis. The committee noted it had not seen any firm evidence to show that the response-based model was an adequate method to model long-term outcomes. The ERG preferred to estimate overall and progression-free survival by fitting a generalised gamma function to the trial data. The committee agreed that the response-based approach could be explored for modelling survival but agreed with the ERG that the company’s approach introduced unnecessary complexity into the modelling of survival. Therefore more evidence would be needed to support its appropriateness in preference to established modelling methods. 

## **Using a conventional parametric time-to-event survival analysis is considered to be the most appropriate** 

- 3.10 The clinical experts explained that people with urothelial carcinoma that has been treated with platinum-containing chemotherapy have a mean life expectancy of around 12 months, and that survival at 5 years is 

   - uncommon. The committee understood that the overall-survival data for nivolumab are too immature to provide a reference for the estimates generated in the survival models. The committee was aware that model projections for 5-year survival from other immunotherapy appraisals are 

about 10%, which was in line with the clinical expert opinion. It agreed that Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 10 of 22 





<!-- page 11 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

this estimate was an acceptable reference to validate the estimates produced by the different modelling approaches. The clinical experts stated that around 2 to 3% of people would be expected to be alive 5 years after treatment with current standard of care. It noted that estimates for 5-year survival in the response-based model are around 20% for nivolumab and around 6 to 8% for taxanes (values derived from the company’s economic model). The ERG explained that the 5-year survival estimates from its preferred survival model are around 13% for nivolumab and between 2 to 5% for taxanes. The committee noted that in the company’s model, the slope of the overall-survival curve for people whose disease responds to nivolumab is nearly flat. This suggests that the proportion of people in this group who stay alive would decrease slowly. The committee had concerns that the overall-survival prospects of this group may exceed that of an equivalent disease-free population, which would be implausible. It agreed that the overall-survival curve produces an implausible estimation of survival in the long term, because the response-based model does not appear to accurately characterise survival outcomes in this population. It was concerned that the company’s model overestimates the number of people who would be alive at 5 years and that people surviving past 5 years are effectively considered cured. This claim is not supported by the evidence. The committee preferred the ERG’s approach to modelling survival, because it produces estimates that are more consistent with clinical expert opinion and are therefore more clinically plausible. The committee concluded that the use of conventional parametric time-to-event survival analysis was the most appropriate method on which to base its decision. 

## **The conventional fully fitted survival model is the most appropriate for estimating survival outcomes** 

- 3.11 The company presented an alternative scenario to model survival rather 

   - than the ERG’s non-response-based approach. In the non-responsebased analysis, survival was estimated using a piecewise model (using 

Kaplan–Meier data for the first part of the time horizon before switching to Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 11 of 22 





<!-- page 12 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

a parametric distribution for the extrapolation of longer-term survival estimates). The committee acknowledged that this method of modelling survival has been applied in other immunotherapy appraisals. It understood that both the conventional (fully fitted) and piecewise approaches have enough evidence to support their suitability for modelling survival, and have shown validity in other appraisals. The committee noted that there was little description of the company’s assumptions used in the piecewise model approach. There was also no justification or exploration of the type of extrapolations that could be used or the time point selected to switch from the Kaplan–Meier data to the parametric model. The company commented that the probabilistic sensitivity analysis did not fully work for the piecewise model and could not be presented. For these reasons the committee could not accept the outputs of the piecewise model presented by the company. It reaffirmed its conclusion that for this appraisal the conventional fully fitted survival model (ERG’s approach) was the most appropriate for estimating survival outcomes. 

## **Pooling of utility estimates from CheckMate 275 and CheckMate 032 in the model is appropriate for this appraisal** 

- 3.12 EQ-5D data were collected directly in CheckMate 275 and CheckMate 032, which is the preferred measure of health-related quality of life in adults. Pre-progression utilities (0.736) and post-progression utilities (0.623) were derived from pooling CheckMate 275 and CheckMate 032 values, with missing values being imputed. Disutilities for adverse events were derived from the literature. The ERG explained that the utility decrements used by the company were inconsistent with those used in a previous nivolumab appraisal. The committee noted that in the company’s estimation of the treatment effect of nivolumab, both CheckMate trials were pooled, and it agreed that a consistent approach should be taken for estimating utility values. The committee concluded that the pooling of utility estimates from the 2 trials is acceptable in this case. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 12 of 22 Issue date: May 2018 




<!-- page 13 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## **The application of a 2-year treatment stopping rule reduced costs associated with nivolumab but the effect on long-term efficacy is unknown** 

- 3.13 The company presented evidence from an ongoing study (CheckMate 003) of nivolumab in a different disease area. CheckMate 003 applied a stopping rule for nivolumab at 2 years (when all patients stop treatment). The company noted that people in CheckMate 003 continued to benefit from treatment beyond the point at which it was stopped. The company included a 2-year treatment stopping rule into its revised economic analysis. It explained that a 2-year stopping rule has been accepted in another immunotherapy appraisal of urothelial carcinoma. The committee recalled that when it accepted the 2-year stopping rule for another immunotherapy appraisal, the trial protocol for the study informing the economic analysis had mandated a maximum treatment duration of 2 years. The ERG explained that applying the 2-year stopping rule in the economic model stopped costs associated with nivolumab, but treatment benefit of nivolumab was assumed unchanged. The ERG did not include a 2-year stopping rule in its revised analysis, because the effect on clinical outcomes after treatment stops is unclear. The committee noted that the duration of continued effect after treatment has stopped is an area of uncertainty for new immunotherapies, but it agreed that a lifetime continued treatment effect is implausible. The committee concluded that implementing a treatment stopping rule while assuming lifetime treatment benefit was inappropriate. 

## **Assumption of a lifetime treatment benefit is implausible and scenarios varying this parameter in the economic model produced illogical results** 

- 3.14 The company presented scenarios in which the continued treatment effect ended after 3 years or 5 years from the start of the model. In the comparison of nivolumab and docetaxel, the committee understood that stopping treatment benefit for nivolumab at 3 or 5 years reduced the ICER compared with the company’s base case (using a response-based model in both scenarios). It noted that the ICERs would be expected to increase 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 13 of 22 Issue date: May 2018 




<!-- page 14 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

when the additional benefit of nivolumab on survival was assumed to stop at these time points (that is, assume a hazard ratio of 1). The committee understood that quality-adjusted life year (QALY) gains accrued in the model for nivolumab must happen in the short-term if stopping treatment benefit at 3 years or 5 years decreases the ICER. It agreed that the change seen in the ICER is contradictory to the company’s statement that people treated with nivolumab will have a long-lasting response with nivolumab. Stopping treatment benefit in the comparison of nivolumab and paclitaxel increased the ICER compared with the company’s base case. The committee noted that paclitaxel and docetaxel can be considered clinically equivalent (see section 3.3). It agreed that inconsistencies in the direction of change in the ICERs confirmed its’ concerns about the validity of the company’s economic model outputs, because it implies results that contradict current clinical expectations about the long-term treatment benefit of nivolumab. The committee concluded that the model gives counterintuitive results when different durations of treatment benefit are used. 

## _**Cost-effectiveness estimates**_ 

## **The choice of survival modelling approach has the largest effect on the ICER** 

- 3.15 The company’s deterministic revised base-case ICERs are £28,263 per QALY gained compared with docetaxel and £23,497 per QALY gained compared with paclitaxel, both with a 2-year treatment stopping rule applied for nivolumab. The ERG made 3 amendments to the company’s base case: 

   - used a non-response-based, conventional survival analysis 

   - assumed that only doses delayed by 7 days or more are missed doses 

   - did not include a 2-year treatment stopping rule. 

The change from the company’s preferred response-based modelling 

approach to a conventional time-to-event survival analysis approach has 

the largest effect on the ICER. The ERG’s preferred deterministic ICERs Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 14 of 22 





<!-- page 15 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

including the 3 amendments are £78,869 per QALY gained compared with docetaxel and £58,791 per QALY gained compared with paclitaxel. 

## **Probabilistic ICERs are preferred to deterministic but were not presented** 

- 3.16 The committee noted that the company and the ERG did not produce probabilistic ICERs in their post-appraisal consultation document reports. The ERG stated that the company’s probabilistic ICERs, presented at the first appraisal committee meeting, are unstable even at a high number of iterations. It also explained that the probabilistic sensitivity analysis is insufficient, because it did not include relative effectiveness which was the largest contributor to decision uncertainty. The company explained this omission from the analysis by stating that sampling the time-dependent hazard ratios in each period independently would yield counterintuitive results. The committee recalled that the probabilistic ICERs generated from the response-based model are higher than the deterministic ICERs. It also recalled that the disparity between probabilistic and deterministic ICERs is mostly resolved using the conventional survival modelling approach. The committee concluded that it would have preferred to have made its assessment of cost effectiveness using probabilistic ICERs, and agreed that the probabilistic ICERs would likely be higher than the deterministic values reported. 

## **ICERs produced from the economic model need to be treated with caution, and the most plausible ICER is likely to be over £50,000 per QALY gained** 

- 3.17 The committee agreed that the company’s economic model had not responded as expected in a number of scenarios. It noted that the network meta-analysis produced estimates that are inconsistent with clinical expectations (see section 3.7). The committee recalled that any analyses using these estimates need to be treated with caution, and that the response-based approach resulted in survival estimates that are implausibly high (see section 3.9). It noted that the company’s piecewise survival model could not be accepted because there was little description of the assumptions used in the model (see section 0). The committee 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 15 of 22 





<!-- page 16 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

agreed that stopping the ongoing treatment benefit in the model produced counterintuitive results (see section 3.14). It also understood that the ICERs reported by the company and the ERG are deterministic, and that probabilistic ICERs are likely to be higher (see section 3.16). The committee recognised that all ICERs produced from the analyses need to be treated with caution. It agreed that the assumptions in the ERGs revised base case are mostly consistent with its preferences. Therefore, the committee’s estimate of the most plausible ICER is based on the ERGs revised base-case analysis. The committee agreed that the most plausible ICER is somewhere between the ERG’s estimates of £58,791 per QALY gained (compared with paclitaxel) and £78,869 per QALY gained (compared with docetaxel). The committee would expect probabilistic ICERs to be higher still. Given the counterintuitive results from the economic model, the committee agreed that the ICERs need to be treated with caution. It concluded that, based on the evidence provided, its estimate of the most plausible ICER would be over £50,000 per QALY gained. 

## _**PD-L1 subgroups**_ 

## **There is not enough evidence to suggest that PD-L1 expression is predictive of survival outcomes or treatment response** 

- 3.18 In the first appraisal committee meeting, the committee considered whether there are any subgroups for whom nivolumab may be more cost effective. Nivolumab inhibits the PD-L1 protein and therefore it may be more clinically and cost effective in people with higher levels of PD-L1 expression. In CheckMate 275, there was a statistically significant difference in median overall survival in people with PD-L1 of more than 1% (11.63 months) compared with those with less than 1% (5.95 months). Similar trends were seen for progression-free survival outcomes in both the CheckMate trials. The committee noted that nivolumab appears to be more clinically effective in people with higher levels of PD-L1 based on the subgroup analyses presented by the company. The clinical experts stated 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 16 of 22 





<!-- page 17 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

that there is not enough evidence to separately assess the effectiveness of nivolumab according to PD-L1 expression. The committee considered the European Medicine Agency’s interpretation of the clinical evidence, noting that 12-month survival in the ‘PD-L1 less than 1%’ subgroup appears similar to that seen in larger trials for single-agent chemotherapy. The committee concluded that there is not enough evidence to suggest that PD-L1 expression is predictive of outcome and it was unable to make recommendations for any subgroups based on PD-L1 expression. 

## _**End of life**_ 

## **Life expectancy for people with urothelial carcinoma is less than 24 months** 

- 3.19 The committee considered the advice about life-extending treatments for people with a short life expectancy in NICE’s Cancer Drugs Fund technology appraisal process and methods. 

- 3.20 For people with locally advanced or metastatic disease who have had platinum-containing chemotherapy, data from the company’s model and from the literature show that overall survival is much less than 24 months for people having treatment with standard care (14.4 to 21.4 months). The clinical experts agreed that they would expect people with locally advanced or metastatic urothelial carcinoma, who have had a platinumcontaining chemotherapy, to live for less than 24 months. The committee concluded that the population meets the short life expectancy criterion. 

## **Nivolumab is likely to extend life by at least 3 months** 

- 3.21 The committee noted that because of the lack of phase III data directly comparing nivolumab with other treatments, it is difficult to make robust conclusions about overall survival gain. But, data from the company’s model and from the literature suggest a difference in median survival of at least 17.8 months. The committee noted the ERG’s comments that these estimates are from the economic model, which used data from single-arm trials, and are therefore based on very weak evidence. The committee 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 17 of 22 Issue date: May 2018 




<!-- page 18 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

acknowledged the limitations in the evidence but accepted that, on balance, it is likely that nivolumab extends life by more than 3 months. 

## **Nivolumab meets the criteria for end-of-life treatments** 

- 3.22 The committee recognised that there are important limitations in the evidence available. It concluded that the end-of-life criteria are likely to be met for this population, although it has not been presented with robust evidence for the extension-to-life criterion. 

## _**Routine commissioning**_ 

## **Nivolumab is not recommended for routine use in the NHS** 

- 3.23 The committee concluded that the most plausible ICERs (see section 3.15 ) are higher than those usually considered a cost-effective use of NHS resources for end-of-life treatments. The clinical and costeffectiveness estimates are highly uncertain because they are based on the simulated treatment comparison, and this needed to be accounted for when considering the maximum acceptable ICER. The committee did not recommend nivolumab for routine use in the NHS for people with locally advanced unresectable or metastatic urothelial carcinoma after platinumcontaining therapy. 

## _**Cancer Drugs Fund**_ 

- 3.24 Having concluded that nivolumab could not be recommended for routine use, the committee considered if it could be recommended for treating metastatic or unresectable urothelial cancer after platinum-containing therapy within the Cancer Drugs Fund. The committee discussed the new arrangements for the Cancer Drugs Fund agreed by NICE and NHS England in 2016, noting the addendum to the NICE process and methods guides. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 18 of 22 Issue date: May 2018 




<!-- page 19 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## **Nivolumab does not have the potential to be recommended for routine use** 

- 3.25 The committee’s preferred ICERs are all substantially higher than the range usually considered a cost-effective use of NHS resources for endof-life treatments. Furthermore, given the evidence base and modelling approaches adopted, the uncertainty around the cost effectiveness of nivolumab is increased substantially. The committee concluded that there is no plausible potential that nivolumab will satisfy the criteria for routine use in this population, not least because there are no planned or ongoing trials that could address the key clinical uncertainties identified. It did not recommend nivolumab for use within the Cancer Drugs Fund as an option for people with metastatic or unresectable urothelial cancer after platinumcontaining therapy. 

## _**Other factors**_ 

- 3.26 No equality issues were identified. 

- 3.27 The Pharmaceutical Price Regulation Scheme (2014) payment mechanism is not relevant in considering the cost effectiveness of nivolumab. 

- 3.28 The company did not highlight any additional benefits that had not been captured in the QALY. 

## _**Proposal for the Cancer Drugs Fund**_ 

## **Nivolumab is not suitable for inclusion in the Cancer Drugs Fund** 

- 3.29 After release of the final appraisal determination, the company requested to submit a proposal for use in the Cancer Drugs Fund. This included: 

   - a proposed confidential commercial access agreement (only for use within the Cancer Drugs Fund, and could not be applied in routine commissioning), 

   - an updated simulated treatment comparison including the latest CheckMate 275 data, 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 19 of 22 





<!-- page 20 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

- a revised economic analysis using the committee’s preferred assumptions (see section 3.17) that incorporates the updated simulated treatment comparison, and 

- a proposal for data collection from the CheckMate 275 and 

CheckMate 032 trials to address the clinical uncertainty. 

The committee noted that results of the simulated treatment comparison that includes the latest CheckMate 275 data were substantially different from previous analyses and considerably underestimate the overall survival of people who have taxane therapy compared with the observed data from CheckMate 275. It recalled that the latest CheckMate 275 data was not likely to have a substantial effect on the estimation of clinical effectiveness (see section 3.5). It concluded the latest results of the updated simulated treatment comparison were implausible and agreed that the revised eoncomic analyses incorporating these results could not be used in its decision-making. The committee reinforced the conclusion that any results from the simulated treatment comparison must be treated with caution. 

- 3.30 The committee recalled that it had concluded that the ongoing CheckMate 275 and CheckMate 032 trials could not address the key clinical uncertainties identified, because they provided no comparative data (see section 3.25). The committee considered that this was the main clinical uncertainty in this appraisal. It heard from the Cancer Drugs Fund clinical lead that collection of data from patients treated in the NHS would not improve the robustness of the simulated treatment comparison, because detailed prognostic data could not be collected. The committee fully explored whether the availability of any future data would help to reduce the clinical uncertainty. It agreed that it was unlikely that any data collected during a period in the Cancer Drugs Fund (including from research already underway) will be able to inform a subsequent review of a Cancer Drugs Fund recommendation. The committee concluded it could not recommend nivolumab for use within the Cancer Drugs Fund. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 20 of 22 





<!-- page 21 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

Therefore, nivolumab is not recommended, within its marketing authorisation, as an option for treating locally advanced unresectable or metastatic urothelial carcinoma in adults who have had platinumcontaining therapy. 

## **4 Review of guidance** 

4.1 The guidance on this technology will be considered for review 3 years after publication of the guidance. The guidance executive will decide whether the technology should be reviewed based on information gathered by NICE, and in consultation with consultees and commentators. 

Gary McVeigh 

Chair, appraisal committee 

## **5 Appraisal committee members and NICE project team** 

## _**Appraisal committee members**_ 

The 4 technology appraisal committees are standing advisory committees of NICE. This topic was considered by committee D. 

Committee members are asked to declare any interests in the technology to be appraised. If it is considered there is a conflict of interest, the member is excluded from participating further in that appraisal. 

The minutes of each appraisal committee meeting, which include the names of the members who attended and their declarations of interests, are posted on the NICE website. 

## _**NICE project team**_ 

Each technology appraisal is assigned to a team consisting of 1 or more health technology analysts (who act as technical leads for the appraisal), a technical adviser and a project manager. 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 21 of 22 Issue date: May 2018 




<!-- page 22 -->

**CONFIDENTIAL UNTIL PUBLISHED** 

## **Thomas Paling and Thomas Strong** 

Technical Leads 

## **Christian Griffiths** 

Technical Adviser 

## **Kate Moore** 

Project Manager 

ISBN: [to be added at publication] 

Final appraisal determination – Nivolumab for treating locally advanced unresectable or metastatic urothelial carcinoma after platinum-containing chemotherapy Page 22 of 22 Issue date: May 2018 


