Prognostic Models for Oral Squamous Cell Carcinoma: Comparison
Please note this is a comparison between Version 2 by Lindsay Dong and Version 1 by Giuseppe Troiano.

An accurate prediction of cancer survival is very important for counseling, treatment planning, follow-up, and postoperative risk assessment in patients with Oral Squamous Cell Carcinoma (OSCC). There has been an increased interest in the development of clinical prognostic models and nomograms which are their graphic representation.

  • oral squamous cell carcinoma
  • prognosis
  • prognostic model
  • personalized medicine

1. Background

Head and Neck Cancer (HNC) is the sixth most common type of cancer across the world with nearly 550,000 new cases per year. Most of HNCs are diagnosed as Oral Squamous Cell Carcinomas (OSCC) and oral cancer ranks eighth among the most common causes of cancer-related deaths worldwide [1,2][1][2]. Both pharmacological and surgical protocols for OSCCs diagnosed in early stages are less aggressive and characterized by better outcomes, whilst in advanced stages, very high patients’ morbidity and poor clinical outcomes are expected [3]. Despite the increased knowledge and the encouraging scientific findings of the past 20 years on such diseases, the overall 5-year survival rate for OSCC is still below 50% [4].
Nowadays, the Tumor-Node-Metastasis (TNM) staging system is employed worldwide to predict tumor prognosis and to guide physicians towards the correct treatment choice, however, survival outcomes in patients classified within the same TNM stage class could be dramatically different, with discrepancies in therapy response and tumor management [5].
One of the main limitations of OSCC-related TNM system is its main focus on the anatomical extension of the disease. However, within each staging group, the prognosis can be modified by tumor-related factors, such as genetics, patient age, sex, race or comorbidities. For this reason, the need for a more “personalized” approach to the oncologic patient was underlined in the recent eighth edition of the American Joint Committee On Cancer (AJCC) staging system [6]. It is, therefore, necessary to investigate further prognostic factors to construct prognostic models to carry out a personalized prognosis evaluation [7,8][7][8].
Recently, there has been an increased interest in the development of clinical prognostic models and, in particular, in nomograms which are their graphic representation [9]. These are a set of mathematical algorithms that can be used to predict patient outcomes by incorporating multiple variables. Clinic-pathological and genetic variables are mainly incorporated in OSCC prognostic models, showing interesting evidence of their role in patients’ prognosis [10,11][10][11]. Purpose of these models is to estimate the probability or individual risk that a given condition, such as recurrence or death, will occur in a specific time by combining information from multiple prognostic factors of an individual [12].
Due to the recent interest in these new prognostic tools, and their potential important role in clinical practice, some guidelines have been defined for explanation and elaboration of clinically useful and correctly elaborated prognostic model. These Guidelines are reported in the Prognosis Research Strategy (PROGRESS) 3 and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) [7,13][7][13]. In 2016 the AJCC developed the acceptance criteria for inclusion of risk models for individualized prognosis in the practice of precision medicine in the systematic reviews [14]. In the same year, Debray et al. developed a guide for systematic reviews and meta-analyzes of the performance of prognostic models [15]. Additionally, the Prediction Model Risk of Bias Assessment Tool (PROBAST) was also developed to assess the risk of bias and the applicability of diagnostic and prognostic prediction model studies [16].

2. Prognostic Models for Oral Squamous Cell Carcinoma

Methodological characteristics of prognostic models developed are summarized on Table 31.

Table 31.
Methodological characteristics of prognostic models developed.
Authors and Year Internal Validation Modelling Method Handling of Missing Data Model Discrimination Model Calibration Model Presentation Handling of Continuous Predictors Non-Linearity Internal Validation C-Index External Validation C-Index
Bobdey 2016

Bobdey 2016

1000-time bootstrapping Multivariable Cox proportional hazards regression models and stepdown reduction method n/a C-statistic n/a Nomogram Mixed: Continuous; Categorical/dichotomous none 0.7263 none




1000-time bootstrapping Multivariable Cox proportional hazards regression models n/a C-statistic Calibration plot Nomogram Categorical/dichotomous n/a 0.709 0.691




1000-time bootstrapping Multivariable Cox proportional hazards regression models and stepdown reduction method Imputation C-statistic Calibration plot Nomogram Categorical/dichotomous Cubic splines 0.67 none




Combination of methods: 500-time bootstrapping; 5-fold cross-validation Multivariable Cox proportional hazards regression models n/a C-statistic Calibration plot Nomogram Mixed: Continuous; Categorical/dichotomous none 0.705 0.664




1000-time bootstrapping Multivariable Cox proportional hazards regression models and stepdown reduction method n/a C-statistic n/a Nomogram Categorical/dichotomous n/a 0.7266 0.740




1000-time bootstrapping Multivariable Cox proportional hazards regression models n/a AUC Calibration plot Nomogram Categorical/dichotomous Cubic splines 0.78 0.71

An accurate prediction of cancer survival is very important for counseling, treatment planning, follow-up and postoperative risk assessment in patients with OSCC [27][23]. Although the use of prognosis models is still relatively new for OSCC, these models are already widely used for other human diseases [28,29,30,31][24][25][26][27]. It is now well known that cancer-related outcomes are influenced by several factors that are not included in the TNM system. The vast majority of these factors has not been incorporated into the staging system because they may not predict outcome “independently” in multivariate prognosis models, however many of them may work in tandem and have varying degrees of influence on each other [32,33][28][29].

Six studies included correctly developed models according to the TRIPOD, all the included studies carried out internal validation of the model and four models were also externally validated [21,22,23,24,25,26][18][19][20][21][22][30]. The majority of models assessed OS in patients with squamous cell carcinoma of the tongue [22,24[19][21][30],26], two assessed all possible sites of tumor onset [21,23][18][20], and one model only assessed the buccal mucosa cancer [25][22]. All models rated OS at five years, except for Bobdey et al [25][22]. who only rated it at three years; furthermore, Li et al. and Sun et al., also evaluated OS at eight and three years respectively [21,23][18][20]. Among the clinical factors, those most included in the models are age, race, martial state, comorbidities and smoking; while among the histopathological ones the most investigated were T stage, N stage and M stage.

It is well known that the performance of a prognostic model is overestimated when it is just assessed in the patient sample that was used to build the model [34][31]. Internal validation provides a better estimate of model performance in new patients when done by adjusting overfitting, that is the difference between the accuracy of the apparent prediction and the accuracy of the prediction measured on an independent test set. Resampling techniques are a set of methods to provide an assessment of accuracy for the developed prognostic prediction models [35][32]. As an exception, Sun et al. [23][20] used a combined bootstrapping and cross-validation method, although all other studies used 1000-time bootstrapping as a resampling technique. Nevertheless, an evaluation of a model’s performance by using bootstrapping or cross-validation is not enough to overcome overfitting, such type of studies should also apply shrinkage, which is a method used adjust the regression coefficients [36,37][33][34].

Calibration reflects the agreement between the model’s predictions and the observed outcomes. It is preferably reported graphically, usually with a calibration plot [39][35]. Another key aspect of the characterization of a prognostic model is discrimination, that is, the ability of a forecasting model to differentiate between those who experience the outcome event or not [13]. The most used measure for discrimination is the Concordance Index (C-index), which reflects the probability that for any pair of individuals randomly, one with and one without the outcome, the model assigns a higher probability to the individual with the outcome [40][36]. For survival models, many c-indices have been proposed, so it is important to underline that, from our results, the most commonly used is the discrimination model proposed by Harrell [41][37]. In any case, discrimination can vary in a range from 0 to 1 and is considered good when higher than 0.5, considering that all the studies included in this systematic review presented a C-index at least higher than 0.6, all of them showed a good prognostic accuracy [42][38]. In addition, improvements in study design and analysis are crucial to allow evidence of more reliable prognostic factors that can be incorporated into new prognostic models, or to update existing models, to improve discrimination [43][39]. Another important finding was the almost total lack of handling of the missing data, except for Montero et al. [22][19] who carried out the multivariate imputations by chained equations (MICE) [44][40] before conducting multivariable regression statistical analysis [23][20]. The absence of a mention of the missing data leads to a so-called “full case analysis”. Including only participants with complete data, as well as being inefficient as it reduces the sample, can also lead to biased results due to a subsample [12].

External validation is preferable to internal validation for testing the transportability of a model since it is impossible for the population, or distribution of predictors, in an independent population to be the same as in the model development population [45][41]. Secondly, to improve the generalizability of a model, it should ideally be validated in different contexts with different population [46][42]. Furthermore, in the literature, there are currently no external validation by independent researchers of prognostic models for OS in patients with OSCC. A reliable model should be tested by independent researchers in different contexts to ensure the generalizability of prognostic models [15].

Most of the prognostic models in the literature describe the development of the model, a small number report external validation studies and currently, there are no studies considering clinical impact or utility [7]. Identifying accurate prognostic models and performing impact studies to investigate their influence on decision making, patient outcomes and costs is a fundamental component of stratified medicine because it contributes evidence at multiple stages in translation [47][43].
Multivariable Cox proportional hazards regression models were used to developing the models, as indicated for survival data [48][44]. All included prognostic models used nomogram as model presentation, yet none of the prognostic models reported the original mathematical regression formula. This turns out to be highly limiting, firstly because this presentation format is not a simplification of a developed model, but rather a graphical presentation of the original mathematical regression formula, and secondly, because recalibration, and updating of the original formula is necessary to perform validation [49][45]. Furthermore, it would be advisable to provide readers with the appropriate tools for the interpretation and application of the nomogram [30][26].

3. Conclusions

The following recommendations could be reported: (i) model development studies should weight for overfitting by carrying out internal validation (by resampling techniques such as bootstrapping) and using shrinkage techniques, (ii) model calibration and discrimination should always be examined, (iii) imputation techniques for missing data handling should always be applied, (iv) non-linearity of continuous predictors should be examined, (v) the complete equation of the prognostic model should always be reported to allow external validation and updating by independent research groups; (vi) prospective studies should be performed to reduce the risk of bias (vii) external validation in a new context and impact assessment on health outcomes and cost effectiveness of care should be carried out.


  1. Jiang, X.; Wu, J.; Wang, J.; Huang, R. Tobacco and oral squamous cell carcinoma: A review of carcinogenic pathways. Tob. Induc. Dis. 2019, 17, 1–9.
  2. Fitzmaurice, C.; Abate, D.; Abbasi, N.; Abbastabar, H.; Abd-Allah, F.; Abdel-Rahman, O.; Abdelalim, A.; Abdoli, A.; Abdollahpour, I.; Abdulle, A.S.M.; et al. Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life-Years for 29 Cancer Groups, 1990 to 2017. JAMA Oncol. 2019, 5, 1749.
  3. Vassiliou, L.V.; Acero, J.; Gulati, A.; Hölzle, F.; Hutchison, I.L.; Prabhu, S.; Testelin, S.; Wolff, K.D.; Kalavrezos, N. Management of the clinically N0 neck in early-stage oral squamous cell carcinoma (OSCC). An EACMFS position paper. J. Cranio-Maxillofac. Surg. 2020, 48, 711–718.
  4. Almangush, A.; Mäkitie, A.A.; Triantafyllou, A.; de Bree, R.; Strojan, P.; Rinaldo, A.; Hernandez-Prera, J.C.; Suárez, C.; Kowalski, L.P.; Ferlito, A.; et al. Staging and grading of oral squamous cell carcinoma: An update. Oral Oncol. 2020, 107, 104799.
  5. Rahman, N.; MacNeill, M.; Wallace, W.; Conn, B. Reframing Histological Risk Assessment of Oral Squamous Cell Carcinoma in the Era of UICC 8th Edition TNM Staging. Head Neck Pathol. 2021, 15, 202–211.
  6. Moeckelmann, N.; Ebrahimi, A.; Tou, Y.K.; Gupta, R.; Low, T.H.; Ashford, B.; Ch’ng, S.; Palme, C.E.; Clark, J.R. Prognostic implications of the 8th edition American Joint Committee on Cancer (AJCC) staging system in oral cavity squamous cell carcinoma. Oral Oncol. 2018, 85, 82–86.
  7. Steyerberg, E.W.; Moons, K.G.M.; van der Windt, D.A.; Hayden, J.A.; Perel, P.; Schroter, S.; Riley, R.D.; Hemingway, H.; Altman, D.G. Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research. PLoS Med. 2013, 10, e1001381.
  8. Riley, R.D.; Moons, K.G.M.; Snell, K.I.E.; Ensor, J.; Hooft, L.; Altman, D.G.; Hayden, J.; Collins, G.S.; Debray, T.P.A. A guide to systematic review and meta-analysis of prognostic factor studies. BMJ 2019, 364, k4597.
  9. Tham, T.; Machado, R.; Herman, S.W.; Kraus, D.; Costantino, P.; Roche, A. Personalized prognostication in head and neck cancer: A systematic review of nomograms according to the AJCC precision medicine core (PMC) criteria. Head Neck 2019, 41, 2811–2822.
  10. Mattavelli, D.; Lombardi, D.; Missale, F.; Calza, S.; Battocchio, S.; Paderno, A.; Bozzola, A.; Bossi, P.; Vermi, W.; Piazza, C.; et al. Prognostic nomograms in oral squamous cell carcinoma: The negative impact of low neutrophil to lymphocyte ratio. Front. Oncol. 2019, 9, 339.
  11. Troiano, G.; Caponio, V.C.A.; Botti, G.; Aquino, G.; Losito, N.S.; Pedicillo, M.C.; Zhurakivska, K.; Arena, C.; Ciavarella, D.; Mastrangelo, F.; et al. Immunohistochemical Analysis Revealed a Correlation between Musashi-2 and Cyclin-D1 Expression in Patients with Oral Squamous Cells Carcinoma. Int. J. Mol. Sci. 2019, 21, 121.
  12. Heus, P.; Damen, J.A.A.G.; Pajouheshnia, R.; Scholten, R.J.P.M.; Reitsma, J.B.; Collins, G.S.; Altman, D.G.; Moons, K.G.M.; Hooft, L. Uniformity in measuring adherence to reporting guidelines: The example of TRIPOD for assessing completeness of reporting of prediction model studies. BMJ Open 2019, 9, e025611.
  13. Moons, K.G.M.; Altman, D.G.; Reitsma, J.B.; Ioannidis, J.P.A.; Macaskill, P.; Steyerberg, E.W.; Vickers, A.J.; Ransohoff, D.F.; Collins, G.S. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration. Ann. Intern. Med. 2015, 162, W1–W73.
  14. Kattan, M.W.; Hess, K.R.; Amin, M.B.; Lu, Y.; Moons, K.G.; Gershenwald, J.E.; Gimotty, P.A.; Guinney, J.H.; Halabi, S.; Lazar, A.J.; et al. American Joint Committee on Cancer acceptance criteria for inclusion of risk models for individualized prognosis in the practice of precision medicine. CA Cancer J. Clin. 2016, 66, 370–374.
  15. Debray, T.P.A.; Damen, J.A.A.G.; Snell, K.I.E.; Ensor, J.; Hooft, L.; Reitsma, J.B.; Riley, R.D.; Moons, K.G.M. A guide to systematic review and meta-analysis of prediction model performance. BMJ 2017, 356, i6460.
  16. Wolff, R.F.; Moons, K.G.M.; Riley, R.D.; Whiting, P.F.; Westwood, M.; Collins, G.S.; Reitsma, J.B.; Kleijnen, J.; Mallett, S. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann. Intern. Med. 2019, 170, 51–58.
  17. Bobdey, S.; Balasubramaniam, G.; Mishra, P. Nomogram prediction for survival of patients with oral cavity squamous cell carcinoma. Head Neck 2016, 38, 1826–1831.
  18. Li, Y.; Zhao, Z.; Liu, X.; Ju, J.; Chai, J.; Ni, Q.; Ma, C.; Gao, T.; Sun, M. Nomograms to estimate long-term overall survival and tongue cancer-specific survival of patients with tongue squamous cell carcinoma. Cancer Med. 2017, 6, 1002–1013.
  19. Montero, P.H.; Yu, C.; Palmer, F.L.; Patel, P.D.; Ganly, I.; Shah, J.P.; Shaha, A.R.; Boyle, J.O.; Kraus, D.H.; Singh, B.; et al. Nomograms for preoperative prediction of prognosis in patients with oral cavity squamous cell carcinoma. Cancer 2014, 120, 214–221.
  20. Sun, W.; Cheng, M.; Zhuang, S.; Chen, H.; Yang, S.; Qiu, Z. Nomograms to predict survival of stage IV tongue squamous cell carcinoma after surgery. Medicine 2019, 98, e16206.
  21. Bobdey, S.; Mair, M.; Nair, S.; Nair, D.; Balasubramaniam, G.; Chaturvedi, P. A Nomogram based prognostic score that is superior to conventional TNM staging in predicting outcome of surgically treated T4 buccal mucosa cancer: Time to think beyond TNM. Oral Oncol. 2018, 81, 10–15.
  22. Chang, B.; He, W.; Ouyang, H.; Peng, J.; Shen, L.; Wang, A.; Wu, P. A prognostic nomogram incorporating depth of tumor invasion to predict long-term overall survival for tongue squamous cell carcinoma with R0 resection. J. Cancer 2018, 9, 2107–2115.
  23. Feng, Q.; May, M.T.; Ingle, S.; Lu, M.; Yang, Z.; Tang, J. Prognostic Models for Predicting Overall Survival in Patients with Primary Gastric Cancer: A Systematic Review. Biomed Res. Int. 2019, 2019, 5634598.
  24. Wesdorp, N.J.; van Goor, V.J.; Kemna, R.; Jansma, E.P.; van Waesberghe, J.H.T.M.; Swijnenburg, R.J.; Punt, C.J.A.; Huiskens, J.; Kazemier, G. Advanced image analytics predicting clinical outcomes in patients with colorectal liver metastases: A systematic review of the literature. Surg. Oncol. 2021, 38, 101578.
  25. Bradley, A.; Van Der Meer, R.; McKay, C.J. A systematic review of methodological quality of model development studies predicting prognostic outcome for resectable pancreatic cancer. BMJ Open 2019, 9, e027192.
  26. Kreuzberger, N.; Damen, J.A.; Trivella, M.; Estcourt, L.J.; Aldin, A.; Umlauff, L.; Vazquez-Montes, M.D.; Wolff, R.; Moons, K.G.; Monsef, I.; et al. Prognostic models for newly-diagnosed chronic lymphocytic leukaemia in adults: A systematic review and meta-analysis. Cochrane Database Syst. Rev. 2020.
  27. Wang, S.; Guan, X.; Ma, M.; Zhuang, M.; Ma, T.; Liu, Z.; Chen, H.; Jiang, Z.; Chen, Y.; Wang, G.; et al. Reconsidering the prognostic significance of tumour deposit count in the TNM staging system for colorectal cancer. Sci. Rep. 2020, 10, 89.
  28. Brierley, J.; O’Sullivan, B.; Asamura, H.; Byrd, D.; Huang, S.H.; Lee, A.; Piñeros, M.; Mason, M.; Moraes, F.Y.; Rösler, W.; et al. Global Consultation on Cancer Staging: Promoting consistent understanding and use. Nat. Rev. Clin. Oncol. 2019, 16, 763–771.
  29. Dijkland, S.A.; Retel Helmrich, I.R.A.; Steyerberg, E.W. Validation of prognostic models: Challenges and opportunities. J. Emerg. Crit. Care Med. 2018, 2, 91.
  30. Chang, W.C.; Chang, C.F.; Li, Y.H.; Yang, C.Y.; Su, R.Y.; Lin, C.K.; Chen, Y.W. A histopathological evaluation and potential prognostic implications of oral squamous cell carcinoma with adverse features. Oral Oncol. 2019, 95, 65–73.
  31. Bellou, V.; Belbasis, L.; Konstantinidis, A.K.; Tzoulaki, I.; Evangelou, E. Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: Systematic review and critical appraisal. BMJ 2019, 367, l5358.
  32. Iba, K.; Shinozaki, T.; Maruo, K.; Noma, H. Re-evaluation of the comparative effectiveness of bootstrap-based optimism correction methods in the development of multivariable clinical prediction models. BMC Med. Res. Methodol. 2021, 21, 9.
  33. Zhang, Z.; Cortese, G.; Combescure, C.; Marshall, R.; Lee, M.; Lim, H.; Haller, B. Overview of model validation for survival regression model with competing risks using melanoma study data. Ann. Transl. Med. 2018, 6, 325.
  34. Steyerberg, E.W.; Vickers, A.J.; Cook, N.R.; Gerds, T.; Gonen, M.; Obuchowski, N.; Pencina, M.J.; Kattan, M.W. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology 2010, 21, 128–138.
  35. Pencina, M.J.; D’Agostino, R.B. Evaluating discrimination of risk prediction models: The C statistic. J. Am. Med. Assoc. 2015, 314, 1063–1064.
  36. Harrell, F.E.; Lee, K.L.; Mark, D.B. Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors. In Tutorials in Biostatistics; John Wiley & Sons, Ltd.: Chichester, UK, 2005; Volume 1, pp. 223–249.
  37. Brentnall, A.R.; Cuzick, J. Use of the concordance index for predictors of censored survival data. Stat. Methods Med. Res. 2018, 27, 2359–2373.
  38. Riley, R.D.; Hayden, J.A.; Steyerberg, E.W.; Moons, K.G.M.; Abrams, K.; Kyzas, P.A.; Malats, N.; Briggs, A.; Schroter, S.; Altman, D.G.; et al. Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research. PLoS Med. 2013, 10, e1001380.
  39. van Buuren, S.; Groothuis-Oudshoorn, K. mice: Multivariate Imputation by Chained Equations in R. J. Stat. Softw. 2011, 45, 1–67.
  40. Ramspek, C.L.; Jager, K.J.; Dekker, F.W.; Zoccali, C.; van Diepen, M. External validation of prognostic models: What, why, how, when and where? Clin. Kidney J. 2021, 14, 49–58.
  41. Kundu, S.; Mazumdar, M.; Ferket, B. Impact of correlation of predictors on discrimination of risk models in development and external populations. BMC Med. Res. Methodol. 2017, 17, 63.
  42. Hingorani, A.D.; Van Der Windt, D.A.; Riley, R.D.; Abrams, K.; Moons, K.G.M.; Steyerberg, E.W.; Schroter, S.; Sauerbrei, W.; Altman, D.G.; Hemingway, H.; et al. Prognosis research strategy (PROGRESS) 4: Stratified medicine research. BMJ 2013, 346, e5793.
  43. Bradburn, M.J.; Clark, T.G.; Love, S.B.; Altman, D.G. Survival Analysis Part II: Multivariate data analysis—An introduction to concepts and methods. Br. J. Cancer 2003, 89, 431–436.
  44. Steyerberg, E.W.; Vergouwe, Y. Towards better clinical prediction models: Seven steps for development and an ABCD for validation. Eur. Heart J. 2014, 35, 1925–1931.
  45. Duggan, M.A.; Anderson, W.F.; Altekruse, S.; Penberthy, L.; Sherman, M.E. The Surveillance, Epidemiology, and End Results (SEER) Program and Pathology. Am. J. Surg. Pathol. 2016, 40, e94–e102.
Video Production Service