Multilevel Modelling for the Evaluation of Hospitals Performances

Assessment of service delivery at the local level of government is not a new enterprise in clinical context, but linking the measures, or indicators, to program mission, setting performance targets and regularly reporting on the achievement of target levels of performance are new features in the performance measurement movement sweeping across healthcare systems all over the world. A performance measure is a quantitative representation of public health activities, measured in order to evaluate, and then improve, performances and services.



Methodologies & Results

It has become common to adopt a hierarchical model structure (i.e., a multilevel model that takes into account the grouped nature of data) when comparing the performance of healthcare providers. It is not immediately clear, however, how unusual providers, that is, any with particularly high or low rates, can be identified based on such a model. Outlier detection in provider profiling is a growing interest
topic in decisional processes related to healthcare regulation. In this context  instruments like Funnel Plots enable a suitable recogniction of  under/over performing hospitals in the quality of treatments provided to patients affected by the same pathology. Once these hospitals
have been detected, people in charge of health care planning may set up monitoring and investigation strategies to understand the causes of these occurrences.


Provider profiling is the process of evaluation of the performance of hospitals, doctors and other medical practitioners in order to increase the quality of medical care. Performance indicators for assessing quality in health-care contexts have drawn more and more attention over the last decade because they enable the measurement of several components of the health-care process, clinical outcomes and disease incidence.
The focus of the work carried out at MOX is modelling the outcomes of clinical structures in order to identify ‘similar behaviours’ among clinical structures. These models include variability between institutions (after adjusting for case-mix) and performance indicators are computed starting from data collected through clinical registries and administrative databases.
Clustering techniques based on parametric and nonparametric random effects models, fitted to data using both classical and Bayesian approaches are the core of this research area.



2009-2011 (Completed) –
Exploitation, integration and study of current and future health databases in Lombardia for Acute Myocardial Infarction



LOGO_Prometeo   2008 -today (Ongoing)
PROMETEO (PROgetto Milano Ecg Teletrasmessi ExtraOspedaliero)
Commissioned by 118 (National free toll number for emergencies) of Milan




  1. Guglielmi, A., Ieva, F., Paganoni, A.M., Quintana, F.A. (2016) A semiparametric Bayesian joint model for multiple mixed-type outcomes: an Application to Acute Myocardial Infarction. Advances in Data Analysis and Classification. doi: 10.1007/s11634-016-0273-7
  2. Guglielmi, A., Ieva, F., Paganoni, A.M., Ruggeri, F., Soriano, J. (2014). Semiparametric Bayesian modeling for the classification of patients with high observed survival probabilities. Journal of the Royal Statistical Society – Series C, 63 (1): 25-46
  3. Ieva, F., Paganoni, A.M. (2014) Detecting and Visualizing Outliers in Provider Profiling via Funnel Plots and Mixed Effect Models. Health Care Management Science, Special Issue IMA 2013. doi: 10.1007/s10729-013-9264-9
  4. Ieva, F., Marra, G., Paganoni, A.M., Radice, R. (2014) A semiparametric bivariate probit model for joint modeling of outcomes in STEMI patients. Computational and Mathematical Methods in Medicine.Vol. 2014, Article ID 240435, 7 pages
  5. Ekin, T., Ieva, F., Ruggeri, F., Soyer, R. (2013). Application of Bayesian Methods in Detection of Healthcare Fraud. Chemical Engineering Transaction, 33. To appear [Online]
  6. Azzimonti, L., Ieva, F., Paganoni, A.M. (2013) Nonlinear nonparametric mixed-effects models for unsupervised classification. Computational Statistics, 28:1549–1570
  7. Grieco, N., Corrada, E., Sesana, G., Ieva, F., Paganoni, A.M., Marzegalli, M. (2012). Mortality and ST resolution in patients admitted with STEMI: the MOMI survey of emergency service experience in a complex urban area. European Heart Journal: Acute Cardiovascular Care, 1(3), 192–199.
  8. Guglielmi, A., Ieva, F., Paganoni, A.M., Ruggeri, F (2013) Hospital clustering in the treatment of acute myocardial infarction patients via a Bayesian semiparametric approach. Statistical Models for Data Analysis. In: Studies in Classification, Data Analysis, and Knowledge Organization (eds: P. Giudici, S. Ingrassia, M. Vichi) Springer, pp. 141-149
  9. Grieco, N., Ieva, F., Paganoni, A.M. (2012). Performance assessment using mixed effects models: a case study on coronary patient care. IMA Journal of Management Mathematics, 23(2), 117-131
  10. Guglielmi, A., Ieva, F., Paganoni, A.M., Ruggeri, F. (2012). A Bayesian random-effects model for survival probabilities after Acute Myocardial Infarction. Chilean Journal of Statistics, 3(1): 1-15.
  11. Guglielmi, A., Ieva, F., Paganoni, A.M., Ruggeri, F. (2012). Process indicators and outcome measures in the treatment of Acute Myocardial Infarction patients. Statistical Methods in Healthcare (eds: F. Faltin, R. Kenett, F. Ruggeri), chap. 10, pp. 219-229. Wiley.
  12. Ieva, F., Paganoni, A.M. (2011) Process indicators for assessing quality of hospitals care: a case study on STEMI patients, JP Journal of Biostatistics, 6(1): 53-75
  13. Ieva, F., Paganoni, A.M. (2010). Multilevel models for clinical registers concerning STEMI patients in a complex urban reality: a statistical analysis of MOMI2 survey. Communications in Applied and Industrial Mathematics, 1(1), 128-147



Dr. Elena Prandoni, MD Thesis in Mathematical Engineering (2013)
Modelli Bayesiani Semiparametrici Multivariati per le Probabilità di Sopravvivenza in seguito ad Infarto Miocardico Acuto

Dr. Francesca Ieva, PhD Thesis in Mathematical Models and Methods for Engineering (2012)
Statistical Methods for Classification in Cardiovascular Healthcare

Dr. Laura Azzimonti, MD Thesis in Mathematical Engineering (2010)
Modelli a effetti misti: teoria e applicazioni a dati longitudinali in ambito biologico

Dr. Emanuele Giani, MD Thesis in Mathematical Engineering (2010)
Metodi grafici e inferenziali per l identificazione di outliers: il caso del processo di cura di patologie cardiovascolari

Dr. Francesca Ieva, MD Thesis in Mathematical Engineering (2008)
Modelli statistici per lo studio dei tempi di intervento nell infarto miocardico acuto 



  • S.Co.2011 – 7th Conference on Statistical Computation and Complex Systems. Palazzo Centrale del Bò, Università degli Studi di Padova, Padova (Italy). September 19-21, 2011
    Talk: A new Unsupervised Classification Algorithm for Nonlinear Nonparametric Mixed Effects Models
  • CLADAG2011 – 8th Scientific Meeting on CLAssification and Data Analysis Group of the Italian Statistical Society. University of Pavia, Pavia (Italy). September 7-9, 2011
    Talk: Hospital clustering in the treatment of acute myocardial infarction patients via a Bayesian nonparametric approach
  • GDRR 2011 – Second workshop on Games and Decisions in Risk and Reliability. Belgirate (Italy) May 19-21, 2011
    Talk: Provider Profiling for Supporting Decision Makers in Coronary Patient Care
  • YoungOR 17. New Business School, Jubilee Campus, University of Nottingham (UK). April 5-7, 2011
    Talk: Provider Profiling using Mixed Effect Models
  • SIS 2010 -XLV Scientific Meeting of the Italian Statistical Society 2010, Padua (Italy), June 16-18, 2010
    Talk: A hierarchical random-effects model for survival in patients with Acute Myocardial Infarction
  • ABS-2010, Applied Bayesian Statistics School on Bayesian Machine Learning with Biomedical Applications. Bolzano, Italy, June 10-15, 2010.
    Talk: A Bayesian random-effects model for survival probabilities after Acute Myocardial Infarction.
  • SIS 2008 – XLIV Scientific Meeting of the Italian Statistical Society 2008. Arcavacata di Rende, Cosenza. June 25-27, 2008
    Talk: Door to Balloon Time in Patients with ST-Segment Elevation Myocardial Infarction.