Predictive Validity of High School Grade and other Characteristics on Students' University Careers using ROC Analysis

Laura Pagani and Chiara Seghieri

Abstract

The aim of this paper is to highlight the ability of high school grade and other students' characteristics (such as socio-economic status, high school of origin and so on) to predict success in university careers. To study the predictive capability we use data based on the records of almost 3200 freshmen who entered the University of Udine during the academic years 1992/93-1997/98. The structure of our data-base (students in high schools) suggests the use of logit multilevel models to relate a response variable (a binary variable that takes value 1 if the student gives at least one exam during the first academic year, 0 otherwise) to independent variables. Once the models are fitted, the estimated probabilities are used to construct an evaluation test to discriminate between two conditions: the student that can successfully enter the University (positive subject) and the student with lower abilities (negative subject). The evaluation test is obtained using the Receiver Operating Characteristic (ROC) analysis. If this test turns to be a valid measure of predicting success then it can be used to determine students' eligibility for admission (admission policy) or to guide students in their faculty choice.