Professional Documents
Culture Documents
NMAT
Objective: to upgrade the selection of applicants to medical schools Description: an instrument that measures several abilities deemed necessary for success in medical training and practice
NMAT
Abilities measured: verbal comprehension inductive reasoning quantitative ability perceptual acuity knowledge and analytical proficiency in the sciences
Center for Educational Measurement, Inc.
Test Scores
8 subtest scores APT Composite SA Composite General Performance Score (GPS)
10
11
200475
Center for Educational Measurement, Inc.
VPLA
10
12
13
Future Direction
Development of shorter test forms that produce better ability estimates and efficiently predict academic performance in the medical course
14
15
Technically, predictive validity of NMAT is expressed as a bivariate or multivariate correlation coefficient of NMAT scores and college grades.
The higher this correlation coefficient, the more confidence can be placed on the NMAT as a valid predictor of academic performance.
16
The variables
Predictor variable
1. NMAT General Performance Score (NGPS)
17
School B (Visayas)
School C (Mindanao)
Center for Educational Measurement, Inc.
205
232
18
Data Analysis
Computation of weighted GPAs using the prescribed number of hours per academic subject as weights Computation of correlation coefficients between NMAT test scores and a criterion of medical school performance The accuracy of prediction is usually represented by the correlation coefficients. That is, the extent to which scores on the NMAT are correlated to success in medical college is the extent to which the test fulfills its predictive function.
Center for Educational Measurement, Inc.
19
Adjustment of computed correlation coefficients due to restriction of NGPS within the range above cut-off scores used by schools (which tends to severely decrease the correlation of NGPS with other variables.) Adjusted or otherwise, the values of correlation coefficients are always interpreted within the context in which it was computed. Coefficients as low as .30 are generally considered appropriate and valid for predictive purposes (Bobko, 1995).
Center for Educational Measurement, Inc.
20
Computation of the coefficient of determination, d, which indicates the proportion of variance in the FGPA or in the OGPA that is predicted by NGPS Derivation of prediction equation which estimates the FGPA or OGPA for a given NGPS Generating expectancy table which estimates the probability of a medical school applicant to attain certain FGPA or OGPA based on his/her NGPS
Center for Educational Measurement, Inc.
21
Results
TABLE 2. Pearson correlation coefficients School A Variable pair Restricted Adjusted NGPS x FGPA NGPS x OGPA r12 r13 -0.375 -0.373 -0.545 -0.542 School B
Restricted Adjusted
School C
Restricted Adjusted
-0.347 -0.346
-0.560 -0.558
-0.380 -0.296
-0.604 -0.496
Performance in NMAT is moderately correlated with academic performance in the first year of medical education as well as with performance over three or four years of academic subjects.
22
TABLE 3. Coefficients of determination School A School B Variable pair NGPS x FGPA NGPS x OGPA d12 d13
0.2966 0.2940 0.3131 0.3112
School C
0.3648 0.2461
Table 3 shows that around 30% of the total variation in academic performance can be explained by performance in the NMAT. In other words, the use of the NMAT GPS as predictor makes forecasting 30% to 36% better than mere random guessing.
23
School A:
School B:
School C:
Predicted FGPA Predicted OGPA Predicted FGPA Predicted OGPA Predicted FGPA Predicted OGPA
= = = = = =
-0.0025 (NGPS) + 3.8076 -0.0020 (NGPS) + 3.5354 -0.0026 (NGPS) + 3.8048 -0.0021 (NGPS) + 3.6110 -0.0023 (NGPS) + 3.7591 -0.0011 (NGPS) + 2.9049
The prediction equations can be used to estimate the first year GPA and the overall academic performance of a medical student applying for admission to medical School A, B, or C.
Center for Educational Measurement, Inc.
24
TABLE 5. Index of predictive efficiency School B Predicted variable School A FGPA OGPA 16.13% 15.98% 17.12% 17.00%
Table 5 shows that the prediction of academic performance in the three medical schools based on the derived prediction equations is about 13% to 20% better than prediction based on knowledge of the mean of the obtained FGPA or OGPA.
Center for Educational Measurement, Inc.
25
TABLE 7.2. Expectancy or probability of obtaining a particular FGPA at different ranges of NGPS in School A FGPA 1.75 & 1.76 2.01 2.26 2.51 2.76 NGPS Below -2.00 -2.25 -2.50 -2.75 -3.00 Below 501 501-550 551-600 601-650 651-700 Above 700 1% 2% 1% 11% 4% 4% 10% 15% 26% 44% 10% 19% 20% 29% 40% 22% 34% 34% 35% 27% 20% 11% 46% 35% 27% 22% 9% 11% 6% 9% 7% 5% 4%
Students who obtain an NMAT GPS between 651 and 700 have a 27 percent probability of attaining a first year GPA of 2.00 or higher in School A.
Center for Educational Measurement, Inc.
26
TABLE 7.3. Expectancy or probability of obtaining a particular OGPA at different ranges of NGPS in School A FGPA 1.75 & 1.76 2.01 2.26 2.51 2.76 NGPS Below -2.00 -2.25 -2.50 -2.75 -3.00 Below 501 501-550 551-600 601-650 651-700 Above 700 2% 1% 5% 8% 12% 35% 4% 16% 17% 26% 42% 50% 36% 38% 46% 36% 35% 30% 54% 43% 28% 26% 11% 15% 4% 2% 3% 2%
1% 2% 5%
Students who obtain an NMAT GPS between 651 and 700 have also 12 percent probability of attaining an overall GPA of 2.00 or higher in School A.
Center for Educational Measurement, Inc.
27
28
29
Recommendations
It should be noted that test scores are always subject to measurement errors. Hence, it is advised that other measures that assess other student characteristics (e.g., sociodemographics, motivation levels, etc.) may be considered before coming up with a final decision whether to accept or reject a candidate for admission to School A, B, or C. Test components should be reviewed and new test specifications should be considered to reflect subtests that highly correlate with academic performance
Center for Educational Measurement, Inc.
30
Explore the option of restoring a national cut-off score for quality assurance Address the need for more research studies on the implications of NMAT on quality assurance. This will entail full cooperation among medical schools and other agencies in providing the necessary database.
31