1.

VALIDITY, RELIABILITY & PRACTICALITY

Prof. Jonathan Magdalena
2. QUALITIES OF MEASUREMENT DEVICES
Validity
Does it measure what it is supposed to measure?
Reliability
How representative is the measurement?
Objectivity
Do independent scorers agree?
Practicality
Is it easy to construct, administer, score and interpret?
3. VALIDITY
Validity refers to whether or not a test measures what it intends to measure.
A test with high validity has items closely linked to the tests intended focus. A test with poor
validity does not measure the content and competencies it ought to.
4. VALIDITY - Kinds of Validity
Content: related to objectives and their sampling.
Construct: referring to the theory underlying the target.
Criterion: related to concrete criteria in the real world. It can be concurrent or predictive.
Concurrent: correlating high with another measure already validated.
Predictive: Capable of anticipating some later measure.
Face: related to the test overall appearance.
5. 1. CONTENT VALIDITY
Content validity refers to the connections between the test items and the subject-related
tasks. The test should evaluate only the content related to the field of study in a manner
sufficiently representative, relevant, and comprehensible.
6. 2. CONSTRUCT VALIDITY
It implies using the construct (concepts, ideas, notions) in accordance to the state of the art
in the field. Construct validity seeks agreement between updated subject-matter theories
and the specific measuring components of the test.
For example, a test of intelligence nowadays must include measures of multiple
intelligences, rather than just logical-mathematical and linguistic ability measures.
7. 3. CRITERION-RELATED VALIDITY
Also referred to as instrumental validity, it is used to demonstrate the accuracy of a
measure or procedure by comparing it with another process or method which has been
demonstrated to be valid.
For example, imagine a hands-on driving test has been proved to be an accurate test of
driving skills. A written test can be validated by using a criterion related strategy in which the
hands-on driving test is compared to it.
8. 4. CONCURRENT VALIDITY
Concurrent validity uses statistical methods of correlation to other measures.
Examinees who are known to be either masters or non-masters on the content measured
are identified before the test is administered. Once the tests have been scored, the
relationship between the examinees status as either masters or non-masters and their
performance (i.e., pass or fail) is estimated based on the test.
9. 5. PREDICTIVE VALIDITY
Predictive validity estimates the relationship of test scores to an examinee's future
performance as a master or non-master. Predictive validity considers the question, "How
well does the test predict examinees' future status as masters or non-masters?"
For this type of validity, the correlation that is computed is based on the test results and the
examinees later performance. This type of validity is especially useful for test purposes
such as selection or admissions.
10. 6. FACE VALIDITY
Face validity is determined by a review of the items and not through the use of statistical
analyses. Unlike content validity, face validity is not investigated through formal procedures.
Instead, anyone who looks over the test, including examinees, may develop an informal
opinion as to whether or not the test is measuring what it is supposed to measure.
11. QUALITIES OF MEASUREMENT DEVICES
Validity
Does it measure what it is supposed to measure?
Reliability
How representative is the measurement?
Objectivity
Do independent scorers agree?
Practicality
Is it easy to construct, administer, score and interpret?
12. RELIABILITY
Reliability is the extent to which an experiment, test, or any measuring procedure shows the
same result on repeated trials.
For researchers, four key types of reliability are:
13. RELIABILITY
Equivalency: related to the co-occurrence of two items.
Stability: related to time consistency.
Internal: related to the instruments.
Interrater: related to the examiners criterion.
14. 1. EQUIVALENCY RELIABILITY
Equivalency reliability is the extent to which two items measure identical concepts at an
identical level of difficulty. Equivalency reliability is determined by relating two sets of test
scores to one another to highlight the degree of relationship or association.
15. 2. STABILITY RELIABILITY
Stability reliability (sometimes called test, re-test reliability) is the agreement of measuring
instruments over time. To determine stability, a measure or test is repeated on the same
subjects at a future date. Results are compared and correlated with the initial test to give a
measure of stability. Instruments with a high stability reliability are thermometers,
compasses, measuring cups, etc.
16. 3. INTERNAL CONSISTENCY
Internal consistency is the extent to which tests or procedures assess the same
characteristic, skill or quality. It is a measure of the precision between the measuring
instruments used in a study. This type of reliability often helps researchers interpret data
and predict the value of scores and the limits of the relationship among variables.
17. 4. INTERRATER RELIABILITY
Interraterreliability is the extent to which two or more individuals (coders or raters) agree.
For example, when two or more teachers use a rating scale with which they are rating the
students oral responses in an interview (1 being most negative, 5 being most positive). If
one researcher gives a "1" to a student response, while another researcher gives a "5,"
obviously the interrater reliability would be inconsistent.
18. SOURCES OF ERROR
Examinee (is a human being)
Examiner (is a human being)
Examination (is designed by and for human beings)
19. RELATIONSHIP BETWEEN VALIDITY & RELIABILITY
Validity and reliability are closely related.
A test cannot be considered valid unless the measurements resulting from it are reliable.
Likewise, results from a test can be reliable and not necessarily valid.
20. BACKWASH EFFECT
Backwash (also known as washback) effect is related to the potentially positive and
negative effects of test design and content on the form and content of English language
training courseware.
21. THANKS