Chapter 18 CRITERIA FOR GOOD MEASUREMENT

Zarnigar Altaf

CRITERIA FOR GOOD MEASUREMENT

The first thing that we should consider here is the measuring instrument because the use of better instruments will ensure more accuracy in results, which in turn, will enhance the scientific quality of the research. What should be the characteristics of a good measurement? The tool that we are using for measurement should be an accurate indicator of what we are interested in measuring. In addition, it should be easy and efficient to use. Criteria for evaluating a measurement tool Validity Reliability Sensitivity
2

What is validity?
Validity is the ability of an instrument to measure what it is supposed to measure. Example
That is, when we ask a set of questions (i.e. develop a measuring instrument) with the hope that we are tapping the concept, how can we be reasonably certain that we are indeed measuring the concept we set out to do and not something else? We might research whether an educational program increases artistic ability amongst school children. validity is a measure of whether your research actually measures artistic ability.

Types Of Validity
Types of validity
content validity criterionrelated validity Concurrent validity Predictive validity: construct validity Convergent Validity Discriminant Validity

Pictorial Representation

Content validity
Content validity is a function of how well the dimensions and elements of a concept have been explained.
Example Do the questions on an exam accurately reflect what you have learned in the course, or were the exam questions sampled from only a sub-section of the material? A test to measure your knowledge of mathematics should not be limited to addition problems, nor should it include questions about French literature. It should cover the entire range of appropriate math problems you are trying to measure.

Face Validity Face validity refers to the extent to which a measure appears to measure what it is supposed to measure
Example Does rate of eating really reflect hunger? Does talking measure extroversion? Does GPA or GAT score really reflect intelligence? The above example looks valid on the base of face value but in actual it may not. so Just because a measure has face validity does not ensure that it is a valid measure
6

Criterion-Related Validity
The degree to which content on a test (predictor) correlates with performance on relevant criterion measures (concrete criterion in the "real" world?) Example The musical audition is valid to the extent that it distinguishes between singers who sings well versus those that do not. Two subtypes Concurrent validity: correlating high with another measure already validated . Example we create a new test to measure intelligence. For it to be concurrently valid, it should be highly associated with existing IQ tests (assuming the same definition of intelligence is used). It means that most people who score high on the old measure should also score high on the new one, and vice versa. The two measures may not be perfectly associated, but if they measure the same or a similar construct, it is logical for them to yield similar results. Predictive validity: Criterion validity whereby an indicator predicts future events that are logically related to a construct is called a predictive validity. Example For instance, we might theorize that a measure of math ability should be able to predict how well a person will do in an engineering-based profession.
7

Construct Validity
Construct validity refers to the degree to which a test or other measure assesses the underlying theoretical construct it is supposed to measure. Example An example could be a doctor testing the effectiveness of painkillers on chronic back sufferers. Every day, he asks the test subjects to rate their pain level on a scale of one to ten pain exists, we all know that, but it has to be measured subjectively. In this case, construct validity would test whether the doctor actually was measuring pain and not numbness, discomfort, anxiety or any other factor.

Two sub-types
Convergent validity Measures that Correlate with other measures that it should be related to. Discriminant validity Measures not correlate with measures that it should not correlate with .

Reliability
Means "repeatability" or "consistency". A measure is considered reliable if it would give us the same result over and over again Test-Retest Stability (Repeatability) Parallel Form

Reliability
Internal Consistency

Splitting halves

Inter Item consistency

9

Stability
The ability of the measure to remain the same over time. Two tests of stability are 1. Test-retest reliability 2. Parallel-form reliability. Test-retest Reliability: Used to assess the consistency of a measure from one time to another

Parallel-Form Reliability: Used to assess the consistency of the results of two tests constructed in the same way from the same content domain.
10

Internal Consistency of Measures

Internal consistency of measures is indicative of the homogeneity of the items in the measure that tap the construct. 1. Inter-item Consistency reliability 2. Split-Half reliability Inter-item Consistency reliability: This is a test of consistency of respondents answers to all the items in a measure. Split-Half reliability: Split half reliability reflects the correlations between two halves of an instrument.

11

Validity & Reliability

12

 Sensitivity:
Sensitivity refers to an instruments ability to accurately measure variability in stimuli or responses.

For Example:
Adding strongly agree, mildly agree, neither agree nor disagree, mildly disagree, and strongly disagree as categories increases a scales sensitivity.

Practicality:
Defined as economy, convenience, and interpretability. The scientific requirements of a project call for the measurement process to be reliable and valid, while the operational requirements call for it to be practical.

13