Name : Endah Woro Hapsari

Research Design

1. What is validity?
2. How to measure validity?
3. What formula will be used to measure validity?
4. How to keep validity of valid?
5. Give the model of measuring validity!

6. What is reliability?
7. How to measure reliability?
8. What formula will be used to measure reliability?
9. How to keep reliability of reliable?
10. Give the model of measuring reliability!

Answer

1. There are several experts’ definition regarding to validity. The definition of validity which
i have read. Firstly, it was Gronlund book. According to Gronlund (1977: 131) the
concept of validity, as used in testing, can be clarified further by noting the following
general points:
a. Validity refers to the interpretation of test result (not to the test itself);
b. Validity is inferred from available evidence (not measured);
c. Validity is specific to a particular use (selection, placement, evaluation of learning,
and so forth);
d. Validity is expressed by degree (for example: high, moderate, or low).
In addition, Gronlund (1977: 131) claimed that validity is divided into 3 basic types are:
content validity, criterion-related validities (predictive and concurrent), construct
validity. These 3 basic types of validity will be shown in the table below.

Basic Types of Validity

Type Question to be Answered
Content validity How adequately does the test content sample the
larger universe of situations it represents?
Criterion-related How well does test performance predict future
validities performance (predictive validity) or estimate
presents standing (concurrent validity) on some
other valued measure called a criterion?
Construct validity How well can test performance be explained in
terms of psychological attributes?
Adapted from Gronlund (1977: 131)

The second expert is Heaton. On his book, Heaton explains about criteria and types of
tests. According to Heaton (1990:159) validity of test is the extent to which it measures
what it is supposed measure and nothing else. All the test items both formal and informal
test which will use to test one’s achievement should be valid.
Heaton (1990:159) claims that there are four types of validity. First, face validity. Face
validity is used when the test items looks right to not only for the testers but also for the
testees. The second is content validity. Content validity which are stated by Heaton on his
book has similar definition on which Groundlund book. The third is construct validity.
Construct validity which are stated by Heaton also has similar definition to Gronlund.
Construct validity is emphasized on measuring certain specific characteristics in
accordance with theory of language behavior and learning. Fourth, empirical validity.
Empirical validity is also known as statistical validity. This validity will compare the
result of the test with the result of some criterion. Here are several criterions are used: an
existing test, the teacher’s ratings or any other such form of independent assessment given
at the same time, the subsequence performance of the testees on a certain task measured
 by some valid test, the teacher’s ratings or any other such form of independent assessment
given later.
The third expert is Fulcher. According to Fulcher (2007:3) validity refers to an evaluation
activity to find out the evidence that we would need to collect before conduct a test. In
this case, validity is an activity to find out whether the test items that will be used on a
test measure accurately what the researcher intended to measure or not. Meehl in Futcher
(2007:4) also describes that there are three types of validity – criterion ( predictive validity
and concurrent validity), content validity and construct validity. Third of those validity
types have the same definition with Grondlund and Heaton.
Based on the third experts’s definition about validity, it can conclude that validity is an
evaluation to checks whether the test items are valid or not before they are used to collect
the data on a research and to find out whether the test items which used by the reseracher
is extent to what he/she supposed to measure or not. In quantitative reserach, validity is
the extent to which any measuring instrument measures what it is intended to measure
(Thatcher, 2010:125). Validity of test items will help the reseracher to avoid serious
mistake on measuring testees’ ability. The validity of test items are commenly used in
quantitative reserach are content, criterion and construct validity. Third of them can help
the researcher to find out wheher the test items they will used valid or not. The reseracher
can use of of those validity types based on the reseracher need. Meanwhile according to
Creswell (2014:201) validity in qualitative reserach is used when a researcher uses certain
procedures to check for the accuracy of the research findings.

2. Based on the answer of question number one above, in quantitative research there are
three type of validity which can be used as measurement technique of validity. According
to Gronlund (1977: 131) and Fulcher (2007:3) validity can be measure by three types of
validity – content, criterion and construct validity. Meanwhile Fulcher (2007:3) states
four types of validity – face, content, construct and empirical. The researcher able to
choose the suitable types for check his/her test items.
In addition, when the researcher tries to find out the validity of his/ her test items based
on the content validity, the researcher should provide table specification. Fives (2013:1)
 states that the table of specification can help researcher map the amount of class time
spent on each objective with the cognitive level at which each objective was taught
thereby helping teachers to identify the types of items they need to include on their tests.
In contrast, validity in qualitative research can be measued by the researcher in order to
find out the quality, rigor and trustworthiness. Combining method is a common technique
to find out the validity in qualitative reserach by combinging qualitative and quantitative
method on the testing. The combining method ia known as triangualation. Triangulation
is typically a strategy (test) for improving the validity and reliability of research or
evaluation of findings (Golafshani, 2003). Creswell (2014:201-204, 210-211) claims that
there are seven strategies to check validity on qualitative reserech Triangulate –
Examining evidence of information from different sources of data. Member checking –
This is when the final report or specific descriptions from a research work is
taken back to the participants in order to determine if the participant feel that the results
are accurate. Comprehensive narration –This is the use of a rich, thick description to
convey findings. Bias Clarification –This is when the researcher clarifies the bias he/she
brings to the study. Presenting negative or discrepant information that runs contrary to
the information in the findings. Time – Spending prolonged time in the field in order to
gain an in- depth understanding of the
problem under study. Peer debriefing – Locating a peer debriefed to ask and review
questions about the study so that it
will not be only the researcher asking questions about the problem. External auditor –
Using an external auditor to review the entire project. This individual is not
familiar with the research problem and can be objective in the assessment of the research
work.

3. Here is the formula to measure validity.

 rxy =coefficient of correlation between x and y variable or validity of each item

N =the number of students/subject participating in the test

Σx = the sum of score in each item

Σx2 = the sum of the square score in each item

Σy = the sum of total score from each student

Σy2 = the sum of the square score form each student

Σxy = the sum of multiple of score from each student with the total score in each item.

After conducting the calculation of validity test for each items, the researcher have to
compare his/ her calculation result with significance level 5% . It is 0,374.

4. Based on validity definition, validity is an evaluation to checks whether the test items are
valid or not before they are used to collect the data on a research and to find out whether
the test items which used by the reseracher is extent to what he/she supposed to measure
or not. In quantitative reserach, validity is the extent to which any measuring instrument
measures what it is intended to measure (Thatcher, 2010:125). Validity of test items will
help the reseracher to avoid serious mistake on measuring testees’ ability. Considering on
the definition above, how to keep the validity keep valid become the important thing before
conducting a test on our reserach. In my opinion, the way how to keep validity keep valid
can be done by keeping focus on what we supposed to measure. It can be done by cheking
it based on the type of validity itselves. Commonly, quantitative reserach use three types
of validity to evaluate the the validity of test items. It can be choosen by the reseracher
based on what he/he need. In addition, in quantitaive research especially in content validity,
we have to provide table spesification. Fives (2013:1) states that the table of specification
 can help researcher map the amount of class time spent on each objective with the cognitive
level at which each objective was taught thereby helping teachers to identify the types of
items they need to include on their tests. After constructing the test items, the researcher
have to try it out. It is used to find out the evidence whether the test items valid or not. The
test result can be used by the researcher as an evidence to determine whether his/her test
items valid or not.

In contrast, on qualitative research to keep validity keep valid, the researcher have to use
mixed method to check whether the test instrument are valid or not. According to
(Newman, 2011) content validity provides (oral) indication of consensus by experts in the
content area at hand and try it out to audience then transformed into numeric values, scores
or rankings to support his/her argument on deciding whether test instruments are valid or
not.

Both in qualitative and quantitative research, when the researcher keep focus on what on
what he/she measure, it will help the resercher to keep the validity of valid his/her test
items or instruments. It also have to supported by whether expert’s opinion or validity
calculation result. Basicly validity is the product of research questions. The researcher
should not change his/her research question if the data that he/she want to collect are
similar.

5. Here is the model of measuring validity. Before conducting the calculation, the researcher
have to find out all the elements that needed where have mentioned on the formula. The
formula needs the number of students who particate in a test. The second is, the sum of the
square score in each item. The third, sum of the square score in each item. The fourth, sum
of total score from each student. The fifth, sum of the square score from each student. The
sixth steps is the sum of multiple of score from each student with the total score in each
item. Here is the example calculation accoding to (Nugrahaningsih, 2007)
 The result of her computaion of validity is 0,456 where the significant level 5% is 0,374.
After that, the reseracher have to compare each item computation result to the significant level
value.

6. According to Haris (1969:14) reliability is meant the stability of test scores. A test cannot

measure anything well unless it measures consistently. A test said to be reliable if the
 result is consistent. Brown (2004:20) stated that a reliable test is consistent and

dependable. An appropriate test in a research should have a good coefficient of reliability

to be tested to the students.

7. According to Heaton (1990:163) there are two ways to measure the reliability of a test

items. First by administering parallel forms of the test to the same group. This assumes

that two similar versions of a particular test can be constructed: such as tests must be

identical in the nature of their sampling, difficulty, length, rubrics, etc. Only after a full

statistical analysis of the tests and all the items contained in them can the tests safely be

regarded as paralel. If the correlation between the two test is high (i.e. if the results derived

from two tests correspond closely to each other), then the tests can be termed reliable.

The second way is by using split-half method. This method estimate a different kind of

reliability from that estimate by test/ re-test procedures. The split-half is based on the

principle that, if an accurate measuring instrument were broken into two equal parts, the

measurements obtained with one part would correspond exactly to those obtained with

the other. The test is devided into two and the corresponding scores obtained, the extant

to which they correlate with each other governing the reliability of the test as a whole.

8. Here is the reliability formula :

.

9. Researcher must be able to perform exactly the same experiment under the same
conditions and generate the same results.

10.
Bibliography
Brown, D. (2004). Language Assessment Principles and Classroom Practices. New York: Perason
Edication Press.
Creswell, R. (2014). Reserach design: qualitative, quatitative, and mixed methods approaches.
USA: Sage Publications.
Fives, H. &.-B. (2013). Practical Assessment, Reserach and Evaluation (Classroom Test
Construction: The Power of a Table Spesification). A peer-reviewed electronic journal. ,
1.
Fulcher, G. &. (2007). Language Testing and Assessment an advanced resource book. Canada:
Roultledge.
Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research. The
Qualitative Report, 8(4), 9. Retrieved from http://nsuworks.nova.edu/tqr/vol8/iss4/6
Haris, D. (1969). Teaching English as a Second Language . New York: McGraw-Hill, Inc.
Heaton, J. (1990). Writing English Language Tests. New York: Longman Group UK Limited.
Newman, I. e. (2011, November 2-5). Content Validity using Mixed Methods Approach.
Specifications Methodology1(1), p. 1. Retrieved November Wednesday, 8th , 2017, from
http://comm.eval.org/HigherLogic/System/DownloadDocumentFile.ashx?DocumentFile
Key=30758c14-2e59-4c94-af88-ecdf11b0dabc
Nugrahaningsih, N. (2007). The Use of Total Physical Response (TPR) METHOD IN English
Preposition Teaching. Semarang: English Department Language and Arts Faculty
Semarang State University.