5/22/2018
Introduction
RESEARCH TOOL
Presented by
Josfeena Bashir
Ph.D. Scholar
DEFINITION
 It is an instrument or machine that aids in
accomplishing a task It is a testing device for
measuring a given event, such as , a
questionnaire, an interview or a set of guidelines
or checklist for observation
Characteristics of a Good Research
tool
 The Instrument must be valid and reliable.
 It must be based upon the Conceptual
framework.
 It must gather data suitable for and relevant to the
research topic.
 It must gather data would test the hypotheses or
answer the questions under investigation.
 Tools are instruments used to collect information
for performance assessments, self- evaluations,
and external evaluations.
 Tools need to be strong enough to support what
the evaluations find during research.
 Depending on the nature of the information to
be gathered, different instruments are used to
conduct the assessment forms for gathering data.
MEANING
 The instruments used for the purpose of data
collection, are measurable and observable for
data analysis & interpretation constructed by
researcher according to objectives
Cont….
 It should be free from all kinds of bias.
 It must contain clear and definite directions to
accomplish it.
 It must be accompanied by a good cover letter.
 It must be accompanied, if possible, by a letter of
recommendation from a sponsor.
1
 5/22/2018

Types of research tool-

Questionnaires
 “A questionnaire is a systematic compilation of questions
 1. Questionnaires that are submitted to a sampling of population from
which information is desired.”
 2. Interviews
--Barr, Davis & Johnson
 3. Schedules
 “In general, the word questionnaire refers to a device
 4. Observation Techniques for securing answers to questions by using a form
which the respondent fills in himself.”
 5. Rating Scales --W. J. Goode & K. Hall

Characteristics of a Good
Questionnaire
 It deals with an important or significant topic.
 Its significance is carefully stated on the
questionnaire itself or on its covering letter.
 It seeks only that data which cannot be obtained
from the resources like books, reports and
records.
 It is as short as possible, only long enough to get
the essential data.
Cont….
 It is attractive in appearance, nearly arranged and
clearly duplicated or printed.
 Directions are clear and complete, important
terms are clarified.
 The questions are objective, with no clues, hints
or suggestions.
 Questions are presented in a order from simple
to complex.

Cont…… Merits of Questionnaire Method

 Double negatives, adverbs and descriptive
 It’s very economical.
adjectives are avoided.
 It’s a time saving process.
 Double barred questions or putting two questions
in one question are also avoided.  It covers the research in wide area.
 The questions carry adequate number of  It’s very suitable for special type of responses.
alternatives.  It is most reliable in special cases.
 It is easy to tabulate, summarize and interpret.

2
 5/22/2018

Demerits of Questionnaire Method Cont…

 Chances of receiving incomplete response are
 Through this we get only limited responses.
more.
 Lack of personal contact.
 Sometimes answers may be illegible.
 Greater possibility of wrong answers.
 It may be useless in many problems.

The Interview Cont…..

 Interview is a two way method which permits an exchange of
ideas and information.
 “Interviewing is fundamentally a process of social
interaction.”
-W. J. Goode & P.K. Hatt
 The interview constitutes a social situation between two
persons, the psychological process involved requiring both
individuals mutually respond though the social research
purpose of the interview call for a varied response from the
two parties concerned.
--Vivien Palmar
 “The interview may be regarded as a systematic method
by which a person enters more or less imaginatively into
the inner life of a comparative stranger.”
--P.V. Young
In an interview a rapport is established between the interviewer
and the interviewee. Not only is physical distance
between them annihilated, the social and cultural barrier is
also removed; and a free mutual flow of ideas to and fro
takes place. Both create their respective impression upon
each other.

Difference between Interview and

Cont…. Questionnaire
 The interview brings them both on the same level and Questionnaire Method
an emotional attachment supervenes between them. In an 1. Data is gathered indirectly.
interview all formalities are laid down and the gate is 2. No face to face contact between 2. There is face to face contact
opened for delivering into the intellectuals, emotional and two.
subconscious stirrings of the interviewee. Thus here the
‘depth’ of subject (man) is gone to the very bottom of 3. Interviewer should have the
his emotional pool and may check his truthfulness of general knowledge of the topic.
responses. 4. Interviewee will hesitate to
write it.
5. We get written information
only.
Interview Method
1. Data is gathered directly.
between
interviewer and interviewee.
3. Skill full interviewer is needed.
4. Some confidential information
can also be obtained.
5. We get written and oral both
type of information.

3

Characteristics of an Interview:-
 The interviewer can probe into casual factors, determine
attitudes, discover the origin of problem.
 2. Its appropriate to deal with young children and illiterates
person.
 3. It can make cross questioning possible.
 4. It helps the investigator to gain an impression of the
person concerned.
 5. It can deal with delicate, confidential and even intimate
topics.
TYPES OF INTERVIEW –
 1. Personal Interview-
 2. Telephone Interview
 3. Focus Group Interview-
 4. Depth Interview-
 5. Projective Techniques
Cont…
 Generally an personal interview should go through the
following five/5 stages they are as follows.
 a. Rapport Building- Interviewer should increase
the receptiveness of the respondent, by making him
believe that his opinions are very useful to the
research, and is going to be a pleasure rather than an
ordeal(trial).
 b. Introduction -An introduction involves the
interviewer identifying himself by giving him his name,
purpose and sponsorship if any. An introductory letter
goes a long way in conveying the study’s legitimacy.
5/22/2018
Cont….
 6. It has flexibility.
 7. Sincerity, frankness, truthfulness and insight of the
interviewee can be better judged through cross questioning.
 8. It gives no chance for respondent to modify his earlier
answer.
 9. It is applicable in survey method, but it is also
applicable in historical, experimental, case studies and
clinical studies.
Personal Interview
 communication is between a face to face two way interview
between the interviewer and the respondents.
 Generally the personal interview is carried out in a planned
manner and is referred to as ‘structured interview’.
 This can be done in many forms e.g. door to door or as a
planned formal executive meeting.
 A personal interview involves a lot of preparation.
Stages of interview cont…
 c. Probing Probing is the technique of encouraging the
respondents to answer completely, freely relevantly.
 d. Recording -The interviewer can either write the
response at the time of interview or after the interview. In
certain cases, where the respondent allows for it, audio or
visual aids can be used to record answers
.
 e. Closing After the interview, interviewer should thank
the respondent and once again assure him about the worth of
his answers And also the confidentiality of the same.
4
 5/22/2018

Telephone Interview - . Focus Group Interview

 Telephone interview the information is collected from the
respondent by asking him questions on the phone is called as
telephone interview. The combination of telephone and
computer has made this method even more popular. It has
certain advantages and disadvantages.
 Focus group interview is an unstructured interview which
involves a moderator leading a discussion between a small
group of respondents on a specific topic.

. Depth Interview- Projective Techniques

 Depth interview is nondirective in nature where the
respondent is given freedom to answer within the boundaries
of the topic of interest.
 Projective Techniques involve the presentation of an
ambiguous, unstructured object, activity or person that a
respondent is asked to interpret and explain the colours of
dress In Projective Techniques, the respondents are asked to
interpret the behaviour of others/objects and this way they
indirectly reveal their own behaviour in the same situation.

Merits of Interview: Cont…

 Direct research.  Mutual encouragement is possible.

 Deep research  Supra-observation is possible.

 Knowledge of past and future.  Knowledge of historical and emotional causes.

 Knowledge of special features.  Examination of known data.

5

Disadvantage of Interview:
 May provides misleading information.
 Defects due to interviewee(low level of intelligence or
may be emotionally unbalanced)
 Result may be affected due to prejudices of interviewer.
 Result may be affected due to the difference in the
mental outlook of interviewee and interviewer.
 One sided and incomplete research.
Cont…
 . According to Thomas Carson Macormie, “The
schedule is nothing more than a list of questions
which, it seems necessary to test the hypothesis.”
 Thus schedule is a list of questions
formulated and presented with the specific
purpose of testing an assumption or hypothesis.
Types of schedule-
 Rating Schedules is a schedule used to obtain
opinions, preferences etc, respondents over statements
on the phenomenon studied. The schedule consists of
positive and negative statements of opinion on the
phenomenon.
 Documents Schedules are used to collect
data/information from recorded evidences and/or case
histories. Here the blanks, functional issues related
blanks and the like to be filled up from records and
documents are present.
5/22/2018
Schedule:
 When a researcher is using a set of questionnaires for
interview purpose it is known as schedule.
 “Schedule is the name usually applied to set of
questions, which are asked and filled by an interviewer
in a face to face situation with another.”
--W.J. Goode & P. K. Hatt
 By a schedule we cannot, however, obtain information
about many things at once. It is best suited to the study of a
single item thoroughly
Cont…
 In schedule method interview occupies a central
and plays a vital role. As a matter of fact
success in the use of schedule is largely
determined by the ability and tact of the
interviewer rather than by the quality of the
questions posed
.
 Because the interviewer himself poses the
questions and notes down the answers all by
himself, the quality of questions has not any
great significance.
Cont..
 Survey Schedules are like questionnaires.
 Observation Schedules are schedules used
when observational method of data collection is
used. These could be structured or unstructured
 interview schedules are used for collecting
data when interview method of communication
with the respondents is used.
6
 5/22/2018

Important Features of Schedule: Cont….

 The schedule is presented by the interviewer.  It aims at delimiting the subject.
The questions are asked and the answers are
noted down by him.  In the schedule the list of questions is pre planned
 The list of questions is a mere formal and noted down formally and the interviewer
document, it need not be attractive. is always armed with the formal document
detailing the questions.
 The schedule can be used in a very narrow sphere
of social research.
 Thus interviewer not to depend upon the
 It aids to delimit the scope of the study and to memory.
concentrate on the circumscribed elements
essential to the analysis.

Points to be kept in mind while

designing schedule; Cont…
 Interviewer should not frame long, complex,  Its questions should be simple, clear and relevant
defective questions. to topic

 Unrelated and unnecessary questions should not  Questions be suitable to respondent’s intelligence
be asked. level.

 Schedule should not contain personal and  Impersonal, indirect and unambiguous questions
upsetting questions. should be included in schedule.

Merits of Schedule: Cont…

 Higher percentage of responses.  It is possible to give human touch to schedule.

 Possible to observe personality factors.  Removal of doubts is possible because face to face

 Through interview personal contact is possible. interaction is there.

 It is possible to know about the defects of the

interviewee.

7

Observation Technique:
 This is most commonly used technique of evaluation
research.
 It is used for evaluating cognitive and non-cognitive aspects
of a person.
 It is used in evaluation performance, interests, attitudes,
values towards their life problems and situations.
 It is most useful technique for evaluating the behaviours of
children.
 It is technique of evaluation in which behaviour are observed
in a natural situations.
Cont…
 The cause- effect relationship and study of events in
original form, is known as observation. Observation seeks
to ascertain what people think and do by watching them
in action as they express themselves in various situations and
activities.
 Observation is recognized as the most direct means of
studying people when one is interested in their overt
behaviour. In questionnaires and interview people may
write answer as they think, they do but this is often
different from what they actually do. These restrictions
are missing in observation so observation is a more
natural way of gathering data.
Characteristics of Observation
 It serves a formulated research purpose.
 It is planned systematically rather than occurring
haphazardly.
 It is systematically recorded and related to
more general propositions.
 It is subjected to checks and controls with respect
to validity , reliability and precision.
5/22/2018
Defination
 “It is thorough study based on visual observation. Under
this technique group behaviours and social institutions
problems are evaluated.”
--C. Y. Younge
 “Observation employs relatively more visual and senses
than audio and vocal organs.”
-C.A. Mourse
Cont….
 Artificiality and formality of questionnaires and
interview is replaced by reality and informality in
observation. Data obtained through observation are more
real and true than the data collected by any other method. It
also plays a particular part in survey procedure.
cont….
 It is a direct technique to study an object, an
event or a problem.
 It is based mainly on visual –audio scene.
 It employs own experiences.
 It establishes cause-effect relationship.
8
 5/22/2018

Cont…. Type of observation –

 It is an objective technique of data collection.  Participant and Nonparticipant
 It is both objective and subjective evaluation
observation
technique.
 Structured and Unstructured Observation
 It is formal as well as informal technique.

 It is quantitative as well as qualitative  Controlled and Uncontrolled Observation

technique for data collection.

Advantages: Limitation:
•It is reliable and valid technique of
 It has a limited scope for its use because all the
collecting data and information.
events cannot be observed directly.
•We get first hand data through this method.
 It is subjective method.
•Record of observation is also available
immediately.  It is very time consuming process.

•It is simple, broad and comprehensive method.  Costly so energy consuming also.
•It is an oldest technique of data collection .
and getting direct information.

Cont… Rating Scale:-

 Ratting is term applied to express opinion or
 Presence of observer influences the behaviour of the
judgment regarding some situation, object or
person i.e. subject becomes conscious. character. Opinions are usually expressed on a
scale of values; rating techniques are devices by
 In case covert behaviour, which can’t be which such judgments may be quantified.

observed, it is not useful.

 Observer should be trained and experienced

9

Defination
 “Rating is an essence and direct observation.”
--Ruth Strong
 “A rating scale ascertains the degree, intensity
and frequency of a variable.”
--Von Dallen
Cont…
 The special feature of rating scale is that the attitudes
are evaluated not on the basis of the opinions of the
subjects but on the basis of the opinions and judgments of
the experimenter himself.
 In rating scale data are collected by; Verbal behaviour,
facial expression, personal documents, clinical type
interview, projective techniques and immediate
experiences as emotions, thoughts and perceptions.
Numerical Scales--
 It consists of a sequence of defined numbers which is
supplied to the rater or the observer therater assigns to each
stimulus, to be rated, an appropriate number as defined
rated, or describes.
5/22/2018
Cont….
 Rating techniques are more commonly used in
scaling traits and attributes. A rating method is
a method by which one systematizes, the
expression of opinion concerning a trait. The
rating is done by parents, teachers, a board of
interviewers and judges and even by the self as
well.
Type-
 Numerical Scales
 Graphical Scales-
 Percentage Rating-
 Standard Scale
Graphical Scales--
 The scales are presented graphically in which
descriptive cues corresponding to the different
scale steps are given. In this scale, a straight line
is drawn vertically or horizontally with various
cues to help the ratter. The line may be
segmented in units or continuous.
10
 5/22/2018

Percentage Rating- Standard Scale

 This technique involves placing objects, persons,
etc among different specified percentage groups
or into different percentiles. For example: in
percentiles or quartiles in the highest 5 per cent
of the group, in the middle 25 percent of the
group, in the lowest 10 percent, etc.
 A standard scale is one in which the ratter is presented with
some standards with pre-established scale values. These
standards usually consist of objects of same kind.
 For example: Handwriting, Portrait-matching,, man-to-man
scale, etc.

Advantages Cont..
 Writing reports to parents.
 Supplementing other sources of under taking
 Filling out admission blanks for colleges.
about child.
 Finding out students’ needs.
 Stimulating effect upon the rates.
 Making recommendations to employers

 Easy to administer

Limitations: Cont…
 Difference in rating abilities.  Limits of self-rating.

 Difference in reliability as subjects for rating.  Over rating.

 Agreement among ratters of one type of contact  Limits of rating of specific qualities.
only.
 Limits of justifications
 Average superior than single.

 Impact of emotions.

11
 5/22/2018

DEVELOPING RESEARCH TOOL- DEVELOPING RESEARCH TOOL-

 Requirements
 Clearly define the construct of the Research
tool.
 Conduct a pilot study to test the Research tool.
 Determine the validity of the Research tool.
 Documentation
 Clear description of the construct
 Old and new versions of items
 Formulation why certain scorings were chosen
 Results of pilot testing
 Final version of the Research tool

 Executing researcher:
DEVELOPING RESEARCH TOOL-
 Define a clear construct;
 Responsibilities
 Project leaders:  Chose for appropriate measurement method;

 Decide whether a new Research tool is necessary to
develop;
 Inspect the defined construct by reading and adding
comments;
 Search, select and formulate clear items;
 Chose scoring system carefully;
 Perform pilot tests in a small group of the selected population and

 Evaluate each step in the development of the
instrument with the executing researcher;
 Ensure that pilot tests are executed.
 Ensure that the Research tool is carefully evaluated.
make sure to adapt the instrument wherever needed;
 Evaluate the instrument according to the ‘Evaluating instruments’
guidelines.

Step 1: Definition and elaboration of

Process- the construct intended to be measured
 The first step in instrument development is conceptualization,
 Step 1: Definition and elaboration of the construct
which involves defining the construct and the variables to be
intended to be measured measured.
 When the construct is not directly observable (latent variable), the
 Step 2: Choice of measurement method (e.g.
best choice is to develop a multi-item instrument (De Vet et al.
questionnaire/physical test) 2011). When the observable items are consequences of
(reflecting) the construct, this is called a reflective model.
 Step 3: Selecting and formulating items  When the observable items are determinants of the construct, this
 Step 4: Scoring issues
is called a formative model.
 When you are interested in a multidimensional construct, each
 Step 5: Pilot study dimension and its relation to the other dimensions should be
described.
 Step 6: Field-testing

12

Step 2: Choice of measurement
method
 Some constructs form an in dissoluble alliance with a Research
tool, e.g. body temperature is measured with a thermometer; and
a sphygmomanometer is usually used to assess blood pressure in
clinical practice.
 The options are therefore limited in these cases, but in other
situations more options exist.
 For example, physical functioning can be measured with a
performance test, observations, or with an interview or self-
report questionnaire.
 With a performance test for physical functioning, information is
obtained about what a person can do, while by interview or self-
report questionnaire information is obtained about what a person
perceives he/she can do.
Step 4: Scoring issues
 Many multi-item questionnaires contain 5-point item scales,
and therefore are ordinal scales. Often a total score of the
instrument is considered to be an interval scale, which makes
the instrument suitable for more statistical analyses. Several
questions are important to answer:
 How can you calculate (sub)scores? Add the items, use the
mean score of each item, or calculate Z-scores.
Step 5: Pilot study
 Be aware that the first version of the instrument you develop
will (probably) not be the final version. It is sensible to
(regularly) test your instrument in small groups of people. A
pilot test is intended to test the comprehensibility, relevance,
and acceptability and feasibility of your Research tool.
5/22/2018
Step 3: Selecting and formulating
items
 To get input for formulating items for a multi-item
questionnaire you could examine similar existing instruments
from the literature that measure a similar construct, e.g. for
different target population, and talk to experts (both
clinicians and patients) using in-depth interview techniques.
 In addition, you should pay careful attention to the
formulation of response options, instructions, and choosing
an appropriate recall period (Van den Brink & Mellenbergh,
1998).
Cont…
 Are all items equally important or will you use (implicit)
weights? Note that when an instrument has 3 subscales, with
5, 7, and 10 items respectively, the total score calculated as
the mean of the mean score of each subscale differs from the
total score calculated as the mean of all items.
 How will you deal with missing values? In case of many
missings (>5-10%) consider multiple imputation (Eekhout et
al., 2014).
Step 6: Field-testing
13

Standardization of research tool –
 Standardization or standardisation is the process
of implementing and developing technical
standards based on the consensus of different
parties that include firms, users, interest groups,
standards organizations and governments.
Standardization can help to maximize
compatibility, safety, repeatability, or quality
Cont…
 After the approval by the division council, the work of
drafting the tool is allotted to the existing technical
committee or sectional committee.
 The technical committee is to work within the framework of
the governing policies and procedures for the preparation of
the standard tool.
 The members of the committee are required to have good
technical knowledge about what is to be standardized of tool.
Cont…
 The final version of the tool is submitted to the division
council for approval and finally to the general council or its
chairman. Once approved by these offices, the tool gets the
status of a standard.
 The approved tool is then published by the secretariat and the
published tool is then released for use to the public.
5/22/2018
Process-
 The first step consists of emerging of a proposal for a new
tool or the revision or the amendment of an existing tool.
The proposal can emerge from a representative of any sector
of the research
 The directorate of the standardization organization
preliminarily examines the proposal to determine whether it
is consistent with the underlying principles for the
preparation of research tool.
 The division council of the standardization organization
decides to approve or reject the proposal for the preparation
of new tool.
Cont……
 The technical committee reviews tool extensively and then
passes it on to the secretariat for editing and wide
circulation.
 The tool is widely circulated. The aim of wide circulation is
to inform every interest in the country or abroad which may
be affected by the tool and to invite critical review and
comments.
 The comments on the tool are systematically examined by
the technical or sectional committee. In the light of
committee discussions, the final version of the tool is drawn
up by the secretariat incorporating the comments accepted
by the technical or sectional committee.
Reliability of tool.
 It refers to the consistency, stability and repeatability of
results i.e. the result of a researcher is considered reliable if
consistent results have been obtained in identical situations
but different circumstances
 The researcher is always interested in collecting data that are
reliable.The reliability of an instrument concerns its
consistency and stability. If research are using a thermometer
to measure body temperature, he would expect it to provide
the same reading each time it was placed in a constant
temperature water bath.
14

Cont…
 Regardless of the type of research, the reliability of the study
instrument(s) is always of concern. Reliability needs to be
determined whether the instrument is a mechanical device, a
written questionnaire, or a human observer. The degree of
reliability is usually determined by the use of correlation
procedures. A correlation coefficient is determined between
two sets of scores or between the ratings of two judges: The
higher the correlation coefficient, the more reliable is the
instrument or the judges. Correlation coefficients can range
between –1.00 and +1.00.
Cont….
 Correlation coefficients are frequently used to determine the
reliability of an instrument. However, when observers or
ratters are used in a study, the percentage or rate of
agreement may also be used to determine the reliability of
their observations or ratings.
 In general, the more items that an instrument contains, the
more reliable it will be. The likelihood of coming closer to
obtaining a true measurement increases as the sample of
items to measure a variable increases. If a test becomes too
long, however, subjects may get tired or bored. Be cautious
about the reliability of instruments
Types of reliability
 Stability,
 Equivalence, and
 Internal consistency.
5/22/2018
Cont…
 Correlation coefficients computed to test the reliability of an
instrument are expected to be positive correlations.
According to Polit and Beck (2008), it is risky to use an
instrument with reliability lower than .70. These authors
have cautioned researchers to check for reliability as a
routine step in all studies that involve observational tools,
self-report measures, or knowledge tests because of their
susceptibility to measurement errors.
Cont…
 Reliability is not a property of the instrument that, once
established, remains forever.
 Reliability must continually be assessed as the instrument is
used with different subjects and under different
environmental conditions.
 An instrument to measure patient autonomy might be highly
reliable when administered to patients while in their hospital
rooms, but very unreliable when administered to these same
patients while lying on a stretcher outside the operating
room waiting for surgery.
Stability Reliability
 The stability reliability of an instrument refers to its
consistency over time. A physiological instrument, such as a
thermometer, should be very stable and accurate. If a
thermometer were to be used in a study, it would need to be
checked for reliability before the study began and probably
again during the study (test-retest reliability).
 Questionnaires can also be checked for their stability. A
questionnaire might be administered to a group of people,
and, after a time, the instrument would again be
administered to the same people. If subjects’ responses were
almost identical both times, the instrument would be
determined to have high test-retest reliability.
15

Cont…
 If the scores were perfectly correlated, the correlation
coefficient (coefficient Reliability estimates are used to
evaluate of stability) would be 1.00.
 The interval between the two testing periods may vary from
a few days to several months or even longer.
 This period is a very important consideration when trying to
determine the stability of an instrument.
 The period should be long enough for the subjects to forget
their original answers on the questionnaire, but not long
enough that real changes may have occurred in the subjects’
responses.
Cont..
 When two forms of a test are used, both forms should
contain the same number of items, have the same level of
difficulty, and so forth.
 One form of the test is administered to a group of people;
the other form is administered either at the same time or
shortly thereafter to these same people.
 A correlation coefficient (coefficient of equivalence) is
obtained between the two forms.
Internal Consistency Reliability
 Internal consistency reliability, or scale homogeneity, addresses the
extent to which all items on an instrument measure the same
variable.
 This type of reliability is appropriate only when the instrument is
examining one concept or construct at a time.
 This type of reliability is concerned with the sample of items used
to measure the variable of interest.
 If an instrument is supposed to measure depression, all of the
items on the instrument must consistently measure depression. If
some items measure guilt, the instrument is not an internally
consistent tool.
 This type of reliability is of concern to nurse researchers because
of the emphasis on measuring concepts such as assertiveness,
autonomy, and self-esteem.
5/22/2018
Equivalence Reliability
 Equivalence reliability concerns the degree to which two
different forms of an instrument obtain the same results or
two or more observers using a single instrument obtain the
same results.
 Alternate forms reliability or parallel forms reliability are
terms used when two forms of the same instrument are
compared.
 Inter-ratter reliability or inter-observer reliability are terms
applied to the comparisons of ratters or observers using the
same instrument. This type of reliability is determined by the
degree to which two or more independent ratters or
observers are in agreement.
Cont….
 The higher the correlation, the more confidence the
researcher can have that the two forms of the test are
gathering the same information.
 Whenever two forms of an instrument can be developed, this
is the preferred means for assessing reliability.
 Researchers, however, may find it difficult to develop one
form of an instrument, much less two forms!
Cont…
 Before computers, internal consistency was tedious to
calculate.
 Today, it is a simple process, and accurate split-half
procedures have been developed.
 A common type of internal consistency procedure used today
is the coefficient alpha () or Cronbach’s alpha, which provides
an estimate of the reliability of all possible ways of dividing
an instrument into two halves.
 Think about that a minute. How many possible combinations
of two halves could be made from a 30-item questionnaire? A
lot!
16
 5/22/2018

VALIDITY OF THE INSTRUMENT Cont…

 The validity of an instrument concerns its ability to gather
the data that it is intended to gather.
 The content of the instrument is of prime importance in
validity testing.
 If an instrument is expected to measure assertiveness, does
it, in fact, measure assertiveness? It is not difficult to
determine that validity is the most important characteristic
of an instrument.
 The greater the validity of an instrument, the more confidence
you can have that the instrument will obtain data that will answer
the research questions or test the research hypotheses.
 Just as the reliability of an instrument does not remain constant,
neither does an instrument necessarily retain its level of validity
when used with other subjects or in other environmental settings.
 An instrument might accurately measure assertiveness in a group
of subjects from one cultural group. The same instrument might
actually measure authoritarianism in another cultural group
because assertiveness, to this group, means that a person is trying
to act as an authority figure.

Types of validity -
Cont…..
 When attempting to establish the reliability of an instrument, all
of the procedures are based on data obtained through using the
 Face,
instrument with a group of respondents.
 Conversely, some of the procedures for establishing the validity of
an instrument are not based on the administration of the  Content,
instrument to a group of respondents.
 Validity may be established through the use of a panel of experts
or an examination of the existing literature on the topic.  Criterion, and
 Statistical procedures, therefore, may not always be used in trying
to establish validity as they are when trying to establish reliability.
 When statistical procedures are used in trying to establish validity,  Construct.
they generally are correlation procedures.

Face Validity Content Validity

 An instrument is said to have face validity when a preliminary  Content validity is concerned with the scope or range of
examination shows that it is measuring what it is supposed to items used to measure the variable.
measure.
 In other words, are the number and type of items adequate to
 In other words, on the surface or the face of the instrument,
measure the concept or construct of interest? Is there an
it appears to be an adequate means of obtaining the data
adequate sampling of all the possible items that could be used
needed for the research project.
to secure the desired data?
 The face validity of an instrument can be examined through
the use of experts in the content area or through the use of  There are several methods of evaluating the content validity
individuals who have characteristics similar to those of the of an instrument.
potential subjects. Because of the subjective nature of face
validity, this type of validity is rarely used alone

17

Cont…
 The first method is accomplished by comparing the content
of the instrument with material available in the literature on
the topic. A determination can then be made of the adequacy
of the measurement tool in light of existing knowledge in the
content area.
 For example, if a new instrument were being developed to
measure the empathic levels of nurses in hospice settings, the
researcher would need to be familiar with the literature on
both empathy and the hospice setting.
Cont…
 A third method is used when knowledge tests are being
developed. The researcher develops a test blueprint designed
around the objectives for the content being taught and the
level of knowledge that is expected (e.g., retention, recall,
and synthesis).
 The actual degree of content validity is never established. An
instrument is said to possess some degree of validity that can
only be estimated.
Concurrent validity
 Concurrent validity compares an instrument’s ability to obtain a
measurement of subjects’ behaviour that is comparable to some other
criterion of that behaviour.
 For example, a researcher might want to develop a short instrument that
would help evaluate the suicidal potential of people when they call in to
a suicide crisis intervention centre.
 A short, easily administered interview instrument would be of great
help to the staff, but the researcher would want to be sure this
instrument was a valid diagnostic instrument to assess suicide potential.
 Responses received on the short instrument could be compared with
those received when using an already validated, but longer, suicide
assessment tool.
 If both instruments seem to be obtaining the essential information
necessary to make a decision about the suicide potential of a person, the
new instrument might be considered to have criterion validity.
5/22/2018
Cont…
 A second way to examine the content validity of an
instrument is through the use of a panel of experts, a group
of people who have expertise in a given subject area. These
experts are given copies of the instrument and the purpose
and objectives of the study. They then evaluate the
instrument, usually individually rather than in a group.
Comparisons are made between these evaluations, and the
researcher then determines if additions, deletions, or changes
need to be made.
Criterion Validity
 Criterion validity is concerned with the extent to which an
instrument corresponds to or is correlated with some
criterion measure of the variable of interest.
 Criterion validity assesses the ability of an instrument to
determine subjects’ responses at the present time or predict
subjects’ responses in the future.
 These two types of criterion validity are called –
 concurrent and
 predictive validity
Cont….
 The degree of validity would be determined through
correlation of the results of the two tests administered to a
number of people.
 The correlation coefficient must be at least .70 to consider
that the two instruments are obtaining similar data.
18

Predictive validity
 It is concerned with the ability of an instrument to predict
behaviour or responses of subjects in the future.
 If the predictive validity of an instrument is established, it
can be used with confidence to discriminate between people,
at the present time, in relation to their future behaviour.
 This would be a very valuable quality for an instrument to
possess.
 For example, a researcher might be interested in knowing if a
suicidal potential assessment tool would be useful in
predicting actual suicidal behaviour in the future
Cont…
 Many of the variables measured in research are labelled constructs.
 Nursing is concerned with constructs such as anxiety &
assertiveness.
 One method to measure construct validity is called the known-
groups procedure, in which the instrument under consideration is
administered to two groups of people whose responses are
expected to differ on the variable of interest.
 For example, if you were developing an instrument to measure
depression, the theory used to explain depression would indicate
the types of behaviour that would be expected in depressed
people.
 If the tool was administered to a group of supposedly depressed
subjects and to a group of supposedly happy subjects, you would
expect the two groups to score quite differently on the tool.
 If differences were not found, you might suspect that the
instrument was not really measuring depression.
Relationship between Reliability and
Validity
 Reliability and validity are closely associated.
 Both of these qualities are considered when selecting a
research instrument.
 Reliability is usually considered first because it is a necessary
condition for validity. An instrument cannot be valid unless it
is reliable.
 However, the reliability of an instrument tells nothing about
the degree of validity.
 In fact, an instrument can be very reliable and have low
validity.
5/22/2018
Construct Validity
 Of all of the types of validity, construct validity is the most
difficult to measure. Construct validity is concerned with the
degree to which an instrument measures the construct it is
supposed to measure.
 Construct validity involves the measurement of a variable
that is not directly observable, but rather is an abstract
concept derived from observable behaviour. Construct
validity is derived from the underlying theory that is used to
describe or explain the construct.
Cont…
 Another approach to construct validity is called factor
analysis, a method used to identify clusters of related items
on an instrument or scale.
 This type of procedure helps the researcher determine
whether the tool is measuring only one construct or several
constructs.
 Correlation procedures are used to determine if items cluster
together
Cont..
 In actuality, validity is often considered first in the
construction of an instrument.
 Face validity and content validity may be examined, and then
some type of reliability is considered.
 Next, another type of validity may be considered.
 The process is not always the same.
 The type of desired validity and the type of reliability are
decided, and then the procedures for establishing these
criteria for the instrument are determined.
19

Cont…..
 A word of caution about using the term established in regard
to reliability and validity of instruments: Strickland (1995),
in an editorial in the Journal of Nursing Measurement, stated
that reliability and validity cannot be established because
there is always an error component in measurement. She
wrote that it is more correct to use terms like “supported,”
“assessed,” or “prior evidence has shown”
Research Instrument critique
 Step I: Conceptualize the proposed research project
 Step II: Find an existing instrument for the
proposed study
 Step III: Critical assessment of the proposed
measurement instrument
 Step IV: Decision to select or non-select the data
collection instrument for the study of interest
Cont…
 The basic foundational elements include the study problem,
review of the relevant literature, proposed design and
methods; research objectives, questions and/or hypotheses;
and the proposed study variables and their measurement.
 Questions and statement prompts have been developed to
assist the researcher describe and discuss each of the key
foundational elements as they relate to a proposed study.
5/22/2018
CRITIQUE –
 “Systematic, unbiased, careful examination of all aspects of a
tool to judge the merits, limitations, meaning and
significance based on previous research experience and
knowledge of the topic”
- Burns, N. & Grove, S., 2005.
Step I: Conceptualize the proposed
research project
 Without knowledge of the foundational elements of a
proposed study, researchers would have great difficulty
locating and selecting an appropriate existing instrument to
evaluate for potential use.
 begins with the researcher describing and discussing each of
the basic elements of the proposed research project.
Step II: Find an existing instrument for
the proposed study
 Once the key elements of the proposed study have been conceptualized
and one or more proposed measurement approaches determined (e.g.,
physiological, questionnaire etc) the next step is for the researcher to
seek out and locate potential measurement instruments to critically
evaluate for use in their study.
 Therefore the purpose of Step II is to locate an existing instrument that
may align with the key elements of the proposed study identified and
summarized in Step I. To locate existing measurement instruments one
can begin by:
1. Searching computerized databases such as Medline or CINAHL;
2. Searching journals that are devoted specifically to measurement, e.g.,
Journal of Nursing Measurement;
3. Identifying publications in which relevant instruments are used and then
using citation indices to locate other publications that used them to access
the computer database, the Health and Psychological Instruments Online
20

Cont…
4. accessing the Health research websites - these website lists
many surveys (available without charge) that would be of
interest to nurse researchers
5. reviewing one or more of the many reference books that
contain published measurement instruments,
6. reviewing Dissertation Abstracts online; and
7. networking and communicating by word-of-mouth with other
researchers
 Once a measurement instrument for a proposed study has
been located, the next step is for the researcher to document
the basic information about the instrument such as the name
of the instrument and where the instrument can be located.
Cont….
 Thus, the critical assessment of this potential research
instrument must occur within the backdrop of the proposed
study articulated in Step I. To begin, instrument availability
and access must be explored followed by a discussion of the
instrument background, data ownership, variables previous
measured, sampling, measurement and analysis, and other
considerations.
5/22/2018
Step III: Critical assessment of the
proposed measurement instrument
 Once an instrument has been located and before a decision
can be made about the selection of the instrument for use in
a proposed study, the instrument must be critically assessed.
 The purpose of this critical assessment process is to
determine the strengths and weaknesses of the selected data
collection instrument and to ensure that the make-up of the
instrument aligns with the needs of the proposed study.
Step IV: Decision to select or non-select
the data collection instrument for the
study of interest
 Step IV summarizes the information gathered and assessed
during Steps I–III and assists the researcher with justifying
and making an informed decision to select or non-select the
data collection instrument for use in their proposed study.
21