ARTICLE IN PRESS

International Journal of Nursing Studies 40 (2003) 619–625

Validating an instrument for clinical supervision using an

expert panel
Kristiina Hyrk.asa,*, Kaija Appelqvist-Schmidlechnerb, Lea Oksac
a
Faculty of Nursing, University of Alberta, 3-114 Clinical Sciences Building, Edmonton, AB, Canada T6G 2G3
b
Department of Nursing Science, University of Tampere, FIN-33014 Tampere University, Finland
c
Tampere University Hospital, PO. Box 2000, FIN-33521 Tampere, Finland
Received 24 June 2002; received in revised form 9 December 2002; accepted 28 January 2003

Abstract

Use of research instruments standardised in English-speaking countries is commonplace in non-English speaking

countries. This article describes technical, linguistic and conceptual issued raised in the translation and validation of an
acceptable country-specific instrument.
The Manchester Clinical Supervision Scale was validated for use in clinical supervision in Finland using quantitative
and qualitative methods. The approach applied was triangulation. The focus of this paper, is to describe the item
validation process of the scale using an expert panel (n ¼ 11) by means of the content validity index and panel interview.
The process resulted in a country-specific 33-item instrument.
The study indicated that content validity index and panel discussion are easy, but reliable ways of demonstrating the
instrument’s content validity.
r 2003 Elsevier Science Ltd. All rights reserved.

Keywords: Instrument validation; Expert panel; Clinical supervision

1. Introduction December (2000). This purports that healthcare deci-

The Ministry of Social Affairs and Health (1983)
issued the first recommendation on clinical supervision
(CS) for health care professionals in the early 1980s in
Finland. After that, CS has been implemented without
co-ordination, mainly depending on administrators’ and
practitioners’ locally varying interests. The emphasis in
the evaluation of CS has mostly been based on the
supervisors’ own interest in receiving feedback on CS
and based on their supervisees’ subjective assessments.
In general, empirical research has been scarce, and
sharing the experiences has been uncommon. This has
led to the slow and variable development of CS (Hyrk.as
et al., 1999). The new recommendation on CS was issued
by the Ministry of Social Affairs and Health in
*Corresponding author. Tel.: +1-780-492-7112; fax: +1-
780-492-2551.
E-mail address: kristiina.hyrkas@ualberta.ca (K. Hyrk.as).
sion-makers should attend to a more effective planning,
co-ordination, implementation, development and eva-
luation of CS. However, the difficulty with the evalua-
tion so far has been the lack of suitable instruments. An
instrument suitable for general use could make it
possible for supervisors as well as administrators to
follow the trends of, and changes in, CS within the
organisation. From a wider perspective, this could mean
sharing and comparing the experiences and knowledge
concerning CS between the organisations. In other
words, a suitable instrument and the knowledge
generated from it could promote development and
improve the quality of CS in practice.
The purpose of this article is to examine the
translation and validation process of the Manchester
Clinical Supervision Scale (MCSS) in Finnish. The
instrument has been developed in the United Kingdom
for evaluating the efficacy of CS (Winstanely, 2000;
Winstanely, 2001). The focus of this paper is on

0020-7489/03/$ - see front matter r 2003 Elsevier Science Ltd. All rights reserved.
doi:10.1016/S0020-7489(03)00036-1
 ARTICLE IN PRESS
620 . et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas
describing the validation process of the instrument using
an expert panel. During the translation and validation
process the contribution of the expert panel turned out
to be a highly important phase for developing a suitable
instrument.
2. Literature review
2.1. Instrument development
The literature (e.g. Albaum et al., 1989; Guillemin
et al., 1993; Burns and Grove 1997) introduces two
approaches for developing and validating an instrument.
The first option is developing a new instrument. The
development may draw on empirical data collected to
explore the dimensions of the domain under investiga-
tion (Poss, 1999), or on theoretical knowledge of this
domain, after which the instrument developed is
reviewed by experts (Aminzadeh et al., 1999; Kim
et al., 2000; Oberle et al., 2000). The generation of a
brand new instrument is a time-consuming process, in
which the bulk of the effort is devoted to the
conceptualization of the instrument and the selection
and reduction of its items (Guillemin et al., 1993).
The second approach is to use existing instruments
and simply translate them into another language. There
are two options for translating an instrument into the
target language. The first is to directly translate
the instrument as accurately as possible while retaining
the structure and its items. The second and most
commonly used method is that of back-translation,
which enables the development of an instrument fit for
use in international comparative research (Hunt et al.,
1991). This approach may produce a linguistically, but
not necessarily a culturally appropriate instrument,
because it can be assumed that concepts and perceptions
differ across cultures (Guillemin et al., 1993; Poss, 1999;
Hunt et al., 1991). Factors may include language
barriers, different cultural meanings of a particular
construct and varied interpretations of an observed
behaviour based on cultural norms (see e.g. Guillemin
et al., 1993; Ferketich et al., 1993). The findings achieved
with an instrument with low cultural validity have also
very likely low practical value.
Hunt et al. (1991) criticize the back-translation
method. They point out that back translation may
mislead by suggesting a false lexical equivalence. It is
important to know the extent to which literally
equivalent words and phrases convey equivalence of
meaning in the two languages. In other words, the
translation of questionnaires from one language into
another raises not only linguistic, but also conceptual
and technical considerations (Hunt et al., 1991; Im et al.,
1999). To be successful, the second approach, mentioned
earlier, requires a systematic approach to the translation
and cross-cultural adaptation of the instrument (see e.g.
Guillemin et al., 1993). Thus, if the equivalence of items
is not sufficient, it may be necessary to re-phrase or even
re-develop new expressions for items that are appro-
priate for that culture. Co-construction of items with
members of the target culture can help ensure that all
relevant content is included, irrelevant content is
excluded and mutual understanding of the construct is
achieved (Mohr and Tulman, 2000; Leung and Arthur,
1999).
Modifications of instruments through the re-phrased
items for one culture result in instruments tailor-made to
each culture, although there is no longer directly
comparable translation (Flaherty et al., 1988). Guillemin
et al. (1993) point out that the cross-cultural adaptation
must be clearly distinguished from cross-cultural com-
parison since the two processes aim at totally different
goals. Adaptation is oriented towards measuring a
similar phenomenon in different cultures; it is essentially
the production of an equivalent instrument adapted to
another culture. Cross-cultural comparison refers to the
comparative study of a phenomenon across cultures in
order to identify differences attributable to culture
(Guillemin et al., 1993). Flaherty et al. (1988) distinguish
between emic and etic approaches in cross-cultural
research. The emic approach uses culturally defined
variables, and requires therefore an insider’s view of the
culture. Such investigations obtain a fine-grained under-
standing of concepts that are relevant to one culture but
that may not be relevant to other cultures. The emic
approach allows for descriptive comparison of specific
phenomena between two cultures and theories to
account for observed phenomena. It does not, however,
allow quantitative comparison across time or between
cultures. The etic approach applies, for example, the
concepts of a behaviour and techniques for measuring
that behaviour in one culture to a different culture.
2.2. Content validity
Before introducing an instrument to the target
culture, the content of an instrument developed in
another culture is important to validate in terms of
content. Content validity can be examined at the level of
the entire instrument and at that of individual items.
Content validity at the instrument level expresses how
well the instrument’s sub-scales represent the target or
content domain being measured. Content validity at the
item level expresses the extent to which each item
measures the target or content domain, which it is
supposed to measure. The importance and significance
of the content, that is, whether the translated instrument
measures important issues relevant to the domain is also
a central focus of assessment (Lynn, 1986; Per.al.a, 1995).
Content validity can be assessed through face validity
and expert assessment, which are expressed by means of
 ARTICLE IN PRESS
. et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas
the validity index and percentage agreement (e.g. Per.al.a,
1995).
Expert validity is thus a form of content validity,
which is demonstrated by asking experts to review the
content of the instrument. The aim is to eliminate totally
irrelevant items from the instrument (Voutilainen and
Liukkonen, 1995; Chaiyawat and Brown, 2000), but also
to re-phrase or supply new wording for items related to
the measured constructs where necessary (Hughes, 1998;
Aminzadeh et al., 1999). Expert assessment may be
quantitative and qualitative in nature. Expert assess-
ment enables the determination of the content validity
index when the survey is done quantitatively (Lynn,
1986). The experts rate the instrument item by item on a
rating scale. According to Lynn (1986), the minimum
number of experts is five. Ten experts would provide a
reliable determination of an instrument’s content
validity. If 80% of the experts agree that an item is
valid, it can be incorporated into the instrument
(Voutilainen and Liukkonen, 1995). Qualitative data
can be used for exploring content validity of an
instrument from another perspective and for providing
experts with the chance, for example, to suggest re-
phrasing or supply new items, if so required by the
results of a quantitative survey (Tilden et al., 1990).
Sequential and simultaneous triangulation (see e.g.
Morse, 1991) have been applied for instrument valida-
tion, when both quantitative and qualitative methods
have been used for validation (see. e.g. Friedemann and
Smith, 1997). The approach has been found useful for
instrument validation and cultural adaptation since it,
for example, improves the likelihood of getting a
comprehensive perspective for the domain of an instru-
ment under development in a particular culture. It also
provides an opportunity to explore and discover what
might undermine the validity and the biases of an
instrument (see e.g. Friedemann and Smith, 1997; Bruke
and Smith, 2000).
3. Study
3.1. Aim of the study
The aim of this paper is to describe how an instrument
for CS (MCSS) evaluation was validated for use in the
Finnish language and CS practice. The paper focuses on
the use of an expert panel in instrument validation.
3.2. The instrument
The MCSS is a 36-item questionnaire with a Likert-
type (1–5) scale comprising 7 sub-scales: trust and
rapport (TR), supervisor advice and support (AS),
improved care and skills (IC), importance and value of
CS (IV), finding time (FT), personal issues and reflection
621
(PR) and total score. Each item is rated on a rating scale
from strongly disagree (1=strongly disagree) to strongly
agree (5=strongly agree). The rating scores are sum-
mated by sub-scales. The higher the score on a sub-scale
is, the better the rating of CS.
3.3. Procedure
The translating and validating procedure was con-
ducted in four phases. In the first phase, the MCSS was
translated into Finnish using the method of back-
translation. The translation was conducted in collabora-
tion with the instrument developer and the authors of
this paper. A licensed translator translated the instru-
ment into Finnish. A native British language teacher in
the University language Centre performed the blind
back translation into English. The translations were
compared by both collaborative parties from Finland
and the UK, and by three experienced Finnish super-
visors. In the second phase a pilot sample (n ¼ 182) was
collected to test the translated instrument. An English
sample (n ¼ 463), supplied by the instrument developer,
was used as the comparison group. The equivalence of
the instrument was tested using various statistical tests
such as Cronbach’s alpha values and intra-class
correlation coefficient. The statistical tests showed
moderately high reliability between the translated and
original instrument. By contrast, the intra-class correla-
tion values revealed that the content equivalence needed
further investigation in the Finnish CS culture and
practice. The values showed that the variation in the
sub-scales ‘improved care and skills’, ‘finding time’ and
‘importance and value of CS’ was explained 35–42% by
the country. In the third phase, a sample (n ¼ 799) was
collected to further test the instrument. In the fourth
phase, a panel of experts (n ¼ 11) was assembled to asses
the face and construct validity of the instrument in the
Finnish CS culture and practice (Fig. 1). This article
focuses on the results of the expert review.
3.4. Panel of experts
The expert review and the panel discussion were
conducted at the Department of Nursing Science at the
University of Tampere with 11 experts in CS. The panel
consisted of seven specialized nurses, one ward manager
and three directors of nursing. The educational back-
ground for ten of the experts was a health care college
degree while one had a master’s degree in health
sciences. Seven panel members represented psychiatric
specialities and four, somatic specialities. Ten panel
members were acting as supervisors, with six of them
having over ten years experience with duties related to
CS. The experts were first asked to respond indepen-
dently to a questionnaire developed for the assessment
of the instrument. They were asked to rate the clarity,
 ARTICLE IN PRESS
622 . et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas

PHASE 1. PHASE 2. PHASE 3. PHASE 4.

Aim: translating the
instrument
Method: 1) translation,
2) back-translation, 3)
establishment of
equivalency
Aim: pilot testing the
translated instrument
Method: Sample of
nursing professionals
receiving clinical
supervision (n=182), a
British sample (n=463) as
the comparison group
Aim: testing the
instrument
Method: sample of
nursing professionals
receiving clinical
supervision (n=799)
Aim: validating the
instrument for Finnish
culture
Method: panel of experts
(n=11)

TIME: summer 1999 TIME: autumn 1999 TIME: autumn 2000 TIME: autumn 2001
Fig. 1. Instrument translation and validation process.

concreteness, centrality and importance of each item
using a 4-point rating scale (e.g. 1=not clear, 4=very
clear). They were also asked to assess the relevance of
the instrument’s sub-scales to assessing the efficacy of
CS. The content validity index (CVI) of each item was
calculated based on the experts’ ratings. The CVI score
was computed by summing the percentage agreement
scores of all items that were given a rating of ‘3’ or ‘4’ by
the experts.
number of raters giving a rating of ’3’ or ’4’
CVI ¼ : clear and 32 as concrete. Four items were rated as
total number of raters
A similar content validity index has been used in a
number of other studies (Hughes, 1998; Leung and
Arthur, 1999; Kyng.as et al., 2000; Lin et al., 2001). The
criterion for content validity of items was set. Items were
considered adequate if there was >79% agreement,
questionable if there was 70–79% agreement and
unacceptable if there was o69% agreement. The review
was followed by a panel discussion where the experts
examined the instrument item by item and discussed the
suitability of the questionnaire for the Finnish CS
culture and practice. Panel discussions were tape
recorded and transcribed, and notes made during
discussions were later transcribed. The approach applied
in this study was simultaneous triangulation (quant.+q-
ual.) in nature (see e.g. Morse 1991).
3.5. Ethical viewpoints of procedure
Permission for the translation and use of the MCSS
was obtained from the original author of the instrument.
Permission for the first (n ¼ 182) and for the second
samples (n ¼ 799) were obtained from the hospitals in
question. All participants were informed in a letter
attached to the questionnaire that their answers would
be used for research purposes, that their identity would
not be revealed at any stage, and that the researcher
would maintain the confidentiality of the information.
The voluntary nature of participation was emphasized.
To ensure anonymity, the experts’ background data are
presented as scantily as possible. Permission to tape
record interviews during the panel of experts was
obtained.
4. Results
Of the instrument’s 36 items, the experts rated 31 as
unacceptable in terms of clarity and concreteness. The
experts’ rating of the centrality and importance of the
items were highly critical. Nineteen (19) of the instru-
ment’s 36 items were rated as central, ten were rated as
questionable and seven as unacceptable for the evalua-
tion of the efficacy of CS. As for importance, 19 items
were rated as good, 12 questionable and five unaccep-
table. Item validity was considered adequate if an item
achieved 80% agreement among experts. Unacceptable
items were eliminated from the instrument after careful
consideration. Adequate items were accepted as such
and questionable items were further evaluated in the
expert discussion following the review. These items were
either re-worded or retained in their original form.
Among the experts, there was 100% agreement on the
relevance of all the instrument’s sub-scales, with the
exception the ‘finding time’ sub-scale. The sub-scale
‘Finding time’ was rated as questionable and this was
eliminated after long consideration. The justification for
this decision was based on the recommendation (The
Ministry of Social Affairs and Health, 1983) of
agreement for CS from all parities (i.e. supervisor,
supervisee and first-line manager) and the guarantee
from this agreement that time-resource must be in use
without exception. The panel also discussed the im-
portance of the relationship between CS and the quality
of care, which, according to the experts, received too
little attention after the elimination of unacceptable
 ARTICLE IN PRESS
. et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas
items. Experts felt it was important to re-phrase, re-
word items, or develop and add new items concerning
this domain to the instrument. Through this process, the
MCSS was reduced to a 33-item instrument.
Items not achieving 80% agreement were mostly
culture-bound. Cultural specificity was evident most of
all in items which supposed the supervisor’s availability
to the supervisee for consultation in his or her daily
work and the provision of concrete advice on nursing.
Such items were, for example, ‘If there is something I
don’t understand there is always someone to ask’, which
was not perceived to be an item related to CS in Finland.
The item ‘My supervisor is never available when needed’,
was also seen to show the difference between the CS
cultures in Finland and in England. In the Finnish CS
practice, the supervisor is very often a person who does
not work in the same setting with the supervisee, who
does not necessarily have a nursing background and is
thus not available to the supervisees for consultation in
their daily work. Cultural specificity was also evident in
the items in the sub-scale ‘Supervisor advice and
support’. Instead of communicating concrete and direct
advice, the Finnish CS culture emphasizes insightful
learning. Thus, for example, the item ‘My supervisor
offers me guidance with patient care’, was judged
questionable in terms of centrality and importance.
Items judged culture-bound were either eliminated
from the instrument or further revised to better suit the
Finnish CS culture and practice. For example, the item
‘CS does not solve personal issues’ was converted into
‘CS does not solve personal work related issues’. The
experts gave reasons for their suggestions by stating that
it is typical of the Finnish CS practice that it deals
specifically with work-related issues. The need for re-
wording and re-phrasing items suitable for the Finnish
CS culture and practice arose during the panel discus-
sion most of all in the sub-scales ‘Supervisor advice and
support’ and ‘Improved care and skills’. For example,
the following items were re-worded from the earlier
version of the MCSS and added to the instrument ‘My
supervisor makes me aware of my professional strengths
and development needs in skill areas’ (AS), ‘Clinical
supervision makes me aware of the importance of my
personal contribution to the wellbeing of the work
community’ (AS), ‘Clinical supervision motivates me to
acquire information on work-related issues’ (IC), and
‘Clinical supervision clarified my course of action in
practical work’ (IC) (Table 1).
5. Discussion and conclusion
Assessment of the instrument’s content validity
resulted in the following conclusions and measures:
items which were rated by experts as unimportant and
not central to measuring the efficacy of CS, were
623
eliminated or further revised to better suit the Finnish
CS culture and practice based on the results of the
inquiry and panel discussion. These included items,
which were rated as inadequate in terms of clarity and
concreteness. Since the instrument’s sub-scales, except
the ‘Finding time’ sub-scale, were rated as important for
the evaluation of CS, we wanted to retain the structure
of the instrument along with its sub-scales. As a
consequence, re-phrasing and wording of items was
needed to substitute the ones that were eliminated. This
was done in the panel discussion following the sugges-
tions of the experts and reviewing the earlier versions of
MCSS.
Kyng.as et al. (2000) point out that statistical values
are objective indicators of variables suitable for an
instrument, whereas the researcher’s decision about
suitable items based on content is subjective and
therefore unreliable. Simultaneous and sequential trian-
gulation has been used quite often for instrument
development and testing, since this approach enables
instrument validation using both statistical and qualita-
tive methods (see e.g. Friedemann and Smith, 1997;
Etter and Perneger, 1997). Quantitative and qualitative
research methods offer varied, but complementary
perspectives on a study and this is why triangulation is
seen as being advantageous. In our study the benefit was
achieved through integrating the different methods and
thus providing a more comprehensive approach and
view to the complex phenomenon of CS under study (see
Hyrk.as et al., 1999; Morse, 1991; Poss, 1999). This also
holds true for instrument development and validation in
this study. Results obtained through the inquiry were
possible to complement by the results of panel discus-
sion. The merit of the findings achieved during the panel
discussion focus especially on the instrument validation
and re-phrasing or -wording of items.
Determination of the content validity index through
expert ratings and inference based on the index value is a
reliable, simple and widely used method of demonstrat-
ing an instrument’s content validity. The results from
the expert review reported in this paper can be
considered reliable as a whole. Panel members were
purposefully selected because they were known to have
expertise in CS. They represented both somatic and
psychiatric specialities and operated in practical nursing
work as well as in hospital administration. This was
considered important because the intent was to develop
and validate the instrument for use by nursing profes-
sionals in different specialities. The instrument under
investigation consisted of 36 items, so it was assumed
that responding to a validation questionnaire was not
too extensive and heavy a task for the experts, which
could have weakened the reliability of the validation.
The number of the experts (n ¼ 11) also speaks in favour
of the high reliability of the results. Lynn (1986) has
recommended that five to ten experts be used for
 ARTICLE IN PRESS
624 . et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas

Table 1
Percentage agreement for items in the expert review

Sub- Item Clarity Concrete- Centrality Import-

scale ness ance

FT 1. Other work pressures interfere with CS sessions 90.9 90.9 90.9 90.9
FT 2. It is difficult to find the time for CS sessions 90.9 100.0 90.9 90.9
IV 3. CS sessions are not necessary/don’t solve anything 90.9 81.8 100.0 90.9
IV 4. Time spent on CS takes me away from my real work in the clinical area 100.0 100.0 63.6 72.7
PR 5. I can ‘underload’ during my CS session 100.0 90.9 100.0 100.0
FT 6. Fitting CS sessions in can lead to more pressure at work 81.8 90.9 81.8 63.6
FT 7. I find CS sessions time consuming 90.9 90.9 63.6 72.7
TR 8. My supervisor gives me support and encouragement 100.0 81.8 90.9 100.0
PR 9. CS does not solve personal issues 100.0 100.0 70.0 70.0
IV 10. CS sessions are intrusive 72.7 81.8 72.7 63.6
PR 11. CS gives me time to ‘reflect’ 90.9 81.8 100.0 100.0
PR 12. Work problems can be tackled constructively during CS sessions 81.8 81.8 90.9 81.8
PR 13. CS sessions facilitate reflective practice 90.9 90.9 100.0 100.0
AS 14. If there is something I don’t understand there is always someone to ask 18.2 36.4 18.1 9.1
TR 15. My supervisor offers an ‘unbiased’ opinion 90.9 81.8 100.0 100.0
TR 16. I can discuss sensitive issues y with my superior 100.0 81.8 100.0 100.0
PR 17. Having someone different to talk about personal issues was a great help 90.9 90.9 54.6 81.8
IC 18. My CS sessions are an important part of my work routine 90.9 90.9 81.8 90.9
TR 19. My supervisor is never available when needed 63.6 63.6 36.4 45.5
AS 20. I learn from my supervisor’s experiences 100.0 90.9 72.7 70.0
IV 21. It is important to make time for CS sessions 100.0 100.0 100.0 100.0
AS 22. My supervisor provides me with valuable advice 100.0 90.9 54.6 72.7
TR 23. My supervisor is very open with me 54.6 40.0 60.0 45.5
AS 24. Sessions with my supervisor widen my clinical knowledge base 100.0 90.9 72.7 81.8
IV 25. CS is unnecessary for experienced/established staff 100.0 100.0 72.7 72.7
TR 26. My supervisor puts me off asking about sensitive issues 100.0 90.9 100.0 81.8
TR 27. My supervisor acts in a superior manner during our sessions 100.0 81.8 90.9 72.7
IV 28. CS is for newly qualified/inexperienced staff only 100.0 100.0 72.7 72.7
IC 29. Clinical supervision makes me a better nurse 63.6 45.5 72.7 72.7
IC 30. Without CS the quality of patient care would deteriorate 100.0 90.9 72.7 72.7
IC 31. CS sessions motivate staff 100.0 90.9 72.7 72.7
IC 32. I feel less stressed after seeing my supervisor 100.0 81.8 90.0 81.8
IC 33. CS improves the quality of care I give to my patients 90.9 80.0 81.8 90.9
AS 34. I can widen my skill base during my CS sessions 100.0 81.8 100.0 100.0
AS 35. My supervisor offers me guidance with patient care 100.0 90.9 70.0 70.0
IC 36. I think receiving clinical supervisions improves the quality of care I give 90.9 80.0 90.9 90.9

instrument validation. The expert panel could have
included a larger number of end users of the instrument,
that is, supervisees, so as to gain certainty of issues
central to the efficacy of CS. On the other hand, all panel
members received CS, so it was assumed that they were
capable of examining the issue from the perspective of a
supervisee. Future instrument validation using a panel
of experts should include supervisees who are not acting
as supervisors.
The results of this study illustrate the impact of
cultural differences of the instrument used in the study.
Simple translation of an instrument from one language
into another is inappropriate especially when the
phenomenon under study is not universal in nature. A
translated instrument needs to be adapted in a culturally
relevant and comprehensible form, testing the instru-
ment more than one time both in cross-cultural studies
and in those aiming at the development of an instrument
for one culture. Triangulation, as illustrated in this
article, may greatly improve the validity of the instru-
ment. However, before the instrument validated in this
study can be introduced to the Finnish CS practice as an
evaluation tool, it must be re-piloted and tested using
statistical methods in order ensure that the developed
instrument is an easily manageable, quick and reliable
evaluation tool, as well as appropriate for Finnish CS
practice.
References
Albaum, G., Strandskov, J., Erickson, R. 1989. Translation in
questionnaire design for international and cross-cultural
 ARTICLE IN PRESS
. et al. / International Journal of Nursing Studies 40 (2003) 619–625
K. Hyrkas
research. Working papers 2/1989. Institute for International
Ekonomi og Virksomhedsledelse, Copenhagen.
Aminzadeh, F., Plotnikoff, R., Edwards, N., 1999. Develop-
ment and Evaluation of the Cane Use Cognitive Mediator
Instrument. Nursing Research 5, 269–275.
Burke, L., Smith, P., 2000. Developing an audit tool for health
promoting learning opportunities in clinical placements.
Nurse Education Today 20, 475–484.
Burns, N., Grove, S., 1997. The practice of nursing research.
Conduct, critique & utilization, 3rd Edition. Saunders
Company, Philadelphia.
Chaiyawat, W., Brown, J., 2000. Psychometric Properties of the
Thai versions of State-Trait anxiety inventory for children
and child medical fear scale. Research in Nursing and
Health 23, 406–414.
Etter, J.-F., Perneger, T., 1997. Validating a satisfaction
questionnaire using multiple approaches: a case study.
Social Science and Medicine 6, 879–885.
Ferketich, S., Phillips, L., Verran, J., 1993. Development and
administration of a survey instrument for cross-cultural
research. Research in Nursing and Health 16, 227–230.
Flaherty, J., Gaviria, M., Pathak, D., Mitchell, T., Wintrop, R.,
Richman, J., Birz, S., 1988. Developing Instruments for
Cross-cultural Psychiatric Research. The Journal of Ner-
vous and Mental Disease 5, 257–263.
Friedemann, M.-L., Smith, A., 1997. A triangulation approach
to testing a family instrument. Western Journal of Nursing
Research 3, 364–378.
Guillemin, F., Bonbardier, C., Beaton, D., 1993. Cross-cultural
adaptation of health-related quality of life measures:
literature review and proposed guidelines. Journal of
Clinical Epidemiology 12, 1417–1432.
Hughes, L., 1998. Development of an instrument to measure
caring peer group interactions. Journal of Nursing Educa-
tion 5, 202–207.
Hunt, S., Alonso, J., Bucquet, D., Niero, M., Wiklund, I.,
McKenna, S., 1991. Cross-cultural adaptation of health
measures. Health Policy 19, 33–44.
Hyrk.as, K., Koivula, M., Paunonen, M., 1999. Clinical
supervision in nursing in the 1990s—current state of
concepts, theory and research. Journal of Nursing Manage-
ment 7, 177–187.
Im, E.-O., Meleis, A., Lee, K., 1999. Cultural competence of
measurement scales of menopausal symptoms: use in
research among Korean woman. International Journal of
Nursing Studies 36, 455–463.
Kim, M., Hill, M., Bone, L., Levine, D., 2000. Development
and Testing of the Hill-Bone Compliance to High Blood
pressure Therapy Scale. Progress in Cardiovascular 15,
90–96.
Kyng.as, H., Skaar-Chandler, C., Duffy, M., 2000. The
development of an instrument to measure the compliance
625
of adolescents with a chronic disease. Journal of Advanced
Nursing 6, 1499–1506.
Leung, S., Arthur, D., 1999. The alcohol use disorders
identification test (AUDIT): validation of an instrument
for enhancing nursing practice in Hong Kong. International
Journal of Nursing Studies 37, 57–64.
Lin, L.-C., Chen, M.-Y., Chen, Y.-C., Portwood, M., 2001.
Psychometrics of a Chinese translation of the swallowing
questionnaire. Journal of Advanced Nursing 3, 296–303.
Lynn, M., 1986. Determination and quantification of content
validity. Nursing Research 6, 382–385.
Mohr, W., Tulman, L., 2000. Children exposed to violence:
measurement considerations within an ecological frame-
work. Advances in Nursing Science 1, 59–68.
Morse, J., 1991. Approaches to qualitative–quantitative meth-
odological triangulation. Nursing Research 1, 120–123.
Oberle, K., Singhal, N., Huber, J., Burgess, E., 2000.
Development of an Instrument to investigate parents’
perceptions of research with newborn babies. Nursing
Ethics 4, 327–338.
Per.al.a, M.-L., 1995. Potilaan hoidon laadun arviointi: Laatu-
mittarin (Qualpacs) validaatio (Quality assessment of
patient care: validation of a quality measure (Qualpacs)).
Stakes tutkimuksia 1995, 56. Gummerus Oy, Jyv.askyl.a. [In
Finnish]
Poss, J., 1999. Developing an instrument to study the
tuberculosis screening behaviors of Mexican migrant farm-
workers. Journal of Transcultural Nursing 4, 306–319.
.
The Ministry of Social Affairs and Health, 1983. Tyonohjausty-
aryhm.an muistio (Memorandum of the working group for
clinical supervision). [In Finnish]
The Ministry of Social Affairs and Health, 2000. Terveyden-
huollon ammatillisen valmiuksen kehitt.aminen tyoel. . am.as-
s.a—SUED-malli terveyspalvelujen laadunhallinnan
perustana. (Developing professional competence in prac-
tice—SUED model as a basis for quality management for
health care services). Terveydenhuollon ammattihenkiloiden.
neuvottelukunta. Sosiaali-ja terveys-ministerion . monisteita
2000:16. [In Finnish]
Tilden, V., Nelson, C., May, B., 1990. Use of Qualitative
Methods to Enhance Content Validity. Nursing Research 3,
172–175.
Voutilainen, P., Liukkonen, A., 1995. Senior Monitor—laadun
. validiteetin m.aa. ritt.aminen. (Se-
arviointimittarin sis.allon
nior Monitor—determining content validity for quality
assessment instrument). Hoitotiede 1, 51–56. [In Finnish]
Winstanley, J., 2000. Manchester clinical supervision scale.
Nursing Standard 14, 31–32.
Winstanley, J., 2001. Developing methods for evaluating
clinical supervision. In: Cutcliffe, J., Butterworth, T.,
Proctor, B. (Eds.), Fundamental Themes in Clinical Super-
vision. Routledge, London, pp. 210–224.