Nurse Education in Practice (2005) 5, 97102

Nurse Education in Practice

www.elsevierhealth.com/journals/nepr

Introducing the health of the nation outcomes scales into undergraduate nurse preparation: opportunities for development a pilot project
Kate Simpsona,*, Tim Coombsb,1, Alan Averyc,2
a

Mental Health Administration: New England Area Health Service, P.O. Box 83, Tamworth, NSW 2340, Australia b NSW Institute of Psychiatry, 5 Fleet Street, North Parramatta, NSW 2151, Australia c School of Health, University of New England, Armidale, NSW 2350, Australia
Accepted 20 April 2004

KEYWORDS
Outcomes measures; Undergraduate nurse training; Clinical placement

Summary The New South Wales Mental Health Outcomes and Assessment Tools (MH-OAT) initiative introduces The Health of the Nation Outcomes Scales (HoNOS) to routine clinical practice. This is a signicant change in the way nurses document clinical information. This paper outlines a pilot project that introduced training in the use of the tool into an undergraduate nursing education program and subsequent clinical placement. The project aimed to introduce student nurses to the measure and highlighted the nature of contemporary and future Australian mental health nursing practice that will include the routine use of outcome measures. Twenty two of 25 third year student nurses completed a questionnaire that indicated that they did not nd the completion of the tool difcult and developed knowledge, skills and condence in completion of the measure during clinical placement. Ten student nurse preceptors indicated that practice in rating the measure as part of clinical placement was a useful exercise. The paper will identify opportunities for program development in relation to the content of nursing education, including the potential use of outcome measures by nurses and the use of the tool as a framework for preceptors supervision activities. Potential areas for future research will be outlined. c 2004 Elsevier Ltd. All rights reserved.

* Corresponding author. Tel.: +61-(02)67677945; fax: +61267664923. E-mail addresses: ksimpson@doh.health.nsw.gov.au; timcoombs@bigpond.com; aavery@metz.une.edu.au. 1 Tel.: +61-408601326. 2 Tel.: +61-267733641.

Background
The aim of nursing education is the preparation of competent and capable nursing professionals who

1471-5953/$ - see front matter c 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.nepr.2004.04.002

98 can meet the needs of contemporary working life (Munnukka et al., 2002). The changing work environment therefore must lead to changes in the process and content of nursing education. The Australian National Standards for Mental Health (Australian Health Ministers, 1996) are a change in mental health service provision in that they set benchmarks for contemporary practice. These changes include the need to monitor the outcomes of care for consumers. Standard 9.30 states that the Mental Health Services (MHS) routinely monitor health outcomes for individual consumers using a combination of accepted quantitative and qualitative methods, including measures of change in individual health status (Australian Health Ministers, 1996, p. 23). This information should not only be used to monitor the individual consumer change but also as standard 11.3.19 states, staff work loads in terms of number and mix of client cases, frequency of contact and outcomes of care (Australian Health Ministers, 1996, p. 33). In part to meet these standards, the introduction of the Health of the Nation Outcomes Scales (HoNOS) (Wing et al., 1998) (see Fig. 1) into routine clinical practice is an essential component of the information development strategy for Australian mental health services (Department of Health and Aged Care, 1999). The HoNOS has been identied as a measure of outcome and useful in routine clinical practice for monitoring the progress of consumers (Stedman et al., 1997). The HoNOS consists of 12 scales, each measuring a type of issue commonly identied by clients of mental health services (self-harm, substance use, cognitive problems, problems associated with hallucinations and delusions and depressed mood). Although the inter rater reliability of the instru-

K. Simpson et al. ment has been questioned (Bebbington et al., 1999), there is a general agreement that the instrument is valid and reliable and easy to use (Orrell et al., 1999) and should be introduced into routine clinical practice (Bebbington et al., 1999). The routine collection data using this measure and its use to monitor both individual and service outcomes are increasingly becoming an integral part of contemporary nursing practice. As White (1999) points out, nurses learn their professional identity by being involved in real nursing work. However, student nurses often face difculties with conicting expectations between the clinical area and education setting (Jarratt, 1983; Higgins, 1989; OBrien, 1995; Cross, 1998). Effective collaboration between the health care and tertiary education sectors is crucial if students are to achieve optimum learning outcomes (Dunn and Hansford, 1997; Clinton, 2001). As a result of the introduction of routine outcome measurement using the HoNOS, there is a need to integrate theoretical studies and practical training more closely. It is interesting to note that the authors during their literature review for this paper were unable to locate a mental health nursing needs analysis that called for training in standard measures or monitoring outcomes. One of the primary aims of this pilot project was to bring together the demands of contemporary clinical practice and undergraduate nurse preparation by integrating the training in the HoNOS into educational preparation and clinical practicum. Evaluation was aimed at identifying the development of knowledge, skills and condence as a result of training and practice and the degree of difculty students experienced in completing the measure during clinical practicum. The pilot project also aimed to evaluate the utility of the HoNOS

Overactive, aggressive, disruptive or agitated behaviour. Include behaviour due to any cause (drugs/alcohol/dementia/ psychosis/ depression) etc. Do not include bizarre behaviour, rated at Scale 6. Rate sexual behaviours at scale 8 (I) but rate any violence intimidation here. Score 9 If no Information available. 0 No problems of this kind during the period rated. 1 Some irritability, quarrels, restlessness, disruptive behaviour etc. 2 Includes occasional aggressive gestures, pushing, or provoking others; threats or verbal aggression; lesser damage to property (e.g. broken cup or window, cigarette burns); marked overactivity or agitation. 3 Physically aggressive to others or animals (short of rating 4); persistently threatening manner, more serious overactivity or destruction of property (e.g. broken doors, minor fire setting to bins/ashtrays etc). 4 At least one serious physical attack on others or on animals; destructive of property (e.g. dangerous fire setting); use of weapons; persistent serious intimidation.
Figure 1 Example Health of the Nation Outcomes Scale 1.

Introducing the health of the nation as a tool for preceptors which could guide and support clinical teaching around assessment and outcome measurement in routine clinical practice. Twenty ve nal year undergraduate nursing students studying their mental health units at a rural Australian University were involved in the learning project. These nursing students study mental health nursing in their nal year of study in a Bachelor of Nursing program. There are two units of study in mental health that students must complete during this nal year in order to meet the course requirements. The rst unit is offered in semester one (over a 10-week period) and the second unit is offered in semester two (over a 10-week period). Each unit comprises of 10 weeks of lectures and tutorials as well as four weeks of clinical experience of which two weeks comprise of mental health nursing practice experience in community and inpatient mental health settings. The pilot evaluation of the project was undertaken in the second semester acknowledging that students already had some mental health nursing experience. It is important to note that students undertaking this course will graduate as registered nurses and not as specialist mental health nurses. However, mental health is considered as an integral component of this course and to the nursing profession generally (Clinton, 2001). As part of the process of ensuring quality of teaching and learning for the undergraduate mental health component of this course, an evaluation of a mental health assessment tool recently introduced to clinical practice has been undertaken to explore the skills and competencies of the third year nursing students. Students are given instruction in the use of this tool in a tutorial workshop environment as well as during their mental health clinical placements, which occur in a mental health setting over an intensive two-week period. Students gave consent to participate and the evaluation was approved by the university unit and course coordinator as part of the university quality assurance program. Students were advised that refusal to participate would have no effect on the quality of their clinical placement or their course outcomes.

99 tion as all students were third year nursing students, who had been on previous mental health clinical placements. The questionnaire consisted of three statements generated by the authors to determine the students self-reported knowledge, skills and condence in the completion of the HoNOS. Responses to the statements were rated on a 6-point likert scale with 1 indicating strong disagreement and 6 indicating strong agreement. Students are forced with this scale to choose if they agree or disagree with the statements; 3 indicating disagreement and 4 indicating agreement. Following the completion of the questionnaire, training in the use of the HoNOS was provided through a 2-hour tutorial. This tutorial involved orientating students to the structure and rating rules of the HoNOS and provided them with an opportunity to rate the measure using pencil and paper vignettes specically developed for HoNOS training (MH-OAT, 2001). During their two-week clinical placement, students participated in a mental health assessment and used this assessment to rate the HoNOS. These ratings were then discussed with the preceptor. Prior to student clinical placement, all preceptors were individually contacted and asked to provide assistance to their student in completing the HoNOS and to encourage discussion regarding the use of the measure for assessment and outcome measurement. All preceptors had completed area based MH-OAT training that included skills development in completion of the HoNOS. At the end of their clinical placement (Post HoNOS exposure), students were again asked to rate the original three statements of knowledge, skills and condence, along with an additional statement that asked about the degree of difculty students experienced in completing the HoNOS as part of the clinical placement. Preceptors were also asked to rate two statements using a similar 6point likert scale to that used for students, which aimed to identify if preceptors found the students preparation helpful and completing the HoNOS a useful activity. Preceptors were also given the opportunity to make comments. For the purposes of statistical analysis, each questionnaire was coded so that pre- and post-test could be compared. Individual students or preceptors could not be identied.

Method
During a third year clinical tutorial prior to their clinical placement, students were given an overview of the activity and an initial orientation to the HoNOS. At this point (pre-HoNOS exposure), they were asked to complete a short questionnaire. This questionnaire collected no demographic informa-

Results
Twenty-two student nurses completed both preand post-test questionnaires and these will be the subject of analysis (see Table 1).

100
Table 1 Student pre- and post-mean HoNOS exposure knowledge, skills and condence scores N 22 1 2 3 4 (post-test only) Statement Pre M I have the necessary knowledge to rate the Health of the Nation Outcome Scales I have the necessary skills to rate the Health of the Nation Outcome Scales I am condent that I can rate the Health of the Nation Outcome Scales I found completing the Health of the Nations Outcome Scales a difcult part of my placement 3.45 3.41 3.32 SD 0.96 1.01 1.09

K. Simpson et al.

Post M 4.05 4.05 3.95 3.23 SD 1.05 1.00 1.17 1.37

In general, students pre-HoNOS exposure disagreed that they had the knowledge, skills and condence to complete the HoNOS. In general, students post-HoNOS exposure agreed that they had the knowledge, skills and condence to rate the HoNOS. These differences in average scores were all statistically signicant using paired t-tests at the p < 0:05. 36% of students indicated that completing the HoNOS as part of clinical placement was difcult. Ten preceptors rated the clinical placement activity post-HoNOS exposure (see Table 2). In general, preceptors agreed that it was helpful that students were aware of the HoNOS prior to their arrival on clinical placement. 86% of preceptors agreed that it was a useful activity to discuss the HoNOS with students in the context or measuring outcome.

Discussion
This project was a collaborative program between a rural area health service and a rural university in Australia offering undergraduate nursing education. The aim was to better inform nursing education providers of the changes in contemporary nursing practice that have resulted from the introduction of the HoNOS to measure mental health client outcomes in mental health services. Poor alignment between educational preparation and

work practice has been identied as an impediment to nurse preparation (Clinton, 2001). This project aimed to ensure alignment between preparation and practice. The results demonstrate that student nurses develop knowledge, skills and condence as a result of exposure to the HoNOS as part of undergraduate tutorials and practical eld experience. It has also been shown that students did not nd the activity particularly difcult to undertake. However, the self-rating of knowledge, skills and condence as evidenced by average scores of 4 on a 6-point likert scale indicates that these constructs were not strongly endorsed. This may be in part the result of the quality of the practical experience or the attitudes of preceptors. The considerable standard deviation of ratings, however, should be noted and indicates that some students felt they had the knowledge, skills and condence prior to HoNOS exposure and others continued to lack these qualities after exposure, with 36% of students participating and identifying the activity as difcult. This may be specic to the completion of the HoNOS or simply the result of a general lack of condence in the assessment of consumers with mental health problems on the part of these students. The results of this project should be approached with some caution. The number of students involved was small and limited to one rural geographic area. This makes the ability to generalize

Table 2 Preceptors mean post-clinical placement perspective of HoNOS exposure N 10 1 I found it helpful that nursing students knew about the HoNOS and its use as an outcome measure prior to their arrival on placement I found it a useful exercise with nursing students to review the HoNOS in the context of measuring outcome M 4.2 SD 1.47

4.3

1.63

Introducing the health of the nation these results to other situations problematic. There are a number of variables that may have inuenced the results including the amount of previous exposure to rating scales or mental health clinical experience students had prior to their clinical placement. Alternatively, variation in the qualities the preceptors brought to the activity may have inuenced results with one preceptor identifying their own need for additional training, as they were still learning. However, the majority of preceptors indicated that they agreed that it was useful for students to have an understanding of the HoNOS prior to clinical practicum and that it was useful to discuss HoNOS ratings with students. As one preceptor noted, it was interesting to identify the difference in interpretation, of the HoNOS and scoring between the student and the preceptor. Ritter et al. (1996) describe a model of clinical supervision for nurses undertaking short placements in mental health. This model uses genograms as a supervision tool, which encourages students to explore the biopsychosocial context of the client. The authors would argue that the HoNOS would provide a similar opportunity, exploring substance use, self-harm, relationships and social situations. As a tool then, this may be useful as an aid to the clinical supervision of more established practitioners. Similarly, as a forerunner to the delivery of care, Guttman (1996) identied the need for a simple tool to orientate students to the activity of reviewing the medical record. The HoNOS may be used as such a tool, with students being asked to review the client medical record and using the information gathered to rate the HoNOS. These ratings could then be shared with preceptors to explore areas requiring greater assessment, treatment options or specic interventions.

101 the effectiveness of service provision so that nursing students develop greater understanding of the use of these to plan and implement care. The need for strong links between education and service providers in the preparation of mental health nurses is essential in undergraduate preparation. The use of the HONOS in routine clinical practice presents an ideal opportunity to meet the needs of both these providers. Including the HoNOS in undergraduate training ensures that educational preparation reects contemporary nursing practice, while service providers will have access to nursing graduates prepared to work in a system that uses and measures outcomes. Although the results should be approached with caution, the current project has identied the potential of introducing the HoNOS into undergraduate nursing preparation. Its utility to support student learning and clinical supervision requires further research.

References
Australian Health Ministers, 1996. National Standards for Mental Health Services. Commonwealth Department of Health and Family Services (Mental Health Branch), Canberra. Bebbington P., Brugha T., Hill T., Marsden L., Window, S., 1999. Validation of the Health of the Nation Outcome Scales. British Journal of Psychiatry 174, 389394. Clinton, M., 2001. Scoping Study of the Australian Mental Health Nursing Workforce 1999. Commonwealth Department of Health and Aged Care, Canberra. Cross, W., 1998. Practical matters: The alienation of students in clinical placements. Papers presented at the 9th Annual Rozelle Hospital Winter Symposium, Rozelle Hospital, Sydney. Department of Health and Aged Care, 1999. Mental Health Information Development: National Information Priorities and Strategies under the Second National Mental Health Plan 19982003. Commonwealth Department of Health and Family Services, Canberra. Dunn S., Hansford, B., 1997. Undergraduate nursing students perceptions of the clinical learning environment. Journal of Advanced Nursing 25 (6), 12991306. Guttman, R., 1996. Orientation to the clients chart: a clinical learning tool. Nurse Educator 21 (5), 56. Higgins, L., 1989. Students perceptions of priorities in nurse education: research ndings and their curriculum implications. Higher Education Research and Development 8 (2), 117127. Jarratt, V., 1983. the time has come the Walrus said, to talk to many things. Nursing and Health Care (November), 498503. Munnukka T., Pukuri T., Linnainmaa P., Kilkku, N., 2002. Integration of theory and practice in learning mental health nursing. Journal of Psychiatric and Mental Health Nursing 9, 514. New South Wales Health Department, 2001. Mental Health Outcomes and Assessment Training (MH-OAT): Facilitators Manual. NSW Health Department, Sydney. OBrien, A., 1995. Measuring graduate attitudes to educational preparation for practice in mental health nursing.

Conclusion
Contemporary Australian nursing practice involves the routine measurement of consumer outcomes using the HoNOS. This pilot project demonstrates that students can develop knowledge, skills and condence in completing the HoNOS and that preceptors generally nd discussing the ratings with the students a useful activity. The preparation of undergraduates should therefore include training in completion of the HoNOS along with eld experience. It should also include material on the use of outcomes measures to monitor clinical change and

102
Australian & NewZealand Journal of Mental Health Nursing 14, 132142. Orrell M., Yard P., Handysides J., Schapira, R., 1999. Validity and reliability of the Health of the Nation Outcomes Scales in psychiatric patients in the community. British Journal of Psychiatry 174, 409412. Ritter S., Norman I., Rentoul L., Bodley, D., 1996. A model of clinical supervision for nurses undertaking short placements in mental health care settings. Journal of Clinical Nursing 5 (3), 149158.

K. Simpson et al.
Stedman T., Yellowlees P., Mellsop G., Clarke R., Drake, S., 1997. Measuring Consumer Outcomes in Mental Health. Department of Health and Family Services, Canberra, ACT. White, J., 1999. The impact of clinical experiences during preregistration diploma in nursing courses on initial career choice. Journal of Nursing Management 73 (3), 157165. Wing J., Beevor A., Curtis R., Park S., Hadden S., Burns, A., 1998. Health of the Nation Outcome Scales (HoNOS). British Journal of Psychiatry 172, 1118.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.