OPTIONAL LOGO HERE

Classroom Training Versus On-Site Support Supervision on Clinical Performance of Mid-level Practitioners
1. University of Washington, Seattle Washington; 2. University Research Company-Center for Human Services, Bethesda , Maryland ; 3. Infectious Diseases Institute, Kampala, Uganda; 4. Accordia Global Health Foundation, Washington, D.C.

OPTIONAL LOGO HERE

P. Imani1, I. Kirunda2, M. Mbonye, S. Naikoba, L. Mpanga-Sebuyira3 , K. Willis4, M. Weaver1

Background
Little has been done to test the effectiveness of disparate approaches to improving healthcare service delivery through human resource development. As the global health community seeks to expand training efforts to address Mid-level practitioners (MLP) (nurses and clinical officers), rigorous evaluation of which training approaches yield the best and most lasting results is needed to discern which types of training programs to implement or fund. The Integrated Infectious Disease Capacity-Building Evaluation (IDCAP) sought to measure the effect of the Integrated Management of Infectious Disease (IMID) training and on-site support (OSS) on clinical care for childhood illnesses.

Results (contd)
Table 2: Percent of patients for whom trainees performance was correct by clinical category: comparing Arm A and Arm B at baseline using all observations and observations for whom observers were present both at baseline and endline
Baseline with whole sample N=337 Category History Physical Exam Labs Diagnosis Treatment Education 63 (111) 47 (89) 70 (141) 79 (155) 78 (160) 83 (156) 63 (134) 48 (118) 81 (159) 64 (175) 78 (175) 78 (167) Baseline with only observations with balanced observers sample N=217 63 (58) 58 (49) 77 (88) 83 (96) 81 (99) 81 (96) 65 (78) 54 (72) 87 (102) 75 (115) 78 (115) 81 (108)

% correct in A (N) % correct in B (N) % correct in A (N) % correct in B (N)

Objectives
To evaluate the impact of IMID training and OSS on clinical performance of MLP. The primary hypothesis was that clinical performance would improve after IMID. The secondary hypotheses were: There would be no difference in performance between arms A and B at baseline; and, Performance would increase more among trainees at sites with OSS

Methods
Two MLP from each of the 36 facilities participated in IMID training. Eighteen of the 36 facilities were randomly assigned to participate in OSS. From January to March 2010, clinical faculty conducted 337 baseline assessments of care for children less than 15 years. From December 2010 to February 2011, they conducted 350 endline clinical assessments. Sixty six percent of trainees were clinical officers and others were nurses or midwives. Clinical faculty (58% Medical Officers and 42% Clinical Officers) underwent an orientation to clinical mentoring, IMID training, and assessment training prior to the clinical assessment. The majority of children on whom trainees were assessed were less than five years; 84% at baseline and 97% at endline. Performance was tested with a mixed design with pre/post and cluster randomized control elements. Clinical Assessment of Trainees A standardized assessment tool was used to assess trainees and record anonymous patient information on 6 aspects of patient care: History taking Physical examination Laboratory investigations ordered Diagnosis Treatment and Patient Education Clinical faculty observed trainees at each step of patient care indicating what the trainee had done correctly, incorrectly or missed. The clinical faculty conducted a history and physical examination after the trainee to complete missing information, and then continued the assessment. Each trainee was assessed on at least 5 patients prior to the IMID training (baseline assessment) and 5 after the interventions which were IMID training (for both Arms A and B) and OSS (for Arm A).

Percentage of history questions, systems examined, treatment prescribed correctly and components of patient education done were generally comparable in both groups at baseline. Arm B had slightly higher scores in labs ordered correctly while Arm A had better scores for diagnosis. Table 3. Adjusted odds ratios (confidence interval) across time periods and arms of performing task correctly Y = 0 + 1PhaseA + 2Post + 3Post*PhaseA + .. jXj + plus random effect for trainee
History Physical exam Patient Lab tests Diagnosis Treatment education ordered

Arm A to Arm 1.09 0.92 0.50 1.88 1.01 1.49 B at Baseline (0.85-1.39) (0.58-1.43) (0.25-1.00) (0.88-3.99) (0.70-1.47) (0.82-2.70)

Analysis
Dependent variables were generated by calculating the proportion of that the trainee got correct using what the examiners (observers) thought was appropriate as gold standard. Multivariate analysis using binomial regression was used to determine the odds ratio (OR). Other variables adjusted for in these models included trainee and observer cadre, complexity and number of the cases on which trainees were assessed, whether or not the site had participated in a Health Care Improvement program, and whether or not it had participated in a Baylor University facility-based pediatric HIV/AIDS program.

Pre/Post 1.41 2.32 0.75 1.13 1.07 1.13 effect of IMID (1.09-1.84) (1.34-4.01) (0.33-1.68) (0.59-2.17) (0.71-1.61) (0.56-2.26) Arm A to Arm B RCT effect 2.62 2.89 2.41 0.74 1.43 2.68 of OSS (1.63-4.21) (1.57-5.34) (0.86-6.77) (0.28-1.9) (0.82-2.52) (0.98-7.30)

IMID and OSS were associated with an improvement in history taking and physical exam when comparing performance at baseline and endline in Arm B.
Table 4. Adjusted odds ratios (confidence interval) across time periods and arms of performing task correctly Balanced Sample Y = 0 + 1PhaseA + 2Post + 3Post*PhaseA + .. jXj + plus random effect for trainee Patient Physical Lab tests Treatmen History Diagnosis educatio exam ordered t n Arm A to Arm B at Baseline
0.91 (0.6-1.25) 1.10 (0.651.87) 0.41 (0.16-1.09) 1.20 1.25 1.41 (0.51-2.84) (0.83-1.87) (0.67-3.00)

Results
Table 1 Patients characteristics Baseline Endline Phase A Phase B Phase A Phase B N=161 N=176 N=177 N=173 N (%) N (%) N (%) N (%) 12 (9) 131 (82) 27 (20) 91 (57) 48 (30) 43 (43) 9 (6) 99 (69) 10 (6) 58 (36) 29 (18) 24 (15) 12 (7) 74 (46) 21 (13) 13 (8) 8 (5) 154 (88) 35 (22) 130 (74) 52 (30) 55 (45) 4 (2) 97 (62) 19 (11) 65 (37) 37 (21) 30 (17) 4 (2) 83 (49) 33 (19) 26 (15) 12 (7) 151 (86) 6 (4) 122 (69) 44 (25) 131 (78) 8 (5) 64 (39) 20 (11) 72 (41) 33 (19) 19 (11) 9 (5) 61 (34) 15 (8) 22 (12) 10 (6) 143 (84) 4 (3) 114 (67) 48 (28) 139 (89) 3 (2) 63 (37) 22 (13) 66 (38) 23 (13) 26 (15) 8 (5) 62 (36) 19 (11) 12 (7)

Photo Credit: Charles Steinberg

Photo Credit: Tiera Kendle

Image 1. Patient examined by IDCAP trainee during mentoring session as part of OSS.

Image 2. Staff at an IDCAP Health Center attend an OSS session

Variable Patient symptoms Danger sign Fever HIV-status * Cough Diarrhea Immunization uptodate Ear discharge Other Diagnoses Anemia Cough (no pneumonia) Pneumonia Diarrhea acute Ear infection Malaria (uncomplicated) Malaria (complicated) Malnutrition

Pre/post effect 1.00 1.40 0.39 0.45 0.83 1.13 of IMID (0.74-1.35) (0.76-2.56) (0.12-1.21) (0.24-0.85) (0.54-1.29) (0.59-2.89) Arm A to Arm B RCT effect of 3.36 3.03 3.99 2.37 1.53 1.89 OSS (1.86-5.98) (1.50-6.12) (0.96-16.50) (0.81-6.95) (0.86-2.74) (0.58-6.16)

In the sample with observers who were present at baseline and endline, the effects of IMID were not statistically significant. OSS had a statistically significant effect on history taking and physical exam.

Conclusions
OSS is associated with improvements in clinical performance for treatment of children among MLP.

Acknowledgments
We would like to acknowledge the mobile team, the health facilities that agreed to participate in this assessment and most especially the patients that took part in this study. Contact email: mweaver@uw.edu
About IDCAP

*At baseline, reflects children and mothers (for children less than 5 years) whose HIV status was known regardless of results while at endline, it reflects those who were HIV positive. For trainee assessment, of interest was whether the trainee made an effort to ask or not.

Patient samples across arms at baseline and endline were comparable.

Accordias Integrated Infectious Disease Capacity-Building Evaluation (IDCAP) is a 3-year program funded by the Bill and Melinda Gates Foundation with the goal of evaluating the cost-effectiveness of methods to build capacity among mid-level health practitioners in subSaharan Africa for the treatment and prevention of infectious diseases. IDCAP will measure the impact of a novel package of classroom training, distance learning, and on-site support on individual competence and clinical practice, facility performance, and health outcomes in the surrounding communities.

Poster Number: 47.009

TEMPLATE DESIGN 2008

www.PosterPresentations.com