USING EXPERIMENTAL DESIGNS TO IMPROVE CLINICAL PROCESSES Colletta H. (K.K.

) Moore QualPro, Knoxville, TN

ABSTRACT
Most hospitals now use simple quality tools such as flowcharting, brainstorming, fishbone charts, etc. in their process improvement efforts. While these tools are effective, they are limited in their use-potential. This session will use a case study to demonstrate how one hospital used experimental design techniques to improve quality.

THE STRATEGY
In a typical CQI team setting, team members might hold a brainstorming session to identify possible solutions to a problem. Afterward, a consensus is often reached on which one possible solution, also called a!actor, to test. This type of experimentation, known as one-factor-at-a-time experimentation (OFAT), is very common in industry. Unfortunately, OFAT is problematic: (1) it's inefficient and (2) it often provides invalid results. In contrast, DOE tests factors simultaneously as opposed to sequentially. In addition, DOE provides information on which combinations of factors are significant. Statisticians call these combinations interactions. Interactions often hold the key to true breakthroughs in process improvement [1].

INTRODUCTION
Most health care organizations recognize the need for continuous quality improvement (CQI) and, therefore, are using many of the basic CQI techniques such as flowcharting, brainstorming, fishbone charts, etc. While these tools are effective, they are limited in their use-potential. Although initial success can be obtained using these techniques, often the process is still not capable of meeting customers' needs. As a result, team members are left frustrated and feeling as if CQI implementation is a futile effort. This paper will illustrate how design of experiments (DOE) can dramatically improve process capability once obvious changes have already been implemented, and the improved process is predictable.

CASE STUDY; RADIOLOGY TURNAROUND TIME

A hospital in the Southeastern U.S. used DOE in its CQI effort to significantly reduce turnaround time for radiology reports. After nine months, the turnaround time decreased from approximately twelve hours to four hours. A team used the following procedure for improving the process:

Quest for Quality and Productivity in Health Services 1993 Conference Proceedings. Institute of Industrial Engineers.
1993 QualPro. Printed with pennission from QualPro.

130

 Determine key measure(s) and establish the measurement system Achieve stability Experiment with the process

Determine Key Measure(s) and Establish the Measurement System A key measure is a quantifiable outcome that provides information on how well the process is behaving from the external customers' viewpoint. The key measure for radiology reports was timeliness, which was operationally defined to be the time from patient arrival until the time the transcription was sent.
A measurement system was developed to track the time throughout the process. Each day, ten folders (which had the data collection tool attached) were randomly selected, and the key measure was recorded. The data were analyzed through the use of X, R charts. As expected, the process was unstable, or not predictable, as shown in Figure 1.

Achieye Stability Since stability is an important prerequisite to DOE, eliminating special cause variation, or variation which is not inherent in the system, is a critical step in process improvement. After investigation, the cause of this variation pointed to reports which had been requested on the nursing units before the film had been interpreted. The return of the folders to the Radiology Department was untimely; therefore, a process measure of film on floor time was analyzed to improve the key measure, total turnaround time.
The team identified potential solutions which would decrease film on floor time. After learning that the nursing units were not completely aware that the film had not been interpreted, one member suggested the presence of stickers on the front of the folder for easy identification. A sticker would indicate the need to return the folder as quickly as possible to the Radiology Department so the process could be completed. In addition, the team learned that each nursing unit did not have a standard location for radiology folders.

35 30 25 '" 20 l-.
CL UCL

=22.97

o = =: 15

=12.31 =1.65

10

5
LCL

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Date

Figure 1. Turnaround Time for Radiology Reports; Initial X Chart (Simulated Data)
131

Some units even used the regular, internal mail! courier process which also contributed to the delay. Therefore, one member suggested the provision of in-out boxes on each unit. Instead of reaching consensus on which one, or perhaps both, of these ideas should be implemented, the team used a more valid, scientific approach in its decision making: DOE. Ironically, however, this first DOE was used to achieve stability. Most DOEs are used after stability is achieved. Once the factors in an experiment are determined, the settings, also known as levels, of each factor need to be defined. The "low level" is typically the current condition, whereas the "high level" is the proposed change in current condition. Conventional codes for the low level and high level are - and +, respectively. The factors and levels for the design are found in Table 1.

Run
1

AB
+

2
3 4
2

+ + + + +

12.48 14.00 7.44 5.72 11.89 9.19 7.89 7.09

I
~

I
i

Table 2. 2 full factorial. The results are also provided in Table 2.Yrepresents the average film on floor time in hours for each treatment combination, or week. R represents the range of film on floor time in hours for each treatment combination. The DOE concluded that stickers significantly reduced film on floor time by 4.52 hours. In-out boxes, as well as the combination of stickers and in-out boxes, proved to be insignificant. This information prevented the wasted expense of in-out boxes throughout the hospital. After this initial DOE, the stickers eliminated special causes in the key measure. Surprisingly, however, a pattern became obvious: weekends were different than weekdays. By analyzing them separately, it became clear that the stickers not only eliminated special causes, but it also shifted common cause variation in weekday turnaround time. The average turnaround time was reduced by 53%. Ordinarily, many CQI efforts stop at initial success. However, team members still felt more gains could be realized through other possible process changes. Since the process was stable, DOE was used again to improve process capability.

Factor
A: Stickers

Level
-Not used + Used -Not used + Used

B: In-out boxes

Table 1. Factors and levels for initial DOE. A 22 full factorial, a type of experimental design shown in Table 2 , was used to assess the effect of stickers (A) and in-out boxes (B) on film on floor time [2]. Once the design is selected, the symbols (-,+) can then be decoded. For example, Run 1 specifies A- and B-. Therefore, for one week, stickers will not be used, nor in-out boxes on the units. Run 2 specifies A + and B-. Therefore, for a different week, stickers will be used, but in-out boxes on the units will not be used. The other runs are decoded similarly. Each week, film on floor times are recorded.

Experiment with the Process

Practical, feasible, cost-efficient factors were identified during a brainstorming session. These factors and their levels are found in Table 3 [3]. For this experiment, a Plackett-Burman [4] screening design was selected to provide an

Should use each week's daily averages as replicates within treatment to estimate exptl ("random") error. Also, depending on how many cases within each day, may use box- 132 and-whisker plots to further examine variation and check for outliers.

efficient way to reduce a large number of factors to a smaller set of important factors for subsequent experimentation [5].

A.
(-)
(+)

Short-Term Printer Orders were not reprinted to short-term file area. Folders were searched for after the current procedure was performed. A short-term printer was simulated which provided a process for obtaining the radiology folder before the current procedure was completed. Schedule Board Schedule board was not utilized. A schedule board was used to provide an organized method for tracking patients, staff, workload, etc. Sorting/Combo Separate areas were utilized for film sorting and short-term filing. Combined film sorting and short-term file areas. Runner File room was staffed as usual. An additional person was employed on a temporary basis to transport films to and from various areas. Quality Control (QC) All routine films were routed through QC where a technologist rechecked all films for quality. QC was discontinued. Technologists assumed full responsibility and accountability for film quality.

B.

(-)
(+)

A 16-run Plackett-Burman screening design is given in Table 4. This design allowed estimation of each main effect as well as each two-factor interaction. Similar to the smaller 22 full factorial previously mentioned, the design symbols are decoded. For example, Run 1 specifies A+, B-, C-, D-, and E+. Therefore, for a four-day time period, the short-term printer was used, the schedule board was not utilized, separate areas were utilized for ftlm sorting and short-term filing, a runner was not used, and the QC function was discontinued. As before, the other treatment combinations are decoded similarly. For each treatment combination, the average turnaround time was recorded for each of the four days.

Run
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A
+ + + + +

E
+

C.

(-)
(+)

+ + + + +

+ + + + +

+ + + + + + + +

D. (-)
(+)

+ +

+ +

+ +

+ + +

+ + + + +

E.

(-)
(+)

Table4. 16-run Plackett-Burman design.

Table 3. Factors and levels for screening design.

See note on previous page.

This is not a Plackett-Burman but rather a "jumbled" 2**(5-1) fractional factorial whose defining contrast is E = (ABCD). That is why 2fi's can be estimated.
133

A Pareto chart of main effects and interaction effects is shown in Figure 2. Without going into detail of the calculations, the DOE showed that the schedule board/runner (BD) interaction was the only significant effect on total turnaround time. Figure 3 shows a quasi-interaction plot which helps illustrate the BD interaction. Although this technique provides a crude estimate of the AE interaction, it works well in practice. As shown, the combination of the schedule board with no runner significantly reduced turnaround time. This new knowledge prevented an unnecessary added position in the department. As a result of the implemented changes, average turnaround time decreased to approximately four hours.

14 12

6
B: Schedule Board

Figure 3. Quasi-interaction plot of BD interaction.

4.034

=2.42 -------------------------------"Control Limits"

BD

CD DE

AC BE AD AB Factor

AE CE

BC

Figure 2. Pareto chart of main effects and interaction effects.

134

CONCLUSIONS
Certainly, simple tools are effective in initial process improvement efforts. However, managers should not be complacent with initial success. To continuously improve processes to meet customer needs and reduce variation, process innovation and design changes are required. Contrary to the belief that process changes should only be made one factor at a time, DOE provides a proven, efficient, statistically sound way to foster innovation in the organization. Therefore, DOE should be built into the overall strategy for any process improvement effort.

BIOGRAPHICAL SKETCH
Colletta H. (K.K.) Moore, Consultant, QualPro, Knoxville, Tennessee. KK has extensive experience in statistical methods, quality management, and training and education in the health care industry. In addition, she has authored and coauthored manual texts for many of the QualPro seminars, including Basic Statistical Quality Improvement Techniques for the Health Care Industry and Experimental Design Techniquesfor Service Processes. KK holds a B.S. in secondary mathematics education and an M.S. in statistics, both from the University of Tennessee. She is a member of the American Society for Quality Control (ASQC).

REFERENCES
1.

2.

3.

4.

5.

Experimental Design Techniquesfor Service Processes. (1992). Knoxville: QualPro, Inc. Hicks, Charles R. (1982). Fundamental Concepts in the Design ofExperiments, New York: CBS College Publishing. Basic Statistical Quality Improvement Techniques for Clinical Processes. (1992). Knoxville: QualPro, Inc. Plackett, R. L. and Burman, J. P. (1946). "The Design of Optimum Multifactorial Experiments, " Biometrika, 33, 305-325. Schmidt, Stephen R. and Robert G. Launsby. (1988). Understanding Industrial Designed Experiments, Colorado Springs: Air Academy Press.

135