PERSONNEL

PSYCHOLOGY
1984, 37

REACTION OF EMPLOYEES TO PERFORMANCE

APPRAISAL INTERVIEWS AS A FUNCTION OF THEIR
PARTICIPATION IN RATING SCALE DEVELOPMENT
STANLEY B. SILVERMAN
The University of Akron
KENNETH N. WEXLEY
Michigan State University
A substantial amount of research has been conducted on the performance appraisal feedback interview. The purpose of the present field
study was to see whether employee involvement in the development
of rating scales used in the feedback interview affected ratees perceptions of the interview. Behaviorally anchored rating scales (BARS)
were developed for five job families in a midwestem hospital. A participation group consisted of employees from each of the job families
who were involved in all phases of BARS construction. A nonparticipation group involved an equal number of employees from each
of the job families who were not involved in the BARS development.
The results showed that participation in BARS construction led to
favorable perceptions regarding the performance appraisal interview
process as well as positive outcomes.

A great deal of research on performance feedback has appeared in

the literature in recent years (Burke, Weitzel and Weir, 1978; Ilgen,
Fisher and Taylor, 1979). Much of this literature has focused on the
relationship between performance appraisal interview process
characteristics (e.g., opportunity for employee participation, amount of
derogation) and outcome measures such as motivation to improve and
satisfaction with the appraisal (Burke et al., 1978; Greller, 1975, 1978;
Nememff and Wexley, 1979; Wexley, Singh, and Yukl, 1973). Reviews
of this literature by Wexley (1979) and Burke et al. (1978) suggest six
general characteristics of effective interviews: participation, support, goal
setting, discussing problems, minimum of criticisms, and split sessions.
Friedman and Cornelius (1976), although they were concerned more
with psychometric characteristics as the dependent variable, showed that
rater participation in scale construction can lead to greater convergent
Requests for reprints should be addressed to Stanley B. Silverman, Division of Associate
Studies, The University of Akron, Akron, Ohio, 44325.

0 1984 Copyright Personnel Psychology, Inc.

703

704

PERSONNEL PSYCHOLOGY

validity, less halo, and lower levels of variance attributable to rating

errors.
The purpose of the present study was to investigate the impact of participation in the development of behaviorally anchored rating scales
(BARS) (Jacobs, Kafry, and Zedeck, 1980) on ratees reactions to the
performance appraisal interview that they received from their managers.
Extrapolating from the results reported above and previous findings on
the effects of participation in performance appraisal interviews (Kay,
Meyer, and French, 1965; Wexley et al., 1973), it was hypothesized
that ratee participation in the construction of the instrument would result
in more favorable perceptions by them of various process characteristics
such as their opportunity to participate, the supportiveness of their
managers, their contribution, and the utility of the session. It was also
hypothesized that ratee participation in instrument development would
generate greater ratee motivation to improve performance and more
satisfaction with the appraisal. It was believed that these effects would
occur because ratee participation should provide a better understanding
of the appraisal process, more trust in the instrument used, and more
feelings of ownership and involvement in the entire appraisal process.

Method
BARS Developmen1
BARS were developed in a small hospital for each of the following
five job families: clerical (secretaries, typists), non-clerical staff
(maintenance, housekeeping), technical and professional (lab technicians,
pharmacists), nursing (L.P.N.s, R.N.s), and managementlsupervisory.
An approach adapted from Smith and Kendall(l963) incorporating the
following four steps was undertaken: 1) generation and definition of performance dimensions; 2) generation of examples of effective, average
and ineffective behaviors; 3) simultaneous retranslation and scaling of
each item; and 4) final instrument development. Employees who participated in the development of the 7-point scales for their job family
were involved in only the first three steps; the employees managers
were not involved in any of these steps. The authors then chose the items
for the final instruments based upon their mean values and standard deviations of less than 1.5. The clerical and non-clerical BARS each contained four scales, the technical/professional and the managementlsupervisory each contained five scales, and the nursing BARS contained seven
scales.

Appraisal Questionnaire Developmenl

Based upon a review of the performance appraisal interview literature,

SILVERMAN AND WEXLEY

705

the authors assembled a 34-item questionnaire tapping employee reactions to 10 aspects of the performance appraisal interview (Burke et al.,
1978; Greller, 1975, 1978; Wexley et al., 1973). A four-point rating
scale was used with each item. The scale ranged from (1) Not at all,
I do not feel this way at all, Does not describe it at all, Describes it
very poorly to (4) Completely, I feel exactly this way, Describes it extremely well. Shown below are the 10 aspects with their respective alpha
coefficients in parentheses followed by an example of an item: Perceived Utility (.88)-I learned a lot from the appraisal; Anxiety/Tenseness
(.75)-The interview was upsetting to me; Derogation (.47)-My
manager really did not have enough information about my performance;
Supportive Appraisal Behavior (.80)-My manager praised me for what
I had done well; Opportunity to Participate (.59)-My manager gave
me the opportunity to state my side of the issues; Degree of Criticism
(.52)-My manager was quite critical of my performance; Goal Setting
(.84)-Specific objectives were set; Contribution (.77)-I made suggestions about how the job might be done differently; Satisfaction with the
Appraisal (.63)-I am satisfied with the interview; and Motivation to
Improve-At the end of the interview I really wanted to improve my
performance. All aspects contained between three and five items with
the exception of Degree of Criticism which contained two items. No
alpha coefficient could be calculated for Motivation to Improve because
it consisted of only one item. The intercomlations among the 10 variables
were found to range from -.43 to .77 with an average correlation,
calculated using Fisher z Transformations, of .33. Motivation to Improve and Satisfaction with the Appraisal are the performance appraisal
interview outcome measures, whereas the remaining eight aspects
measured by the questionnaire are the appraisal interview process
characteristics.
Sample and Procedure
Sixty-five employees from a 120-bed midwestern hospital employing 240 individuals participated in the study. All 65 employees filled
out the appraisal questionnaire immediately following a meeting with
their manager to discuss their performance as related to the BARS. Ten
managers conducted the performance appraisal interviews with the 65
employees. None of these managers had received any previous formal
training in conducting feedback interviews. These managers were told
to distribute a copy of the appropriate BARS to each of the 65 employees
in order to familiarize themselves with the scales that would be discussed during their performance appraisal interview. Out of the 65 employees,
32 participated in the development of the BARS while 33 did not. The
employees in the participation and nonparticipation groups were evenly

706

PERSONNEL PSYCHOLOGY

distributed across the five job families. Each of the ten managers held
appraisal interviews with approximately the same number of employees
from the participation group and the nonparticipation group. Performance
appraisal interviews were conducted annually at the hospital, and all
the interviews involved in the research were conducted approximately
ten months after the previous interviews. All interviews were conducted
over a two week period that occurred three months after the completion
of the BARS. All employees knew that these BARS ratings were done
for administrative purposes and would be used in the same fashion as
previous ratings. The 65 employees were chosen randomly from within
each of the three shifts and each of the five job families throughout the
hospital. The 32 employees that participated in the BARS development
were chosen in a similar random basis.
Before the project began, the entire hospital was sent a memo explaining
that the rating scales were being designed for the staff and that some
employees would be selected on a random basis to help provide information for the development of the scales. The 32 employees who participated in the development of the BARS attended all meetings, while
the remaining 33 did not participate in any stage of the scale development. The 32 participating employees were asked not to discuss the
specifics of these meetings with their managers or any of the hospital
employees. Therefore, the managers were not aware of who had participated in these scale development meetings. Even though the employees
had been involved in previous annual performance appraisal interviews
with their managers, none of the employees had ever participated in scale
development prior to this research project.
Results and Discussion
To ensure that those individuals who participated in the development
of the BARS showed no difference in their job performance from those
who did not participate, an average score was calculated for each individual on their respective set of BARS. T-tests were then performed
between the two groups and no significant differences were found. Ttests were also conducted in order to investigate whether ratees who were
involved in the BARS development differed in their perceptions of the
appraisal interviews from those ratees who were not involved. A stepwise discriminant analysis was then conducted among the variables in
order to select the most useful discriminating variables.
Table 1 shows the means, standard deviations, and the resultant ttests. Those employees who participated in the development of the scales
on which they were rated differed in their perceptions of the appraisal
interview process characteristics as well as the outcome measures.
Specifically, those ratees who participated perceived the appraisal in-

707

SILVERMAN AND WEXLEY

TABLE 1
Means, Standard Deviations and t Values For Ratees
Perceived Differences On Feedback Characteristics And Outcomes
Based Upon Participation In Development of BARS
Feedback Characteristic
Or Outcome
Utility
Anxiety/Tenseness
Derogation
Supportive Appraisal
Behavior
Opportunity to
Participate
Degree of Criticism
Goal Setting
Contribution
Satisfaction with
Appraisal
Motivation to
Improve

Sample

Mean

SD

Participation
No Participation
Participation
No Participation
Participation
No Participation

3.26
2.60
1.35
1.26
1.36
1.44

0.53
0.75
0.67
0.51
0.52
0.44

Participation
No Participation

3.90
3.67

0.22
0.51

2.26*

Participation
No Participation
Participation
No Participation
Participation
No Participation
Participation
No Participation

3.73
3.32
1.58
1.49
3.24
2.72
3.20
2.74

0.41
0.65
0.80
0.66
0.65
0.86
0.59
0.69

2.92**

Participation
No Participation

3.60
3.02

0.87
0.72

2.92**

Participation
No Participation

3.52
2.75

0.63
0.94

3.77**

3.87**
0.55
-0.69

0.49
2.67**
2.76**

*p c .05
**p < .01

terview to be significantly more useful, that their supervisor was more

supportive, that they were given more of an opportunity to participate,
that goals and objectives were set to a greater extent and that they made
more of an impact or contribution to the appraisal interview. In addition, there were significant differences on both of the appraisal interview outcome measures. Those employees who participated in the
development of the BARS were more satisfied with the interviews and
were more motivated to improve their job performance. On the other
hand, no significant differences were found between the two groups in
their perceptions of how anxious o r tense the interviews made them,
how much the manager detracted fmm the interview by being unprepared
and arbitrary, and how critical the manager was of their performance.
The stepwise discriminant analysis was then conducted using the ten
appraisal interview variables shown in Table 1. As a result of the step-

708

PERSONNEL PSYCHOLOGY

wise discriminant analysis, two variables were entered into the resulting
discriminant function. Motivation to Improve (Wilks lambda = .74,
p c .001) was the first variable entered and the other variable entered
was Opportunity to Participate (Wilks lambda = .71, p c .OOl). The
resulting discriminant function had an eigenvalue of .40, Wilks lambda = .7 1, x2 (2) = 16.95, p < .001. The standardized discriminant
function coefficients for Motivation to Improve and Opportunity to Participate were .85 and .39, respectively.
The fact that motivation to improve performance on the part of the
ratee proved to be the most discriminating variable is quite encouraging. Involving workers in the development of the rating instrument led
workers to respond that they were more motivated to improve their performance. Although no measures were available to determine whether
or not there were actual improvements in subsequent job performance,
Burke and Wilcox (1969) found that actual improvement in job performance correlated .57 with an individuals motivation to improve performance. One possible reason that those employees who participated in the
development of the BARS felt they were given more of an opportunity
to participate in the interview by their managers might be because they
had more of a sense of ownership (Greller, 1978) in the appraisal
process. It seems logical that those employees who actually participated
in the development of the BARS would perceive more psychological
participation (Wexley et al., 1973) in the appraisal interview.
Specifically, our results suggest that involving workers in the development of BARS can lead to more positive reactions by employees to the
performance appraisal interview process as well as subsequent outcome
measures. These results are supportive of Landy and Trumbo (1980)
who pointed out that one of the advantages of BARS may have nothing
to do with the measurement of performance per se, but stem from the
high degree of involvement of the workers and supervisors in scale
development. This study supports their hypothesis with regards to
workers. Those workers who were involved in the BARS development
clearly showed more positive reactions to performance appraisal than
those who were not involved.
This study also suggests that perhaps similar positive outcomes could
be achieved by involving employees in the development of other types
of performance appraisal instruments, such as Behavioral Observation
Scales (Latham and Wexley, 1977) and Mixed Standard Scales (Blanz
and Ghiselli, 1972). Friedman and Cornelius (1976) showed that rater
participation in scale construction was able to enhance the psychometric
characteristics of two instruments, BARS and graphc rating scales. Future
research should examine the generalizability of these results to various
appraisal instrument.

SILVERMAN AND WEXLEY

709

In the present study, those employees that participated in the development of the BARS participated in three steps. Employee involvement
to this full extent may not be necessary and/or practical to achieve similar
results. Future research should be directed toward determining the extent of employee involvement needed to achieve similar positive results.
Knowing these results, an issue that needs to be dealt with in the future
studies is whether the experimental manipulation influenced only the
subordinates perceptions of what transpired during the appraisal interviews or whether it also influenced the managers actual behaviors. Unfortunately, this issue could not be considered in the present study because
of the infeasibility of measuring managerial behavior during the appraisal
sessions. This would have necessitated the audiotaping or videotaping
of the sessions which is something the hospital administration would
not permit. Further understanding could also be obtained by having a
3-group study with a placebo participation group in which the participants would be involved in a consultative nature on an appraisalirrelevant issue. An additional experimental condition might have been
a nonparticipative training session on the developed BARS.
The study raises the following practical questions that should be investigated in future research: (1) Can the expense of participation in
scale development be justified from a cost/benefit viewpoint? (2) Are
these gains greater than could be achieved by other indoctrination and
training procedures regarding existing appraisal programs? (3) How much
variation in ratee participation can be permitted across organizational
units without causing feelings of unfairness? (4) Do the effects of ratee
participation wear off over time or persist over repeated appraisals?
REFERENCES
Blanz, F . , and Ghiselli, E. E. (1972). The mixed standard scale: A new rating system.
PERSONNEL PSYCHOLOGY,
25, 185-199.
Burke, R . J . , Weitzel, W . , and Weir, T . (1978). Characteristics of effective employee
performance review and development interviews: Replication and extension. PERSONNEL PSYCHOLOGY, 31, 903-9 19.
Burke, R . J . , and Wilcox, D. S . (1969). Characteristics of effective employee performance reviews and developmental interviews. PERSONNEL
PSYCHOLOGY,
22,
29 1-305.
Friedman, B. A , , and Cornelius, E. T. (1976). Effects o f rater participation in scale
construction on the psychometric characteristics of two rating scale formats. Journal
of Applied Psychology, 61, 210-216.
Greller, M. M. (1975). Subordinate participation and reactions to the appraisal interview. Journal of Applied Psychology, 60,544- 549.
Greller, M. M. (1978). The nature of subordinate participation in the appraisal interview. Academy of Management Journal, 22, 646- 658.
Ilgen, D. R., Fisher, C. D., andTaylor, M . S. (1979). Consequencesof individual feedback on behavior in organizations. Journal of Applied PsychoZogy, 64 349-371.

7 10

PERSONNEL PSYCHOLOGY

Jacobs, R., Kafry, D., and Zedeck, S. (1980). Expectations of behaviorally anchored
rating scales. PERSONNEL
PSYCHOLOGY,
33, 595-640.
Kay, E., Meyer, H. H., and French, J . R. P., Jr. (1965). Effects of threat in a performance appraisal interview. Journal of Applied Psychology, 49, 31 1-317.
Landy. F. J . , and Trumbo, D. A . (1980). Tne Psychologyof Work Behavior (rev. ed).
Homewood, IL: Dorsey Press.
Latham, G . P., and Wexley, K. N. (1977). Behavioral observation scales for performance appraisal purposes. PERSONNEL
PSYCHOLOGY,
30, 255- 268.
Nemeroff, W. F., and Wexley, K. N. (1979). An exploration of the relationship between performance feedback interview characteristics and interview outcomes as
perceived by managers and subordinates. Journal of Occupational Psychology,
52, 25-34.
Smith, P. C . , and Kendall, L. M . (1963). Retranslation of expectations: An approach
to the construction of unambiguous anchors for rating scales. Journal of Applied
Psychology, 47, 149-155.
Wexley, K. N. (1979). Performance appraisal and feedback. In S. Kerr (Ed.), Organizafional Behavior. (pp. 241-259). Columbus, Ohio: Grid Publishing.
Werley, K. N., Singh, J. P., and Yukl, G. A. (1973). Subordinate personality as a
moderator of the effects of participation in three types of appraisal interviews. Journal of Applied Psychology, 58, 54-59.