ASTD 2008 International Conference & Exposition

Using Multi-Rater Assessments to Measure Emotional

Intelligence and Leadership Behaviors in Middle Managers
Session # TU-215
Denise Sawyer-Johnson, D.M.
Leadership and Talent Development Manager
BMW of North America, LLC
denise.sawyer-johnson@bmwna.com

Ben Palmer, PhD.

Chief Executive Officer
Genos
ben.palmer@genos.com.au

Session Learning Objectives:

1. Assess the value of emotionally

intelligent middle managers in your
organization.

2. Develop the business case based on

research findings.

3. Examine a best practice multi-rater

assessment approach.

www.astd.org

Page 1
 Emotional Intelligence and Leadership Styles

Overview
A number of research studies have examined the relationship between emotional
intelligence and leadership styles. However, few studies have examined these
relationships within the middle level manager population. In this presentation we
present the findings of a study examining the relationship between emotional
intelligence and the leadership styles of middle level managers at BMW North America.
The assessments used, methodology adopted and findings of the study are presented
to help you:
1. Assess the value of emotionally intelligent middle managers in your
organization.
2. Develop the business case based on research findings.
3. Examine a best practice multi-rater assessment approach.

Emotional Intelligence
Emotional intelligence (EI) was first conceptualized in 1990 by Peter Salovey and Jack
Mayer as “a set of abilities to do with emotions and the processing of emotional
information”. Since this time a large number of different models and measures of EI
have emerged, providing alternative approaches to the application of the concept in the
workplace. These different models and measures of EI can be placed into three major
categories:
1. The ability approach to EI
• Much like IQ assessments, EI-ability assessments measure individual
differences in emotional abilities
2. The trait approach to EI
• Much like personality assessments, trait EI assessments measure
individual differences in emotion-related preferences and styles e.g., “It’s
fairly easy for me to express my feelings”; and “I’m sensitive to others
feelings”.
3. The competency approach to emotional intelligence
• Much like typical multi-rater competency assessments, competency EI
assessments measure individual differences in how often EI related
behaviors are demonstrated.

2
 Emotional Intelligence and Leadership Styles
The development of alternative approaches to EI raises the question of “which
approach to the measurement and application of EI best suits learning and
development initiatives?

What are the attributes of an ideal EI model and assessment for learning and
development applications?

• Which approach “ability, trait or competency”

• How many dimensions would the assessment measure?

• How long would it take to do?

• What information would be presented in the feedback report?

• What else would be important?

3

The Genos model of emotional intelligence
The EI Skill
Emotional Self-
Awareness; perceiving
and understanding your own
feelings
Emotional Expression;
effectively expressing your
own feelings
Emotional Awareness of
Others’; perceiving and
understanding others’
feelings
Emotional Reasoning;
using emotional information
in decision-making
Emotional Self-
Management; effectively
managing your own feelings
Emotional Management of
Others; positively
influencing others’ feelings
Emotional Self-Control;
effectively controlling strong
emotions
Emotional Intelligence and Leadership Styles
Leaders with it:
Awareness of how their
feelings influence their
decisions, behavior and
performance
Build trust and
understanding
Generate engagement and
extra effort
Make effective decisions
and facilitate buy-in
Demonstrate resiliency, role
model positive behavior
Facilitate high productivity
and motivation
Provide stability & focus in
the face of adversity
Leadership without it
Blind Spot
Create a culture of artificial
harmony and fear of
confrontation
Generate high turnover,
high satisfaction and KPI
compliance
Create ambiguity and a lack
of ownership
Focus on emotions, suffer
from stress, demonstrate
inconsistent behavior
Low morale & productivity
Put out fires they create
4
 Emotional Intelligence and Leadership Styles

Emotional intelligence and middle level managers

Genos Role Analysis Profiler 1 = Almost Never; 2 = Rarely; 3 = Sometimes; 4 = Often; 5 = Almost Always
To perform the role successfully, how often should a middle level manager…
Be aware of the things that upset him/her at work. c d e f g
Be aware of how feelings influence the way he/she responds to colleagues. c d e f g
Be aware of his/her body language at work. c d e f g
Be aware of his/her mood state at work. c d e f g
Emotional Self-Awareness Score (out of 20)
Provide positive feedback to staff. c d e f g
Effectively express how he/she feels when upset by someone at work. c d e f g
Effectively express optimism at work. c d e f g
When frustrated by something at work, discuss their frustration appropriately. c d e f g
Emotional Expression Score (out of 20)
Understand the things that cause others to feel engaged at work. c d e f g
Identify others’ non-verbal emotional cues (e.g., body language). c d e f g
Understand the things that make people feel optimistic at work. c d e f g
Understand what makes people feel valued at work. c d e f g
Emotional Awareness of Others Score (out of 20
Ask others how they feel about different solutions when problem solving. c d e f g
Demonstrate to colleagues that he/she has considered others’ feelings. c d e f g
Consider the organization’s values when making important decisions. c d e f g
Communicate decisions at work in a way that captures others’ attention. c d e f g
Emotional Reasoning (score out of 20)
Engage in activities that make him/her feel positive at work. c d e f g
Effectively deal with things that annoy him/her at work. c d e f g
Appropriately respond to colleagues who frustrate him/her at work. c d e f g
Quickly adjust to new conditions at work. c d e f g
Emotional Self-Management (score out of 20)
Create a positive work environment for others. c d e f g
Motivate others toward work related goals. c d e f g
Help people find effective ways of responding to upsetting events. c d e f g
Help people deal with issues that cause them frustration at work. c d e f g
Emotional Management of Others (score out of 20)
Remain focused when anxious about something at work. c d e f g
Demonstrate excitement at work appropriately. c d e f g
Hold back his/her initial reaction when something upsets him/her at work. c d e f g
Still think clearly when upset at work. c d e f g
Emotional Self-Control (score out of 20)

5
 Emotional Intelligence and Leadership Styles

Emotional intelligence and middle level managers

How does EI relate to middle level management success?

Discussion Notes:

6
 Emotional Intelligence and Leadership Styles

The research - research measures

Genos Multi-Rater Emotional Intelligence Assessment Scale

• A behavior-based competency assessment – 70 questions
• Measures how often individuals demonstrate emotionally intelligent
workplace behaviors aligned to the Genos model
• Has a rater familiarity and consistency index
• Provides benchmarked results via percentile scores

Multifactor Leadership Questionnaire

• Behavior-based leadership style assessment – 45 questions
• Measures how often individuals demonstrate different leadership styles
according to the Bass & Avolio transformational/transaction leadership model
• Provides benchmarked results

7
 Emotional Intelligence and Leadership Styles

The research question

•Extra Effort
Emotional intelligence

•Satisfaction
•Effectiveness

Leadership effectiveness

The research process

8
 Emotional Intelligence and Leadership Styles

Research findings
Middle level managers’ skill at managing emotions within themselves and others
predicts their skill at:
ƒ Encouraging Innovative Thinking (IS; 21%)
ƒ Acting with Integrity (IB; 21%)
ƒ Inspiring Others (IM; 39%)
ƒ Building Trust (IA; 55%)
ƒ Coaching People (IC; 35%)
ƒ Rewarding Achievements (CR; 22%)
ƒ Generating Extra Effort (30%)
o Getting others to do more than they expected to do
o Heighten others’ desire to succeed
o Increase others’ willingness to try harder
ƒ Effectiveness (46%)
o Are effective in meeting others’ job related needs
o Are effective in representing others to higher authority
o Are effective in meeting organizational commitments
ƒ Satisfaction (48%)
o Uses methods of leadership that are satisfying
o Work with others in a satisfactory way
Middle level managers skill at demonstrating emotional self-awareness and emotional
expression predicts their skill at:
ƒ Acting with integrity (IB; 8%)
ƒ Inspiring Others (IM; 6.8%)
ƒ Rewarding Achievements (CR; 5.5%)

Middle level managers’ skill at perceiving and understanding others’ feelings (Emotional
Awareness of Others’) predicts their skill at:
ƒ Building Trust (IA; 3.6%)
ƒ Extra Effort (30%)
ƒ Effectiveness (36%)
ƒ Satisfaction (34%)

9
 Emotional Intelligence and Leadership Styles

Research conclusions:
Middle level managers’ skill at effectively managing emotions within oneself and
others; understanding others’ emotions; and expressing emotions:

ƒ Provides a foundation for leaders to be open to challenging assumptions and

seeking different opinions
ƒ Enables the expression of values, purpose and the ability to garner trust
ƒ Facilitates the expression of behaviors that encourage constructive actions,
instill pride, show confidence and promote enthusiasm
ƒ Facilitates the expression of motivation and inspiration in remote locations
ƒ Facilitates outcomes of effective leadership (extra effort, effectiveness &
satisfaction).

Notes:

10
 Emotional Intelligence and Leadership Styles

Implications for leadership practice

ƒ Can EI be developed and does that development result in

enhanced leadership capability?
ƒ Can EI be used in recruitment and selection to identify
leadership talent and if so what does this produce?
ƒ Should we be using multi-rater EI & leadership assessments
to challenge conventional top down methods of performance
evaluation?

Case Study 1 – Senior Finance Executives

ƒ 10 senior finance executives
ƒ EI & leadership capability assessment pre-and-post the program
ƒ 8 personal coaching sessions over a 3 month period (6 mth time 2 assessment
period)

Percentage of change (raters)

Emotional Self-
Awareness/Expression
25%

Emotional Awareness of
Others 59%

Emotional Reasoning 76%

Emotional Management of
Self/Others
54%

Emotional Self-Control 34%

0% 20% 40% 60% 80% 100%

Percentage

11
 Emotional Intelligence and Leadership Styles

Case Study 1 – Senior Finance Executives (cont)

Percentage of change (Leadership Capability)

Role Model 11%

Developing Staff 11%

Motivating Staff 13%

Innovative Thought 9%

Customer Focus 13%

0% 5% 10% 15% 20%

Percentage

Case study 2 – Project team leaders – using EI to enhance team

performance

Emotional Self-Control - Time 2

72
Emotional Self-Control - Time 1
62
Emotional Management Self/Others - Time 2 67
Emotional Management Self/Others - Time 1 61
Emotional Reasoning - Time 2 76
Emotional Reasoning - Time 1 34
1
Emotional Understanding of Others - Time 2 82

Emotional Understanding of Others - Time 1 59

80
Emotional Self-Awareness/Expression - Time 2
52
Emotional Self-Awareness/Expression - Time 1
76
Total EI - Time 2
54
Total EI - Time 1

0 20 40 60 80 100
Percentile Scores

12
 Emotional Intelligence and Leadership Styles

Case study 2 – Project team leaders – using EI to enhance team

performance

Percentage of improvement (Time 1 versus Time 2)

The team's interpersonal

effectiveness 45%

The team's organising and planning

ability 28%

Quality of the team's output 25%

Timeliness of the team's output 33%

Sense of identity with the

team/Intention to stay 19%

Perceived value of the team by the

broader business 20%

0% 10% 20% 30% 40% 50%

Case Study 3 – sanofi-aventis Sales Performance Enhancement

$540,000
Retail sales per territory ($)

$520,000

$500,000

$480,000

$460,000
Development Group
Control Group
$440,000

$420,000
Sep-05 Oct-05 Nov-05

13
 Emotional Intelligence and Leadership Styles

Case Study 3 – Recruiting emotionally intelligent employees

• A net revenue increase of $2,558,360.00

• L’Oreal, selected a cohort of sales professionals on the basis of higher EI
and the cohort generated significantly more revenue than existing
employees

• 37% more life insurance policies sold in first two years of employment
• New sales professionals at Metlife high in EI sold more life insurance than
their less emotionally intelligent peers

Notes:

14
 Emotional Intelligence and Leadership Styles

References:

Bar-On, R. (1997). Bar-On Emotional Quotient Inventory. Toronto, ON: Multi-Health

Systems.

Gardner, L. and Stough, C. (2002). Examining the relationship between leadership and
emotional intelligence in senior level managers. Leadership and Organisation
Development Journal, Vol 23, pp. 68-78.

Goleman, D. (1995). Emotional Intelligence. New York: Bantam Books.

Jennings, S. & Palmer, B.R. (2007). Enhancing sales performance through emotional
intelligence development. Organisations and People, 14, 55- 61.

Lynn, C.M, Mason, G.U. (2000). The relationships between emotional intelligence,
personality, critical thinking and organisational leadership performance at upper
levels of management. Dissertation Abstracts International: Section B: The
Sciences & Engineering, Vol 61 (2-B), pp.1121.

Palmer, B., Walls, M., Burgess, Z. and Stough, C. (2001), Emotional intelligence and effective
leadership. Leadership and Organisation Development Journal, Vol 22, pp.1-7.

Palmer, B.R., Gardner, L., & Stough, C. (2003). Measuring emotional intelligence in the
workplace. Emotional Intelligence in Organisations, Published by ICCM ISBN: 0-
6396828-7

Palmer, B.R. (2007). Models and measures of emotional intelligence. Organisations

and People, 14, 3-10.

Internet Links to Emotional Intelligence

1. http://www.genosamericas.com

2. http://www.eiconsortium.org

3. http://ei.haygroup.com

4. http://eqi.mhs.com

5. http://www.cjwolfe.com

6. http://www.eq.org

15
 Emotional Intelligence and Leadership Styles

Job Aid
Emotional Intelligence L&D Enhancement Methodology

1. Why - develop the “why” for EI, i.e., business case

2. Position – position EI as the method not the outcome

3. Research – present the theory & relevant research

4. Relevancy – have participants explore the relevancy of the

concept to what they do day-to-day

5. Insight – provide insight into their strengths and

opportunities for development with a multi-rater behavior-
based emotional intelligence assessment

6. Tools & Techniques – provide tools and techniques to help

them enhance the application of EI in their work

7. Practice Application – have the participants practice

applying the EI tools & techniques with their peers

8. Action & Follow-Up – have the participants complete

personal action plans, diary events as they complete them
and, in follow-up sessions, start with reflecting on what has
been done

16