Learning for Change Survey

Olivier Serrat
2010

The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the Asian Development Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data included in this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply any view on ADBs part as to sovereignty or independent status or necessarily conform to ADBs terminology.

The Learning Organization Model

A learning organization knows the role that learning plays in developing organizational effectiveness. It demonstrates this by having an inspiring vision for learning and a learning strategy. Organizational learning is neither possible nor sustainable without understanding what drives it.
The organization subsystem values leadership, structure, communication systems, allocation of adequate resources, planned and emergent learning, and failures and unintended outcomes. A learning organization needs reflective people. This subsystem values teamwork, learning and knowledge development, individual and collective learning for development, and development of leadership competencies.

The Learning Organization Model

Knowledge is critical in a learning organization because it is both a product of knowledge and its source. This subsystem values individual and collective knowledge production, systems and infrastructure of knowledge management, feedback mechanisms, resilient organizational memory, and collaborative mutual learning arrangements. In a learning organization, technology is harnessed without constraining knowledge management and learning. This subsystem values creative use of information and communication technologies, and provides opportunities for staff to learn how to make use of technologies for knowledge management and learning.

The Learning Organization Model

Society

Economy
Environment

Polity
Technology

Learning
Organization People Knowledge Technology

Organization

People

Knowledge

Technology

Multiple Approaches, including

Management Science Organizational Development Cognitive Psychology Computer Science

Learning for Change Survey

The 2010 Learning for Change Survey
aimed to introduce a new diagnostic tool that examines organizational learning and deepens understanding of progress toward creating and sustaining a learning organization; featured positive statements depicting ideal levels of organizational competence across four subsystems of organizational learning(i) organization, (ii) people, (iii) knowledge, and (iv) technology; and captured perceptions of ADB staff in absolute confidence using a six-point Likert scale(1) strongly agree, (2) agree, (3) neutral, (4) disagree, (5) strongly disagree, and (6) don't know.

Survey Responses

The Organization Subsystem

Statement 1 (Inspiring vision) There is an inspiring vision for learning and an organizational learning strategy that clearly communicates that learning is critical to organizational success. (59% Strongly Agreed, Agreed)
Statement 2 (Learning culture) Leaders take an exemplary leading role in creating and sustaining a supportive learning culture. (51% Agreed) Statement 6 (Resources) Adequate resources are allocated for learning in terms of time, allocation, specialist support staff, budgets for knowledge management infrastructure, formal and informal communities of practice and other value networks, and learning and development programs. (34% Disagreed)

The Organization Subsystem

Statement 9 (Planned learning) Emergent learning is encouraged by creating opportunities for informal sharing of knowledge and experience. (59% Strongly Agreed, Agreed) Statement 10 (Failures and unintended outcomes) Failures and unintended outcomes are the focus of constructive discussions. When such incidents involve clients, care is taken to protect their reputation. (36% Neutral, 13% Don't Know)

The Organization Subsystem

Statement Subject Strongly Agree 13 Agree Neutral Disagree Strongly Disagree 5 Don't Know 2 1. Inspiring vision 46 18 16

2. Learning culture
3. Formal organizational structure 4. Informal organizational structure 5. Communication system 6. Resources 7. Approach to learning 8. Planned learning 9. Emergent learning 10. Failures and unintended outcomes Average

12
8 4 7 5 4 7 7 3 7

39
38 43 41 38 35 42 52 34 41

23
23 29 23 17 34 25 21 36 25

16
23 14 18 26 18 16 13 9 17

7
7 3 6 8 5 7 4 5 6

2
2 7 5 5 4 4 3 13 5

The Organization Subsystem

Statement 1 (Inspiring vision) Responses (i) demonstrate a strong sense of communicated vision on the importance of learning for ADB's success; and (ii) provide a strong platform for building ADB's organizational learning capacity.
Statement 2 (Learning culture) Responses suggest that ADB's leaders are successfully creating a supportive learning culture for their colleagues. Statements 1 and 2 Responses suggest that there may be some examples of good practice to disseminate and learn from.

The Organization Subsystem

Statement 4 (Informal organizational structure) Responses suggest the need to examine current areas of good practice and make these examples more widely known within ADB.
Statement 7 (Approach to learning) Responses suggest lack of understanding of planned and emergent learning. Statement 10 (Failures and unintended outcomes) Responses suggest limited awareness on the subject of protection of client reputations.

The People Subsystem

Statement 7 (Individual and team-based learning and development) Staff members successfully use a wide range of opportunities for individual and team-based learning and development. (32% Neutral)
Statement 9 (Rewards and incentives) ADB uses a wide range of formal and informal rewards and incentives for contributing organizational learning and knowledge development. (44% Strongly Disagreed, Disagreed) Statement 10 (Leadership) Leadership (based on the possession of expertise and knowledge) is expected from staff members at all levels in the organizational hierarchy. (52% Strongly Agreed, Agreed)

The People Subsystem

Statement Subject 1. Reflective practitioners 2. Tools, methods, and approaches 3. Psychological safety and trust 4. Learning communities 5. New ideas, trends, and practices 6. Developing and retaining staff 7. Individual and teambased learning and development 8. Time and performance management systems 9. Rewards and incentives 10. Leadership Strongly Agree 6 1 3 3 9 3 1 Agree 44 33 31 35 38 23 35 Neutral 22 27 29 30 27 29 32 Disagree 20 27 28 22 15 28 23 Strongly Disagree 7 6 6 6 9 12 5 Don't Know 2 5 3 5 2 5 4

3 2 4

37 24 48

23 25 22

22 29 16

11 15 6

4 6 4

Average

35

27

23

The People Subsystem

Statements 2 (Tools, methods, and approaches), 3 (Psychological safety and trust), 4 (Learning communities), 6 (Developing and retaining staff), and 9 (Rewards and incentives) Responses cluster around "agree-neutraldisagree." They suggest (i) a varied perception of experiences, and (ii) the potential for significant improvements in the people subsystem.
Statements 1 (Reflective practice), 4 (Learning communities), 5 (New ideas, trends, and practices), 8 (Time and performance management systems), and 10 (Leadership) Responses suggest that local staff feel more positively about the people subsystem.

The People Subsystem

Statements 2 (Tools, methods, and approaches), 6 (Developing and retaining staff), 7 (Individual and team-based learning and development), 9 (Rewards and incentives), and 10 (Leadership) The highest rate of agreement for professional staff concerns Statement 10 (44%) on leadership. No professional staff strongly agree to statements 2, 6, 7 and 9. This invites further investigation.
Statements 2, 4 (Learning communities), 7, and 8 (Time and performance management systems) There is a difference of agreement of 25 percentage points and above between local and professional staff responses. This invites inquiry.

The Knowledge Subsystem

Statement 1 (Professional networks) There is widespread recognition that while knowledge is created in the minds of individuals, knowledge development thrives in a rich web of professional networks among individuals. (64% Strongly Agreed, Agreed)
Statement 3 (Opportunities for knowledge development and learning) There are creative opportunities for knowledge to be developed and shared with others by facilitating networks between individuals. (59% Strongly Agreed, Agreed) Statement 4 (Products and services) The design and delivery of products and services demonstrate how effective the organization is at applying what it has learned about the nature of good practice. (51% Strongly Agreed, Agreed)

The Knowledge Subsystem

Statement 8 (Organizational memory) The organization has a resilient organizational memory and is not vulnerable to the loss of important knowledge when staff members move to other jobs in the organization or leave. (44% Strongly Disagreed, Disagreed)
Statement 10 (After-action reviews and retrospect's) The adoption of afteraction reviews and retrospect's to learn from experience has been successful. (32% Neutral)

The Knowledge Subsystem

Statement Subject Strongly Agree 9 Agree Neutral Disagree Strongly Disagree 5 Don't Know 5 1. Inspiring vision 55 17 9

2. Learning culture
3. Formal organizational structure 4. Informal organizational structure 5. Communication system 6. Resources 7. Approach to learning 8. Planned learning 9. Emergent learning 10. Failures and unintended outcomes Average

5
4 4 2 4 3 3 2 2 4

42
55 47 34 30 38 27 35 30 39

27
23 26 24 28 29 21 31 32 26

20
14 11 27 20 18 32 20 19 19

4
3 5 8 7 4 12 4 4 6

2
2 7 4 11 7 5 9 13 7

The Knowledge Subsystem

Statement 8 (Organizational memory) Responses suggest a significant concern about loss (or potential loss) of organizational memory. This may be addressed by the introduction of exit interviews in offices and departments and personal commitment to pass on knowledge and experience to improve individual and collective learning.
Statement 5 (Systems and infrastructure) Balanced responses suggest that further investigation should examine what systems and infrastructure for knowledge management need to be developed, better understood, and made more effective. Statement 6 (Evaluations) Responses suggest the need for further investigation.

The Technology Subsystem

Statement 1 (ICTs for knowledge management and learning) There is a thorough and shared understanding of the value of information and communication technologies for knowledge management and learning. (54% Strongly Agreed, Agreed)
Statement 2 (ICTs as facilitator) Information and communication technologies facilitate but do not drive or constrain knowledge management and learning in the organization. (63% Strongly Agreed, Agreed) Statement 3 (Learning communities) Information and communication technologies are successfully used to create and sustain learning communities. (52% Strongly Agreed, Agreed)

The Technology Subsystem

Statement 4 (Corporate developments) Information and communication technologies are successfully used to keep people informed and aware of corporate development. (74% Strongly Agreed, Agreed)

Statement 8 (Internal sources of expertise) Information and communication technologies are successfully used to enable people to identify internal sources of expertise. (31% Neutral)

The Technology Subsystem

Statement 9 (Creative use) Creative use of information and communication technologies is high. At least five of the following have been successfully adopted: shared document drives, intranet pages, online communities and networks, wikis, and databases, staff profile pages, online webinars, podcasts, and social network mapping. (54% Strongly Agreed, Agreed)
Statement 10 (Opportunities) Sufficient opportunities are provided for staff members to learn how to make use of available information and communication technologies for learning and sharing. (26% Strongly Disagreed, Disagreed)

The Technology Subsystem

Statement Subject Strongly Agree 7 Agree Neutral Disagree Strongly Disagree 6 Don't Know 4 1. Its for knowledge management and learning 2. Its as facilitator 3. Learning communities 4. Corporate developments 47 26 10 6 4 8 57 47 66 24 25 16 9 17 6 2 4 3 3 3 1

5. Connections
6. Innovation and creativity 7. Good practices 8. Internal sources of expertise 9. Creative use 10. Opportunities

4
4 4 4 7 5

45
38 44 38 46 43

27
29 30 31 21 23

13
18 15 17 17 19

3
6 3 5 3 7

7
5 4 5 4 3

Average

47

25

14

The Technology Subsystem

Statement 3 (Learning communities) Responses suggest a very positive assessment of the contribution of technology to learning communities. Statement 4 (Corporate developments) Responses suggest a very significant recognition by respondents of the value that technology has in keeping them informed about corporate developments. Personnel who are aware of their place in the wider organization are generally considered to be more likely to contribute their knowledge for the collective good.
Statement 8 (Internal sources of expertise) Responses suggest that there is a need for greater use of technology for internal peer support.

Subsystem Comparison
Highest rate of agreement (74%) concerns Statement 4 (Corporate developments): ADB had made successful use of information and communication technologies to keep people informed and aware of corporate developments (Technology Subsystem).
Highest rate of disagreement (44%) concerns Statement 8 (Organizational memory): ADB has a resilient organizational memory and is not vulnerable to the loss of important knowledge when staff members move to other jobs in the organization or leave (Knowledge Subsystem).

Subsystem Comparison
Highest rate of "don't know" responses concerns
Statement 10 (Failures and unintended outcomes): Failures and unintended outcomes are the focus of constructive discussions leading to new approaches (13%, Organization Subsystem). Statement 6 (Evaluations): Evaluations are carefully designed with learning (as well as accountability) in mind. Systems ensure that the outputs of internal and independent evaluations are made widely available; carefully examined; and used to influence decision making and planning, question orthodox thinking, and trigger creativity and innovation (11%, Knowledge Subsystem).

Statement 10 (After-action reviews and retrospect's): Adoption of after-action review and retrospect's to learn from experiences has been successful (13%, Knowledge Subsystem).

Overall Results

Perceptions of Staff on ADB as a Learning Organization

Overall Results
Feedback from staff members on each statement deserves dedicated attention.
Among the four subsystems, staff members perceive ADB to be most competent in the technology subsystem need more improvement in the people subsystem Based on the average percentage of respondents who collectively agree (Strongly Agree and Agree) to the 10 statements Technology Subsystem ranks first (53%) Organization Subsystem ranks second (48%) Knowledge Subsystem ranks third (43%) People Subsystem ranks fourth (38%)

Concluding Remarks
The survey response rate of 9% is acceptable (given likely survey fatigue in ADB) and compares reasonably with good feedback for an online survey (10%).
Some departments exhibit high response rates that demonstrate what is possible when there is participant interest and management encouragement. Using a five-point Likert scale, a significant number of "strongly agree" and "agree" responses are required to balance "neutral," "disagree," and "strongly disagree" responses. It is therefore highly unlikely that any organization can achieve a mean score of 4 or the ideal score of 5. The survey mean scores per subsystem are all above 3 (the score that represents "neutral"), which indicates a somewhat favorable yet uncertain view of ADB's capacities.

Concluding Remarks
The value of a survey increases when there are two or more data sets to be compared. It is recommended that the Learning for Change Survey be repeated annually. Investigations are needed on what is likely to encourage higher response rates: feedback of findings from the survey, sharing evidence that the survey has led to actions being taken, and management encouragement to participate.
References are invited to Knowledge Solutions (http://www.adb.org/site/knowledgemanagement/knowledge-solutions), the Knowledge Management and Learning Series, and the Learning for Change Primers.

Responses by Department
Department Position Not Indicated Administrative Staff National Officers Professional Staff No. of Respondents No. of Staff Response Rate

BPMS
COSO CTL CWRD DER EARD ERD

0
0 0 0 0 0 0

5
3 2 2 1 1 1

0
0 2 0 1 1 2

1
1 2 4 1 12 1

6
4 6 6 3 14 4

149
82 157 267 27 62 63

4
5 4 2 11 23 6

IED
OAS OCO OCRP

0
0 0 0

4
3 0 1

1
12 1 0

4
3 2 0

9
18 3 1

48
147 37 5

19
12 8 20

OGC
OIST OSPF

0
0 0

4
0 1

0
2 1

3
2 1

7
4 3

57
118 4

12
3 75

Responses by Department
Department Position Not Indicated Administrative Staff National Officers Professional Staff No. of Respondents No. of Staff Response Rate

PARD
PSOD RSDD SARD SEC SERD SPD

0
0 0 0 0 0 0

2
2 8 18 5 26 5

0
3 5 0 2 4 3

2
5 14 12 5 32 5

4
10 27 30 12 62 13

89
91 126 314 33 128 51

4
11 21 10 36 48 25

TD
TRANS
Not Indicated

0
0 5 5

2
0 0 96

2
0 0 42

0
1 0 113

4
0 5 256

93
36

4
3

TOTAL

Knowledge Management Center

Olivier Serrat
Principal Knowledge Management Specialist Knowledge Management Center Regional and Sustainable Development Department Asian Development Bank
knowledge@adb.org www.adb.org/knowledge-management www.facebook.com/adbknowledgesolutions www.scribd.com/knowledge_solutions www.twitter.com/adbknowledge