Professional Documents
Culture Documents
This sheet should be completed and signed and should accompany your research proposal
submission for the Educational Research Design module.
1
1. Manage the design, development, implementation and evaluation of a number of
appropriate eLearning resources;
2. Engage in research to evaluate the effective use of eLearning resources within a Higher
Education environment.
I understand that my work can be returned uncorrected if the criteria above Yes
have not been fulfilled.
Date 20.08.17
X
Alessio Gemma
Signature:
2
Word Count
Proposal: 3079
Appendix: 361
Introduction
The development of the internet technology has been a game-changer in the customer service
industry. On the one side, the communication channels have grown exponentially creating new
opportunities for both companies and customers. Websites, forum, emails, chats box or social
media help brands to understand their customer base while improving satisfaction, loyalty and
revenue as a result. Customers too have their benefits for they can avail of instant messaging,
multichannel and 24/7 access to solve their queries and problems. On the other side, this
abundance in data, tools, products and complexity of customer demands have increased
cognitive load and pressure on advisors.
This changing landscape has created the need for new skills and attitudes as well argued by
two key reports in the area of new job competencies such as Davies & Fidler, 2011 and Schwab
& Samans, 2016. Critical Thinking (CT hereafter) - the ability to see logical connections
between ideas, identify arguments, detect inconsistencies, visualise the way we approach a
dilemma- is key for the customer service industry and it will be at the centre of my research.
The area of Soft Skills in general and critical thinking with its application and impact on
Customer Service is of professional interest.
3
(Customer Advisors, Claims, Sales, Integration and so forth) have to apply their knowledge
and skills to effectively answer customer’s queries and achieve internal targets (key drivers as
First Call Resolution, Service Level/Time Response, Contact Quality or Customer
Satisfaction). Part of my role is the design and delivery of training solutions that help CS
workers overcoming their difficulties and meet customer expectations. Three key difficulties
that I have consistently observed in both New Hire Groups and Tenure Advisors are:
Customers do not always contact brands with simple queries like, “how to I reset my
password?” or “how much does it cost to use this service?”. Mostly they contact CS with
problems that are ill defined and require a degree of interpretation on the advisors side. Here
are some examples
Decision Making
The solution to the problem often requires thinking outside of the box, stretching the rules to
apply to new cases while preserving company core values and interest. Take the notion of
“compelling evidence” which is used to assign a reimbursement claim to either a buyer or a
seller. Compelling evidence in case of non-received items is normally a tracking number.
Intangible items however are not trackable therefore judgment must be applied in terms of what
amounts as compelling evidence. How can we decide if a screenshot sent by a customer amount
to compelling evidence or not?
Data Interpretation
4
Customer service representatives use a variety of tools including database, email macros,
process manuals, decision flows, SandBox Environment while they are talking to the customers
on the phone. Tools are not always tailored-made and effective usage requires the ability to
filter and select what is needed from what is superfluous. This is especially relevant with
knowledge databases where the same query “how do I reset a password” can return several
articles. Interpretation of data is essential.
I believe and want to prove that a training intervention focusing on CT could help CS
representatives to improve their solving skills, ability to practice judgement and effectively
filter through qualitative and quantitative data.
Training soft skill is however hard. Corporations tend to train on process/knowledge and then
attaching “off the shelf” soft skill package with little or no effect. Poor results do not justify
investments and this exacerbates the problem. A change in direction is needed. If what the
World Economic Forum reports is true, corporations will need to pay more and more attention
to the way Training Organisations support and nurture thinking and other soft skills in the
employee life cycle. This attention must focus not just on quantity but also on quality, strategy,
effectiveness. The whole topic of how we train soft skills, how these relate to core technical
product must be revalued and new design strategies explored.
I want to understand if and in what degree teaching critical thinking skills can positively affect
the areas of understanding, judgment/decision making and to a certain degree data
interpretation in Customer Service workers.
5
My research question is:
The objective is to proof that a CT intervention is worth the investment and it should be
included in standard new hire curriculum for CS roles as much as other soft skills like
communication or team working. CT should be part of the arsenal a CS representative has at
his disposal and more attention should be given to its training, transferring and testing.
Literature Review
Critical thinking literature is vast and touches on at least three areas. These are:
Two issues/questions are important for my research: the issue of definition of the CT and the
issue of how to teach or train CT
The problem of the definition of “critical thinking” has been one of the main topic in the
literature (Lajios, 2013).
6
The question is key for training critical thinking skills requires a precise understanding of what
the skills are.
Scholars like M. Scriven and R. Paul argue that the problem of defining critical thinking is the
problem of defining the standards or norms of good reasoning. The word ’’critical’’ derives
etymologically from two Greek roots: "kriticos" (meaning discerning judgment) and
"kriterion" (meaning standards). Etymologically, then, the word implies the development of
"discerning judgment based on standards" or else “judging a fact based on a set of criteria.
(Scriven, M. & Paul, R. 1996)
People who answer negatively however argue that this line of argument postpones the real issue
and question. What are these cross-subject rules? What is their epistemological foundation?
Thus, Mc Peck argues that here are only little disciplines--critical thinking of business
planning, critical thinking of physics, critical thinking of sociology, and so on--and that there
is no such thing as a general or abstract discipline of critical thinking. (McPeck, J. E. 2016)
In my research, I will not tackle these broader questions for this would require an extensive
research and literature analysis that go beyond my objectives. The definition of CT I will
endorse is the one from M. Facione who writes:
What I like about this definition is that it encapsulates two key aspects. First, the claim that the
notion is essentially normative and regulative. When a person thinks critically about something
he matches some regulative idea of thinking and reasoning adhering to rules and norms.
Secondly, the claim that critical thinking involves both skills and attitude. Critical thinking is
a practice, an activity that can be improved over time with the right training and disposition of
7
mind. As Tim Gelder writes “Critical Thinking may be difficult, but it certainly is not
impossible. The key is hidden behind the little word skill. Everyone knows that mastering a
skill takes practice, and lots of it. The skills of critical thinking are no exception.” (T. Gelder,
2005)
We understand critical thinking as a skill and disposition: something we can develop trough
practice.
The question that naturally follows is, “what is the best method to train/develop it?”
Conversely, what are the main barriers to overcome? The literature seems to agree that CT is
hard to learn and to teach and designers should pay attention to certain key enablers and
barriers. Here I will mention three focus points and what impact they have for design.
First, effective teaching requires that teachers have a clear understanding of what CT is and
how to identify and use these skills effectively.
Schumm, for instance, reminds us that if students are to learn and effectively apply critical
thinking skills, then the instructors need to not only understand critical thinking, but be able to
cultivate classrooms and environments that help promote the usage of critical thinking skills
and be able to effectively evaluate its usage (W. R. Schumm, 2006).
This point is important for it affects the role we should assign to the trainer when designing a
module on CT and overall the blend we need to reach between facilitation and instruction. We
can compare CT here with other process subject areas like Project Management. We would not
simply train a course on Project management by facilitating a session on how people manage
projects. We take into account the level of knowledge in the group but we provide additional
knowledge to modify skills and behaviours via instruction. The same can be said for CT. If
content knowledge is essential then trainer’s role cannot be limited to facilitation but must
extend to instruction.
Secondly, interaction or collaborative learning is key for critical thinking and therefore smaller
groups seem to work better than large ones (Mazer, Hunt and Kuznekoff, 2007). Anuradha, A.
Gokhale for instance carried an experiment to compare the effectiveness of individual learning
versus collaborative learning in enhancing drill-and-practice skills and critical-thinking skills
8
with interesting results (A. Gokhale, 1996). What he found was that students who participated
in collaborative learning had performed significantly better on the critical- thinking test than
students who studied individually. It was also found that both groups did equally well on the
drill- and- practice test.
This impacts design too. Successful design must include a right blend between instruction and
activity where trainees can practice and collectively build their knowledge. It is here that
technology can play a role creating the space for asynchronous learning and knowledge sharing
via chats, or groups using platform like Whats up, Slack and so forth.
Third, training CT through domain specific applications tend to be more effective than
standalone modules (Grosser and Lombard, 2008 but also Mc Peck, 2008 and Balins 1999).
The real problem is that of transfer here which is common to training soft skills in general: how
to transfer the new skills from theory to practice.
This point suggests that training must embed skills into scenario and these must be meaningful
to the trainees. Anne Helsdingen carried a research on the influence that training CT has for
decision-making (A. Helsdingen, 2011). One of the challenges she recorded was exactly that
of transfer CT skills learned in class to new cases and scenarios. To solve this problem she used
what is known as an instructional strategy called story model blending meaningful scenarios
and slotting CT skills into the complex stories. The result was that once CT practices were
analysed in the context of real stories, trainees could see how these were affecting their real
life actions and therefore were better equipped to transfer to new scenarios.
Research Design
9
Epistemology/Ontological position
The epistemic frame that subtends this research is that of constructivism understood as “the
idea that truth and meaning do not exist in some external world but are create by the subject’s
interaction with the world” (Gray, 2005, p.12). The theoretical perspective however is that of
Internal Realism understood as the view that “although our sentences do correspond to reality
they are not simply copies of reality” (Putnam, 1981, p. 123). Realism at an ontological level
is combined with an epistemological pluralist framework to explain how different epistemic
subjects can relate to the same world in different ways. Critical thinking requires our ability to
see how our ideas connect to each other but also to factual evidence. Constructivism cum
Internal Realism make better room for pluralism while still remaining faithful to the idea of
right and wrong and this suits a study on Critical Thinking.
Research Methodology
I have been debated in terms of what methodological approach to use and in the end I have
resolved to use mix methods research understood as “research that involves collecting,
analysing, and interpreting quantitative and qualitative data in a single study” (Leech N,
Onwuegbuzie, 2008, p. 13).
This choice relates to the type of data/evidence I want to collect in order to prove or disprove
that a training intervention on CT has an impact on problem solving and decision making in
CS workers:
10
Proof of learning evaluations
Analysis of key metrics such as Resolution Rate normally applied in the Customer
Service Industry
A purely quantitative method would just look at hard numerical evidence such as employee
metrics (Resolve Rate, Handling Time, Net Promote Score, Customer Satisfaction, Agent
Satisfaction and so forth) and results from testing. Of course, this type of evidence –if proving
positive- would delight management but it is hard to materialise. First considering the number
of people I am working with in the project, I would not get a satisfactory amount of data to
prove or disprove any point. Also, my experience tells me that training interventions on soft
skills do not immediately translate into hard numbers and improved metrics. Other non-
quantitative data enter into the picture like employee reactions, learning experience, fulfilment
in personal growth, modification of attitude and knowledge and so forth.
11
Hence, mixed methodology is more suitable.
Proof of Learning (Pre Content)- A pre content POL will be delivered to both Pilot and
Bridge Group to gauge participants knowledge of CT skills and also list their main
difficulties in terms of decision making, judgment and resolution.
Proof of Learning (Post Content)- A Proof of Learning using a scenario based approach
will be delivered to both Pilot and Bridge Group to measure training impact.
Focus group –A post intervention focus group will be organised to collect feedback and
ideas in terms of how effective the training has been, what could have been done
differently and ideas for enhancement and future applications.
Data analysis –Post training delivery analysis of the key driver Resolve Rate used in the
industry will be carried comparing result pre and post training after 30 days.
Ethical Considerations
I will work with a group of 20 Volunteers among my company employees. The following
precautions will be observed in order to respect ethical standards.
a) Informed consent. All volunteers will be fully aware about the objectives and goals of
this research. Appropriate consent forms will be distributed.
b) Voluntary. All volunteers will participate in the projects free from coercion and they
will be able to withdraw at any stage of the process. To guarantee freedom of
participation, request to take part will not be done by the volunteers managers.
c) Safety- Extreme emphasis is placed on critical thinking and the term has a clear
evaluative meaning. My research will include some testing and therefore it might
raise issues in terms of psychological stress. Extreme care will be used to guarantee
that no harm will result because of taking part in this project understood in both
physical and psychological terms.
d) Respect for anonymity and confidentiality- No volunteer personal identification will
be disclosed
12
Delimitations, limitations, and outline of timescale/research plan
The research will be carried over year two of my MCs in Applied eLearning.
I assume that volunteers in the test will answer pre and post training surveys with honesty and
transparency. Anonymity and confidentiality will be guarantee to help people feel free. I also
assume that there will be no mayor external factors that will affect the pilot members metrics
pre and post training such as bugs or mayor technical problems. Insurgence of bugs would of
course alter normal data. To control this I will use a pilot team and benchmark it with another
team that will not receive the training.
Summary
In this presentation, I have explained what the rationale of my research is, what my objectives
are, what I want to achieve and how I want to find my answers. I hope it suffices to give
enough details and showing enough engagement with current literature.
Appendix
An open format questionnaire will be administered to both the pilot and the bridge group
to 1) gauge their understanding of CT and how it compares to formal definitions found in
literature; 2) previous knowledge on this subject (training received), 3) explanation of how
CT might relates to their job.
13
The eLearning module will focus on basic Critical Thinking Strategies that can help
understanding a problem but also make an informed decision based on a good judgment.
Different practices used in CT to define a problem statements such as Formulating Why
Statements, Ask Focusing Questions; Examination previous attempts; Changing
viewpoints; Analysing root cause, ect will be examined.
The module will be blended with a mix of instruction/activities. To create engagement the
core tools/methods of CT investigation will be trained via a gamified approached. Users
will be prompted with problems and see how they can apply CT skills to get a solution and
formulating a sound judgment.
Training evaluation will be delivered at the end of the pilot and will focus on key learnings
(what you have learned compared to expectations), relevance to the job (how what you
have learned applies to your current job) usefulness (how what you have learn can be used
to improve your daily job) and possible changes.
The assessment will be administered to both pilot and bridge group. This will be scenario
based and it will aim to evaluate the way participants use their skills to understand, judge
and take a decision in given scenarios. A strategy known as Explanation-based decision
making will be used to assess the level of metacognition and the application of CT
strategies. Participants will be asked to solve given problems and show evidence of the
approach used in getting the solution. Results between the two groups will be compared
and divergence recorded and discussed.
Data coming from these elements together will provide evidence to establish a link between
my training intervention and improvements in the area of understanding, judgment and
data interpretation.
14
Bibliography
Facione, P. A. (1998). Critical thinking: What it is and why it counts. San Francisco:
Academic Press
Helsdingen, A. S., Van den Bosch, K., Van Gog, T., & van Merriënboer, J. J. (2010). The
effects of critical thinking instruction on training complex decision making. Human
factors, 52(4), 537-545.
Lai, E. R. (2011). Critical thinking: A literature review. Pearson's Research Reports, 6, 40-
41.
Lajos L. Brons (2013). Truth Rhetoric and Critical Thinking. Retrieved August 2017 from
http://lajosbrons.net/wp.html
Mazer, J. P., Hunt, S. K., & Kuznekoff, J. H. (2007). Revising general education: assessing a
critical thinking instructional model in the basic communication course. The Journal of
General Education. 56(3-4), 173-199.
Paul, R. (2005). The state of critical thinking today. New directions for community
colleges, 2005(130), 27-38.
Putnam, H. (1981). Reason, truth and history (Vol. 3). Cambridge University Press.
15
Schwab K. and Samans R. (2016). The Future of jobs. Genève, Switzerland: World
Economic Forum. Retrieved June from https://binged.it/2skdI7Q
Scriven, M. & Paul, R. (1996). Defining critical thinking: A draft statement for the National
Council for Excellence in Critical Thinking. [On-line]. Available HTTP:
http://www.criticalthinking.org/University/univlibrary/library.nclk
Schumm, W. R., Webb, F. J., Turek, D. E., Jones, K. D., & Ballard, G. E. (2006). A
comparison of methods for teaching critical thinking skills for U.S. Officers. The American
Journal of Distance Education. 20(1), 39-50.
Yeh, Y. C. (2009). Integrating e-learning into the direct-instruction model to enhance the
effectiveness of critical-thinking instruction. Instructional Science, 37(2), 185-203.
DECLARATION OF
All research and scholarship proposals, whether funded or not by internal or external funds, must
submit a RESEARCH ETHICS/ASSESSMENT OF RISK FORM to the DIT Research Ethics Committee.
This is a self-declaration process. The researcher is asked to formally identify any possible ethical
issues or risks that might arise in the course of the work, and to sign the documentation.
Please refer to the Guiding Principles and Procedures indicated on the DIT Research Ethics website
prior to completing this form:
16
http://www.dit.ie/DIT/graduate/ethics/index.html
PLEASE NOTE
You are requested to attach a copy of your research application to this form.
The RESEARCH ETHICS /ASSESSMENT OF RISK FORM must be signed by the applicant(s)
Ethical Approval must be granted prior to start of any research/scholarly activity or prior to
funding being released for the project, as appropriate.
No postgraduate research student will normally be registered until the proposal is cleared by
the DIT Research Ethics Committee.
Completed forms should be returned to: Research Ethics Committee, c/o Office of Graduate Studies,
DIT, 143-149 Lower Rathmines Road, Dublin 6.
Present appointment:
School/Department/Centre: DIT
Faculty: Learning Teaching and Technology Centre
Work Tel:
Fax:
E-mail: alessiogemma@gmail.com
17
Other departments/organisations/individuals involved:
a)
b)
c)
Source of Funding:
Has the current research project already received approval from another research ethics
committee?
Generic Projects:
Researchers may receive approval for a cluster of similar research activity by approval of a generic
protocol to cover repetitive methodologies or activities. A generic protocol should comprise a
covering letter setting out the circumstances and rationale for generic approval, outlining the
procedures to be followed in all such projects, in addition to completion of the appropriate
appendices.
If this project is part of a cluster of research with similar methodology, please tick here and submit
a generic protocol to cover all such projects.
Insurance
18
Normally, DIT insurance covers standard research activity, including fieldtrips. Are you aware of
any unusual or exceptional risks or insurance issues to which DIT’s insurance company should be
alerted? If so, please list the issues:
Please note that no contract should be entered into for clinical/medical (including drug testing) or
surgical trials/tests on any human subject until written confirmation has been received from the
DIT’s insurers that the relevant insurance cover is in place.
Are you or any members of the research team a member of any organisation that provides
professional indemnity insurance?
19
Professional Code of Conduct
Please reference, if appropriate, the Code of Ethical Conduct produced by your relevant
professional organization(s), which also informs your research.
Please note that: Where those requirements conflict with DIT requirements, the latter will
normally be followed. In all such circumstances, please contact the Office of Research Ethics for
clarification.
All researchers must confirm with the Data Protection Act 1988. Please consult the DIT Data
Protection Officer for advice.
20
IDENTIFICATION OF ETHICAL ISSUES AND/OR RISK
Do any of the following ethical issues or risks apply in your research? If so, tick all box(es)
which apply and complete the relevant Appendix, which can be downloaded from
http://www.dit.ie/DIT/graduate/ethics/index.html
Y
Consent and advice form given to subjects prior to their participation in the
research [Appendix 2]
N
Consent form for research involving ‘less powerful’ subjects or those under 18
years [Appendix 3]
N
Conflict of interest [Appendix 4]
N
Drugs and Medical Devices [Appendix 5]
N
Ionising Radiation [Appendix 6]
N
Neonatal Material [Appendix 7]
N
Animal Welfare [Appendix 8]
N
General Risk Assessment [Appendix 9]
N
Hazardous Chemical Risk Assessment [Appendix 10]
N
Biological Agents Risk Assessment [Appendix 11]
N
Work involving Genetically Modified Organisms Risk Assessment [Appendix 12]
N
Field Work Risk Assessment [Appendix 13]
If other risk and/or ethical issues are identified please provide a written submission which outlines the
issues and the manner in which they are being addressed.
No, there are no ethical issues and/or risks involved in your research project, please tick here,
and sign the declaration on page 5.
21
Yes, there are ethical issues and/or risks involved in your research, please tick here and complete
the appropriate forms identified above.
22
In accordance with the Principles of the Declaration of Helsinki and DIT Principles and
Procedures, I declare that the information provided in this form is true to the best of my
knowledge and judgement.
I will advise the DIT Research Ethics Committee of any adverse or unforeseen circumstances or
changes in the research which might concern or affect any ethical issues or risks, including if
the project fails to start or is abandoned.
Checklist
Please ensure the following, if appropriate, are attached:
Research Proposal y
23
Questionnaire(s) n
Advertisement/Poster
Generic Protocol n
24