You are on page 1of 33

Running head: EFFECTIVENESS OF PROFESSIONAL DEVELOPMENT MODELS 1

The Effectiveness of Professional Development Models

on School Success and Student Learning

Kaitlyn Valis & Sean Kinney

Loyola University Maryland


EFFECTIVENESS OF PD MODELS 2

Abstract

Professional development plays an important role in the careers of educators and can have a vital

impact on the success of schools and student learning. Professional development models vary

from face to face learning opportunities to online courses to Professional Learning Networks

(PLN). Teacher evaluations and observations are also a form of professional development that

allows for reflection and professional growth. Their effectiveness depends on a variety of

variables that must be taken into consideration. Online learning environments allow for

flexibility and personalization, PLNs give continued collaboration and support, and evaluations

allow for reflection on teaching practices. This paper evaluates, analyzes, and synthesizes

research related to professional development and draws conclusions about the different models

as it relates to teacher perceptions and students learning. Overall, a more personalized

professional development plan that is purposeful about reflection and incorporates online

learning opportunities is the most effective for school success and student learning.
EFFECTIVENESS OF PD MODELS 3

Introduction to Educational Leadership Issue:

Educators play a key role in student learning and their continued education, training, and

support is vital to their professional growth and improvement. With the growing requirements for

continuing education and the constantly changing field of education, finding professional

development that works for a variety of teachers at different levels and disciplines is vital to the

improvement of schools and the success of students. Collaboration and the sharing of ideas

amongst teachers is important for student learning and schools are looking for opportunities and

ways to encourage this amongst teachers. Motivation for professional growth comes from

intrinsic motivation and mandates from administration and education systems (McMillan,

McConnell, & O'Sullivan, 2016). When professional development involves more discipline

specific topics and incorporates personal choice the perceptions of teachers is much more

positive, teachers confidence improves, and teaching practices improve (Carpenter, 2016;

Fishman, et al., 2013). When teachers attitudes are more positive there are more continued and

sustained growth and improvement that occurs.

A variety of professional development models have been implemented and studied for

teachers perceptions of effectiveness and the implications for student learning and school

achievement. Face-to-face Professional Development, Online Professional Development (OPD),

Personal Learning Networks (PLN), teacher evaluations, peer partnerships, and conference are

all evaluated as possible options for models for effective professional development. Online PD

opportunities have been shown to be positive alternatives to face to face learning opportunities

(Marrero, Woodruff, Schuster, & Riccio, 2010). Teacher evaluations and observations,

particularly when seen as formative, allow for reflection and self-evaluation (Chester, 2012;

Delvaux, et al., 2013). Overall, the more individualized and personalized the learning
EFFECTIVENESS OF PD MODELS 4

opportunities is the more skills are developed, the more positive the perceptions of the teachers,

and the more positive effect on student learning (Owen, 2014; Shaha & Ellsworth, 2013).

Statement of Problem:

Schools today face a variety of limitations when it comes to supporting teachers learning

and professional growth. Limitations on time and budgets restricts the resources available to

teachers and schools. With hours of preparation, teaching, supervising, and grading teachers

often have limited time and energy to engage in their own professional growth. On an individual

basis, teachers already struggle to find time for their personal lives and families, so even if they

are motivated to improve their own teaching practices they often struggle to do so because of

lack of time and energy. Also, in person PD opportunities are sometime not local and travel can

be a barrier to attending these opportunities. For these reasons teacher perceptions of

professional development are often negative. Furthermore, school leadership struggles to find

time in the calendar for school-wide professional learning. Courses, conferences, travel, and

consultants often cost extra money which many teachers and schools dont have. Also, when

teachers miss school for PD there is the added cost for substitutes and the disruptions when the

teacher is out of the classroom.

Fortunately, in recent years with the use of technology and other creative solutions a

variety of other professional development models have emerged as alternative solutions to these

limitations. Teacher evaluations have also been used to support professional growth and

improvement. The collaboration and resources of Personal Learning Networks (PLN) have been

encouraged as a way for teachers to grow and learn in a more collaborative and individualized

way. The effectiveness of different models of professional development are evaluated to give

insight into which models should be utilized by teachers and school leadership.
EFFECTIVENESS OF PD MODELS 5

Research Question:

What models of professional development for faculty members are effective for school success

and student learning?

a. Are PLN (Personal Learning Network) effective for improving a teacher's skills and

student learning?

b. Are online training opportunities as effective as face-to-face professional development?

c. Are teacher evaluations an effective form of professional development?

d. What are the effects of teacher professional development on student achievement?

Significance of the Review:

Professional development is a part of every educator's life. Classroom teachers, school

administrators, and the entire school community use some form of professional development to

enhance their teaching skills so that students will be successful in the classroom. There is more

of a variety of professional development opportunities now than ever. Online professional

development has become more popular in recent years, while face to face professional

development remains a popular form of PD. The review of the research was meant to find what

forms of professional development led to achievement for school success and student learning.

This research is important because professional development should guide teachers and

administrators on a path towards student and school achievement. Knowing what types of

professional development are successful, is something that administrators and all staff involved

in the education of students, should be aware of.

Key Terms: Professional Development, Personal Learning Network (PLN), Continuing Learning

Network (CLN), Online Professional Development (OPD), Unconferencing, (VLPD) Virtual

Learning and Professional Development


EFFECTIVENESS OF PD MODELS 6

Review of the Literature:

#1 Name/Citation Kassner, L., & Virginia Commonwealth University, Metropolitan


Educational, Research Consortium. (2014). Opportunities to personalize
teacher learning: Innovative approaches to bridge evaluation and
professional development for continuous improvement. Online
Submission.
http://files.eric.ed.gov/fulltext/ED552977.pdf

What was studied? Seven different innovative approaches for professional development and
training are evaluated with the purpose of finding ways to bridge the
divide between evaluation and professional development.

What methodology The method that was used was to describe each of the seven innovative
was used? approaches, summarize other literature about them, identify strengths
and weaknesses of the approach, and make a recommendation for
implementation.

How was the data This was a literature review that consolidated research from various
analyzed? studies to provide positive and negatives for each approach. Prior
research was evaluated and synthesized.

What did the There is a wide range of options for effective professional development,
researcher (s) but there needs to be clear goals and purposes when deciding which
find? form is most effective for a particular teacher and school. Each type has
its contributions and limitations when it comes to effectiveness for
improvement in teaching practices.

What conclusions There needs to be a balance of formative and summative assessments of


did the researcher teachers. Its important that the methods of evaluating teachers are not
(s) make? only focused on job security, etc. These should be separate and
balanced. Professional development approaches should be reflective and
should be purposeful. Each of the different innovative approaches have
strengths and weaknesses based on the purpose. There is a variation in
approaches from face-to-face to computer based, high tech to low tech,
and internal and external insight.

#2 Name/Citation Lin, S., Cheng, W., & Wu, M. (2015). Uncovering a connection
between the teachers' professional development program and students'
learning. Journal of Education and Practice, 6(23), 66-74.
http://files.eric.ed.gov/fulltext/EJ1079018.pdf
EFFECTIVENESS OF PD MODELS 7

What was studied? Student learning and achievement was evaluated based on the Readers
Theatre Teaching Program (RTTP). There are three main questions that
are being asked. Is teachers knowledge about this method of teaching
improving? Are they implementing these methods into their teaching?
Are students scores improving on Reading Fluency Tests?

What methodology Two grade/veteran teachers and their classes (total 69 students) were
was used? evaluated based on pre-PD interview, pre and post-tests of knowledge,
teacher interviews, surveys, classroom observation, and students
scores. The teachers were also asked to keep a journal and create a
portfolio.

How was the data The teachers pre and post-test percentages for various aspects of the
analyzed? teaching model were displayed in 4 graphs for 4 different aspects of the
teaching model. It is clear that the post test scores were higher. The
authors looked at progress percentages to evaluate student learning on
pre and post-test. Overall, 94% of progress percentages were positive.

What did the Based on the data there was a gain in the teachers knowledge of
researcher (s) practices from the PD, and the participants had a positive attitude
find? toward this type of instruction. Most of the students showed large
progress in reading fluency. Participants had average gain of 35.75%
in overall content knowledge as measured by these pre/post exams (p.
73).

What conclusions Overall the participants had a positive attitude about the RT instruction
did the researcher and agreed that it would have a positive effect on their instructional
(s) make? practices. They also found that it helped students meet their learning
goals. The barriers were a lack of time for implementation.

#3 Name/Citation Pierson, M., & Borthwick, A. (2010). Framing the assessment of


Educational technology professional development in a culture of
learning. Journal of Digital Learning in Teacher Education, 26(4), 126
131.
http://files.eric.ed.gov/fulltext/EJ893870.pdf

What was studied? The article studied different variations of educational technology
professional development (ETPD) and argued that there needs to be
more intentional research regarding ETPD that informs further
professional development opportunities.
EFFECTIVENESS OF PD MODELS 8

What methodology This article was an evaluative article that looked at other articles and
was used? research to summarize their ideas.

How was the data There was no specific data that was analyzed. This article evaluated and
analyzed? synthesized other research done related to educational technology
professional development.

What did the The researchers found that there is more focus on accountability and
researcher (s) data driven approaches to education, although when it comes to ETPD
find? there is little to no effective feedback gathered other than teacher
perceptions of PD opportunities. They also argue that we must view
evaluation as an inseparable component of ongoing teacher action (p.
127).

What conclusions The researchers concluded that we should assess the effectiveness of
did the researcher ETPD through three different models, TPACK, organizational learning,
(s) make? and participant research and inquiry. It was found that PD is most
effective when they focus content and pedagogical knowledge, so it
would suggest that the TPACK model (which combines content,
pedagogy, and technology) would work to evaluate and assess ETPD
opportunities. Also, they concluded that organizational learning is
important to developing and evaluating professional development
opportunities. We need to shift the focus from the technology tools
themselves to the teachers and their developmental needs, and in doing
so, they addressed the uniqueness of learners who participate in any
professional development session (p. 128). The individual needs
should be taken into account with respect to the needs of the whole
organization. Successful educational technology PD initiatives are
characterized by an expanded, informed, and connected view of
learning on both the individual and the organizational level (p. 128).

#4 Name/Citation Ross, C. R., Maninger, R. M., LaPrairie, K. N., & Sullivan, S. (2015).
The use of twitter in the creation of educational professional learning
opportunities. Administrative Issues Journal: Education, Practice, and
Research, 5(1), 55-76.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=eric&AN=EJ1062476&site=eds-
live&scope=site

What was studied? This study sought to examine how educators are using Twitter to
increase their professional learning opportunities.
EFFECTIVENESS OF PD MODELS 9

What methodology A wide range of professionals in the education field were surveyed. The
was used? survey was done through Twitter by following a hashtag.

How was the data The data was analyzed using a mixed method of a quantitative survey
analyzed? and qualitative interviews.

What did the Anytime/Anywhere learning for professional development is becoming


researcher (s) more popular with educators. Educators are using Twitter more now
find? than in the past for professional development opportunities.

What conclusions Administrators should allow for more opportunities for staff to use
did the researcher Twitter for professional development, but that more studies need to be
(s) make? done on its efficiency.

#5 Name/Citation Shaha, S. H., Glassett, K. F., & Copas, A. (2015). The impact of teacher
observations with coordinated professional development on student
performance: A 27-state program evaluation. Journal of College
Teaching & Learning,12(1), 55-64.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=eric&AN=EJ1050979&site=eds-
live&scope=site

What was studied? The impact of professional development coupled with observations
having a positive impact on student achievement in math and reading.

What methodology 292 school in the US with 15 teachers in each school were studied. The
was used? teachers participated in a wide range of professional development
opportunities.

How was the data A quantitative analysis showed that the combined effects of teacher
analyzed? observations and PD wherein the use of PD was often or always focused
on observed areas of teacher need matched with appropriate PD usage.

What did the Schools where observations occur more frequently saw an increased
researcher (s) growth in reading and math scores. Both sets of schools saw an increase
find? in scores associated with observations and PD. There is no way to tell if
improvements in teacher efficacy in student performance were more
related to the observations or to PD.
EFFECTIVENESS OF PD MODELS 10

What conclusions This study shows that a coordinated approach involving PD


did the researcher recommendations and executions is impactful for student advances.
(s) make? This study shows that a well-designed observation approach, combined
with a proven online, on-demand PD, provides a rigorous, differentiated
link for enabling teachers to have their greatest impact on students.

#6 Name/Citation Carpenter, J. P. (2016). Unconference professional development:


Edcamp participant perceptions and motivations for attendance.
Professional Development in Education, 42(1), 78-99.
doi:10.1080/19415257.2015.1036303
http://www.tandfonline.com.ezp.lndlibrary.org/doi/pdf/10.1080/194152
57.2015.1036303

What was studied? One EdCamp event was studied which had 118 educators that took
place in the US in Spring 2014. The focus of the study based on the
research questions were, Why did participants attend and what were
their perceptions of the experience?

What methodology Two brief surveys were used to gather qualitative data about the
was used? experience. The surveys included open ended questions and one Likert-
scale item. The survey was created based on prior EdCamp experience
and were edited by educators familiar with the EdCamp experience. The
data was gathered by using one online survey prior to the event and
another survey after the event. The researchers also identified
limitations to their methodology including the non-random and limited
sample size of the survey data. Also, they caution readers to compare
EdCamp experiences to other PD because of its drastic difference in
structure of the events and the voluntary participation from participants.

How was the data The data was analyzed first based on participation rates of the first
analyzed? survey and the second survey. They looked at participation based on
years of experience, type of educator, and content area. With regard to
the responses they were read and coded based on patterns and themes.
The data was sorted based on codes and analyzed. The codes were then
reconsidered and the code sets were revised.

What did the The majority of different types of educators said that their reason for
researcher (s) attending was due to a recommendation from a colleague, specifically
find? 43% of responses were from recommendations. Other popular reasons
for attending were a desire to learn and grow, as well as, their curiosity
about the unique format of the PD experience. Another key reason for
EFFECTIVENESS OF PD MODELS 11

attending was the opportunity for networking with other educators. The
researchers found that 85% of the respondents rated the experience as
very good or excellent and 91% indicated that they would participate in
future EdCamps. One of the major strengths of the event based on
responses was the positive atmosphere and the energy and excitement
that they experienced. And another equally high strength as the
autonomy of each participant to choose what they wanted to learn and
move between sessions (p. 90).

What conclusions In general, participants were interested and excited to attend the event
did the researcher and were positive about the experience at the end. Participants valued
(s) make? the opportunity to engage in collaborative discussions on topics of their
choice and network with other motivated educators (p. 92). The
researchers were concerned that the energy could not be sustained and
that participants would learn new practices at a surface level but lacked
the deep knowledge and support necessary to successfully enact the new
practice in their classrooms (p. 93). The researchers note that these
EdCamps have the potential to re-engage educators as learners and
encourage further professional development. At this point there is not
enough data and information to assess the impact on schools and student
learning. It was also suggested that those organizing formal PD
opportunities should consider incorporating aspects of the EdCamp into
the formal PD experiences.

#7 Name/Citation Shaha, S. H., & Ellsworth, H. (2013). Predictors of success for


Professional development: Linking student achievement to school and
educator successes through on-demand, online professional learning.
Journal of Instructional Psychology, 40(1-4), 19-26.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=cin20&AN=107866066&site=eds
-live&scope=site

What was studied? Year to year changes in student achievement in 734 schools where
professional staff used a structured online PD. The research questions
were whether schools with a higher use of PD experienced a greater
impact on student achievement than those with lower use of PD. Other
questions were, is viewing professional development alone as strong a
predictor of success and impact as other metrics of educator utilization,
participation, and engagement? And is there a model or framework for
EFFECTIVENESS OF PD MODELS 12

predicting maximum impact from educator utilization, participation, and


engagement in on-demand or online teacher development applications?

What methodology This was a quasi-experimental study contrasting pre-PD versus post-PD
was used? and participation versus non-participation and how it changed student
performance.

How was the data The data analyzed was for only the top and bottom quartiles: the highest
analyzed? quartile of schools (higher engagement schools) versus the lowest
quartile schools (lower engagement schools)

What did the Higher engagement schools outperformed lower engagement school in
researcher (s) most areas being tracked in the study.
find?

What conclusions More research needs to be done and over a longer period of time.
did the researcher Demographics and socioeconomics needs to be considered as well. The
(s) make? type of PD used should also be looked at and studied in future research.

#8 Name/Citation Owen, H. (2014). Putting the PLE into PLD: Virtual professional
learning and development. Journal of Educators Online, 11(2).
http://files.eric.ed.gov/fulltext/EJ1033266.pdf

What was studied? They study looked at longitudinal research that evaluated the
effectiveness of the design of the Virtual Professional Learning and
Development (VPLD) program that was begun by the New Zealand
Ministry of Education in 2009. They studied the opinions of the
participants, the effectiveness of working with a virtual mentor, and the
effects on the participants teaching practices and therefore student
achievement and engagement.

What methodology This was a longitudinal study that identified patterns over time.
was used? Specifically, they used three online surveys per year, they recorded
discussions during virtual mentoring meetings, they gathered data from
webinar sessions, and other contributions to the online community
(blogs, forums, videos, etc). A more qualitative approach was used to
gather information from the open-ended survey results.

How was the data The data was analyzed by using Excel to organize and interpret the data.
analyzed? For the qualitative approach, recurring words were organized by themes
and coded. Comparative methods of analysis were used during coding
EFFECTIVENESS OF PD MODELS 13

(p.10). Two stories of change were shared to represent consistent


trends across the VPLD programs. The two stories that were shared
showed different disciplines as well as school level.

What did the Based on one of the stories that was shared the participant saw a shift
researcher (s) in her own professional identify and practice, as well as the influence
find? she was having across her own context and beyond (p. 17). The key
outcomes from this particular story were improvement in student
achievement, improvement in students metacognition skills, and a shift
in teaching/mentoring practices. In each of the stories their goals and
plans were clearly established and evidence was given for meeting each
goal. For the second story the overall outcomes included improvement
in students assimilation/demonstration of key competencies, increased
involvement of families and wider community, and a significant shift in
teaching practices.

What conclusions The researchers concluded that the individuals who participated in the
did the researcher VPLD, developed a sense of self-efficacy that motivated them to trial
(s) make? alternative approaches (p. 23). The community was found to be vital
for getting and sharing advice as well as resources. They also concluded
that the program scaffolded opportunities for learners (teachers and
leaders) to develop their own capability (skills, knowledge) (p. 24).
What worked best that was unique to this program was the virtual
mentors support as well as the combination of individual guidance and
community to share ideas and gather resources. The flexibility of the
virtual community was also key to its success.

#9 Name/Citation McMillan, D. J., McConnell, B., & O'Sullivan, H. (2016). Continuing


professional development - why bother? perceptions and motivations of
teachers in Ireland. Professional Development in Education, 42(1), 150-
167.
http://www.tandfonline.com.ezp.lndlibrary.org/doi/pdf/10.1080/194152
57.2014.952044

What was studied? The study focused on the motivating and inhibiting factors relating to
teachers engagement with CPD (continuing professional
development) (p. 152). It specifically looked at teachers in Northern
Ireland and the Republic of Ireland and aimed to gain knowledge for
future plans related to professional development in these two area.

What methodology The methodology for this study was a mixed methodology approach
was used? that combine quantitative and qualitative data. There were two stages
EFFECTIVENESS OF PD MODELS 14

for this study. The first stage involved a collection of data from focus
groups that informed the creation of a questionnaire. The questionnaire
focused on motivation factors and barriers to engagement. The
participants were qualified teachers currently in masters level
postgraduate courses.

How was the data A total of 220 questionnaires were given using an online survey and a
analyzed? total of 112 questionnaires were completed and returned. After some
were eliminated 34% of the questionnaires were completed and
analyzed. Analysis of the quantitative data gained from the
questionnaire survey was completed using SPSS version 16 and
descriptive and inferential statistics were obtained (p. 155).

What did the The findings were categorized into three areas - personal, school-
researcher (s) related, and system-wide. The results first highlighted that the
find? responsibility of professional development falls most on the teacher
before any overall educational organization. Personal choice was shown
to be the most popular motivator for participating in professional
development. Teachers intrinsic motivation to seek out their own CPD
continues to apply to actually engaging in CPD (p. 157). The top three
personal motivating factors were the potential for growth,
advancement, and achievement (p. 158). In the school level
interpersonal relations that supported teachers participating in CPD was
the most motivating factor. Also a positive relationship between teacher
and administration went a long way when encouraging professional
development. The salary/financial incentive factor was found to be one
of the lowest motivator which reinforces the personal intrinsic
motivated highlighted above. The most effective system-wide motivator
was those PD opportunities that were mandatory and were held during
school hours.

What conclusions The main conclusions from this study were that the most effective
did the researcher systems for CPD incorporate personal choice, charges schools with
(s) make? providing empowering communities of practice and develops an
overarching system of compulsory professional development (p. 164).
There must be flexibility and autonomy in implementing these CPD
opportunities and the motivating factors should be taken into
consideration when providing opportunities and deciding what is
mandatory.

#10 Name/Citation Vu, P., Cao, V., Vu, L., & Cepero, J. (2014). Factors driving learner
Success in online professional development. International Review of
EFFECTIVENESS OF PD MODELS 15

Research in Open and Distance Learning, 15(3), 120-139.


http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=eric&AN=EJ1033095&site=eds-
live&scope=site

What was studied? This study examined factors that contributed to the success of online
learners in an online professional development course.

What methodology Online surveys and learners activity logs were used to collect data. A
was used? triangulation process of collecting information from different sources
using a variety of methods reduced the risk that conclusions would
reflect systematic biases and allowed a broader understanding of the
studys issues.

How was the data A descriptive qualitative analysis of the activity logs and online surveys
analyzed? were used to analyze the data.

What did the Having self-discipline, school administrator expectation, and


researcher (s) ability to learn with limited support were factors that led to successful
find? online professional development. Most who completed the PD were
between 25-34 yrs old.

What conclusions Further research should be done to take into account other cultures and
did the researcher school administrators expectations.
(s) make?

#11 Name/Citation Tucker, P. D., Stronge, J. H., Gareis, C. R., & Beers, C. S. (2003). The
efficacy of portfolios for teacher evaluation and professional
development: Do they make a difference? Educational Administration
Quarterly, 39(5), 572-602. doi:10.1177/0013161X03257304.
http://eaq.sagepub.com.ezp.lndlibrary.org/content/39/5/572.full.pdf+ht
ml

What was studied? This study looked at the effectiveness of portfolios as an evaluation of
teachers performance for accountability and professional development.
The questions that they are addressing are if portfolios are valid
assessments of performance, are they useful to teacher evaluation
systems, what are perceptions of teachers and administration, and do
they contribute to professional growth? The study looked at three
different perspectives: the contents of the portfolios, the opinions of
teachers and administrators, and summative evaluation results (p. 578).
EFFECTIVENESS OF PD MODELS 16

What methodology The method was a mixed design of qualitative and quantitative
was used? strategies. Data collections included portfolio review and analysis,
surveys, focus groups, and archival record analysis (p. 578). The study
looked at WJC County school district in Virginia who have had
portfolios as a part of their teacher evaluation process for multiple
years. This study looks specifically at 3 years of implementation. The
researchers reviewed 24 portfolios of a random stratified sample that
represented different levels of school and years of experience. Surveys
were distributed to 606 teachers and 23 administrators.

How was the data Descriptive statistics were used to summarize the results of the
analyzed? portfolio analyses, the survey questionnaire, and the archival record
analysis (p. 582). Qualitative results were gathered using a content
analysis which focused on analyzing the information gathered from the
focus groups and open-ended comments.

What did the Overall, the researchers found that portfolios were able to represent
researcher (s) teaching responsibilities well that are required for the evaluation
find? process. Specifically, 90% of all artifacts selected and included by
teachers had content validity as they relate to teacher responsibilities.
They also found that some teaching responsibilities were more easily
represented by the portfolios than others. Artifacts for professional
responsibility and assessment were documented more than artifacts for
management and instruction. In terms of perceptions about the portfolio
process, administrators gave higher ratings as compared to teachers.
The highest ratings by both administrators and teachers were in the
areas of fairness and accuracy (p. 588). Furthermore, there was no
definitive answer to whether portfolios contributed to professional
growth. Teachers slightly valued the self-reflection and assistance in
identifying areas of strength and weakness, but did not agree that they
could improve teaching and instructional practices.

What conclusions Specifically related to professional development, the results were not
did the researcher definitive about the impact of portfolios. Teachers valued the self-
(s) make? reflection piece, but did not see how this could relate directly to
teaching strategies. More guidelines and support practices are necessary
for the portfolios to be a valued part of teacher professional
development. Overall, the study found that portfolios contributed
positively to teacher evaluation and accountability, but research and
guidelines are necessary to improve professional development and
positively impact teaching practices. The study did find that there was
EFFECTIVENESS OF PD MODELS 17

evidence of authentic evaluation of teachers and better self-evaluation


and professionalism after engaging in the portfolio process.

#12 Name/Citation Marrero, M. E., Woodruff, K. A., Schuster, G. S., & Riccio, J. F.
(2010). Live, online short-courses: A case study of innovative teacher
professional development. International Review of Research in Open
and Distance Learning,11(1), 81-95.
http://files.eric.ed.gov/fulltext/EJ881579.pdf

What was studied? In this study live, online, interactive short-courses are evaluated to
determine their effectiveness as a professional development tool
specifically for professional growth and classroom application. The
courses were designed by NASA Office of Education and the NASA
Explorer Schools (NES) initiative.

What methodology Each NASA short-course is made up of 4-6 one hour sessions with
was used? follow-up activities between sessions. Educators had the ability to
interact live with other teachers and professionals while having the
convenience of working from home or school. Participants were self-
selected educators from NES communities. Data was gathered using
questionnaires, reflective essays, personal communication, and field
notes. Most of the information was qualitative data. The questionnaires
allow the researchers to gather information about teachers feelings
and impressions about the strengths and limitations of this approach to
professional development while also collecting quantitative data to
assess their general beliefs about the programs efficacy (p. 85).
Reflective essays were only completed by about 10% of the
participants, those of which were also receiving graduate credit.

How was the data The data was exported from the online survey tool to an Excel
analyzed? spreadsheet and the Likert and yes/no responses were separated to
calculate percentages. The data was coded based on themes and then
each themes frequency was calculated.

What did the The researchers found that 79.6% of respondents reported that they
researcher (s) were using what they learned in the short-course during the current
find? school year (p. 87). And 78% strongly agreed that the short-course
contributed to their professional growth. Also, 55% would say that they
strongly agree that they prefer live, online courses to online courses that
were not live. The qualitative data suggests that a common positive
theme was the interaction and collaboration with other educators. Other
EFFECTIVENESS OF PD MODELS 18

common themes from the short answers was the immediate feedback
from instructors and other professionals and the flexibility of the online
course structure.

What conclusions The researchers concluded that some of the benefits of this type of
did the researcher professional development was the availability of interacting and
(s) make? learning from experts in the field of science. Also this type of PD
promoted an environment where participants felt comfortable sharing
ideas, reflecting on best practices, and interacting with instructors,
scientific researchers, and practitioners (p. 92). The study also
suggested that giving opportunities for continued interaction supports
the advancement of online professional learning communities. Lastly,
there is need for further research on the results for teaching practices
and student learning.

#13 Name/Citation Chester, A. (2012). Peer partnerships in teaching: Evaluation of a


voluntary model of professional development in tertiary education.
Journal of the Scholarship of Teaching and Learning, 12(2), 94-108.
http://files.eric.ed.gov/fulltext/EJ978909.pdf

What was studied? This study looked at a model for peer review and professional
development which they named Peer Partnerships in Teaching. They
studied staff attitudes and perceptions as well as the impact on teaching
practices and pedagogy. The purpose for the program was to engage
academic staff in cross-disciplinary peer partnerships to enhance
reflective practice about teaching and ultimately improve teaching
quality (p. 98).

What methodology Thirty-five academic staff volunteered to participate in the peer


was used? partnership program. Surveys were completed by 18 of a participants
and three staff were in a focus group. The survey gathered quantitative
data on the focus, value, impact, and usefulness of the program.

How was the data Item responses included yes/no answers and 5-point Likert scale. Open-
analyzed? ended questions and focus groups were used to gather more qualitative
data. The data was analyzed in SPSS v.19, but the results are mainly
given descriptively because of the small sample size.

What did the All participants rated their experience as quite useful or very useful.
researcher (s) Majority of participants rated the following outcomes of the programs
find? as 4.5 or above out of a 5-point scale - promote good teaching practices,
EFFECTIVENESS OF PD MODELS 19

reflect on teaching, increased confidence, developed new strategies, and


enhanced relationships with colleagues. The open-ended questions gave
further evidence that teaching practices changed for the better. The
opportunity to reflect on teaching practices and collaborate with
colleagues were the best aspects of the program according to the
participants. The strongest significant relationships were noted
between confidence in ones own reviewing skills and using PPiT to
reflect on teaching (p. 101).

What conclusions The researchers concluded that there were benefits to the PPiT
did the researcher specifically related to focus on reflection, development of new skills,
(s) make? and collegial support (p. 102). Positive relationships were built based
on the willingness of the partners to be vulnerable with each other. It
was also suggested that another benefit was the possibility to use these
experiences for promotion and teaching award reviews. It was
concluded that formative rewards (reflection, development, and
collaboration) were much more motivating than summative rewards.
Overall the study supports the participation in peer partnerships

#14 Name/Citation Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G.,
Johnson, H., et al. (2013). Comparing the impact of online and face-to
face professional development in the context of curriculum
implementation. Journal of Teacher Education, 64(5), 426-438.
doi:10.1177/0022487113494413.
http://dx.doi.org.ezp.lndlibrary.org/10.1177/0022487113494413

What was studied? This study was to see if there was a difference in teacher and student
learning from online professional development and face to face
professional development.

What methodology This was a cluster randomized experiment where two groups of teachers
was used? and students were selected from urban, suburban, and rural areas to
learn a new curriculum. One group worked on the professional
development online at their own pace and the other group had the
professional development face to face over 6 days.

How was the data A two-level model where students at the first level were nested within
analyzed? teachers at the second level.

What did the Both groups of teachers reported increase in confidence with the new
researcher (s) curriculum. Both groups of students also learned from the curriculum
find? that was being taught.
EFFECTIVENESS OF PD MODELS 20

What conclusions Online professional development can be a viable alternative to


did the researcher traditional professional development. Depending on the time, place,
(s) make? cost, or content one mode of PD may be better than the other.

#15 Name/Citation Delvaux, E., Vanhoof, J., Tuytens, M., Vekeman, E., Devos, G., & Van
Petegem, P. (2013). How may teacher evaluation have an impact on
professional development? A multilevel analysis. Teaching and Teacher
Education, 36, 1-11. doi:10.1016/j.tate.2013.06.011.
http://ac.els-cdn.com/S0742051X13001078/1-s2.0-
S0742051X13001078-main.pdf?_tid=fd0c0324-d4ea-11e5-b9b2-
00000aacb361&acdnat=1455654323_7e85294cf9b823eadd3439efa244
af61

What was studied? This study examined which components of a teacher evaluation system
related to their effects on professional development. School leadership
characteristics and other leadership factors are also taken into account.
Teacher evaluations are evaluated from two perspectives: as
accountability which is more summative and for improvement which is
more formative.

What methodology Although the researchers could have taken more of a qualitative
was used? approach they decided to look quantitatively at a large-scale survey.
Questionnaires were used to collect data and prior literature and
interviews were used to develop different scales. All the items on the
questionnaire used a 5-point Likert scale. A sample of teachers from 65
secondary schools were asked to complete the survey and 63.3% of
teachers responded to the questionnaire.

How was the data The data was analyzed using the Tucker Lewis index, comparative fit
analyzed? index, and root mean square error of approximation. The dependent
variable was the effect of the evaluation system on professional
development. The researchers conducted a multilevel regression
analyses using MLWin 2.23.

What did the The researchers found that on average, teachers perceive little effect of
researcher (s) the evaluation system on their professional development (p. 6).
find? Although the standard deviation was high so there was a large variation
in the perceptions of professional development. The researchers also
found that teacher perceived the purpose of the evaluation as more
development-oriented rather than oriented toward controlling or
sanctioning teachers (p. 7). Also it was determined that the amount of
EFFECTIVENESS OF PD MODELS 21

experience a teacher has influenced their perception of the evaluation.


Lastly, instructional leadership and attitudes of school leadership about
the evaluations had a positive significant effect on PD.

What conclusions Overall the study showed that the effects of evaluation on professional
did the researcher development. The conclusions that were drawn by the researchers show
(s) make? that limited teaching experience, useful feedback and a positive
attitude of the principal are the most important characteristics of the
evaluation system (p. 8). Those teachers that reported seeing the
greatest effects of the evaluation system on professional development
were teachers with less than 5 years of teaching experience. And using
the evaluations for a summative purpose have a small but significant
positive impact on professional development. One limitation of the
study was the self-report questionnaire because it didnt actually
measure the effect on teaching practices and student achievement.

#16 Name/Citation de Kramer, R. M., Masters, J., O'Dwyer, L. M., Dash, S., & Russell,
M. (2012). Relationship of online teacher professional development to
seventh-grade teachers' and students' knowledge and practices in
english language arts. Teacher Educator, 47(3), 236-259.
http://dx.doi.org.ezp.lndlibrary.org/10.1080/08878730.2012.685795

What was studied? The effects of a series of online professional development workshops
on teachers' content knowledge, teachers' instructional practices,
students' content knowledge, and students' practices.

What methodology A randomized controlled trial was conducted across multiple states. 35
was used? teachers in the treatment group and 45 teachers in the control group.
Eligible students were defined as the 2,056 students.

How was the data 4-point Likert scale; missing scores were calculated using a
analyzed? stochastic linear regression model, 3 scores were calculated for each
participant for each subject area and a mean calculated. The students
scores were based on a standardized test and given a 1 for correct 0
incorrect and the mean was calculated.

What did the The research showed that the series of three workshops had a
researcher (s) statistically significant positive effect on teachers' vocabulary and
find? overall ELA knowledge and vocabulary and writing instructional
practices. Research also showed that there were statistically significant
effects on students' reading comprehension practices.
EFFECTIVENESS OF PD MODELS 22

What conclusions OPD can have both statistical and practical effects on teachers and
did the researcher students. More studies need to be explored to find out why OPD was
(s) make? effective at changing some, but not all, areas of teacher and student
knowledge and practices, and whether greater changes in teacher
outcomes may translate to changes in student knowledge and practices
over a longer time period.

#17 Name/Citation Dash, S., De Kramer, R. M., O'Dwyer, L. M., Masters, J., & Russell, M.
(2012). Impact of online professional development on teacher quality
and student achievement in fifth grade mathematics. Journal of
Research on Technology in Education (International Society for
Technology in Education), 45(1), 1-26.
http://web.b.ebscohost.com.ezp.lndlibrary.org/ehost/pdfviewer/pdfview
er?sid=0afb1033-fa2d-4406-a464-
3637d3e99ed0%40sessionmgr114&vid=37&hid=105

What was studied? This study was to investigate the effects of online professional
development in mathematics on 5th grade teachers pedagogical content
knowledge and pedagogical practices and their students mathematics
achievement.

What methodology The method used was a randomized controlled trial that was
was used? implemented in three rounds over two school years. Teachers were
randomly assigned to either a control or experimental group. Teachers
in the experimental group completed a series of three online
professional development mathematics courses, offered one course per
semester.

How was the data The data was analyzed using a repeated measures analysis of variance
analyzed? technique to ascertain whether there were significant differences in
participants pedagogical content knowledge (pre/post).

What did the Teachers in the experimental group received significantly higher overall
researcher (s) pedagogical content knowledge scores than did teachers in the control
find? group.

What conclusions The research concluded that intensive, sustained, content-focused


did the researcher online professional development in mathematics can effect positive
(s) make? change in teachers pedagogical content knowledge and pedagogical
practices.
EFFECTIVENESS OF PD MODELS 23

#18 Name/Citation Polly, D., McGee, J., Wang, C., Martin, C., Lambert, R., & Pugalee, D.
K. (2015). Linking professional development, teacher outcomes, and
student achievement: The case of a learner-centered mathematics
program for elementary school teachers. International Journal of
Educational Research,72, 26-37. doi:10.1016/j.ijer.2015.04.002.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=psyh&AN=2013-16802-
005&site=eds-live&scope=site

What was studied? This study looks at what types of PD change teaching practice and lead
to student achievement. The questions asked were, to what extent do
teachers topic coverage, emphasis on memorization and solving novel
problems, and time spent on mathematics instruction, predict student
mathematics achievement growth? To what extent does teacher
participation in content-focused professional development predict the
aspects of instruction found in our first analysis to be related to
increases in student mathematics achievement growth?

What methodology Data from the US Dept. of ED. for the LESCP, observations of students
was used? and interviews with teachers.

How was the data 4-level cross-classified growth model using MLwiN software &
analyzed? hierarchical linear models (HLM) to test the relationship between
instruction and professional development.

What did the When teachers in 3-5 focused more on advanced mathematics topics
researcher (s) and problem solving, student achievement grew more quickly. When
find? teachers focused on basic concepts and memorizing facts, student
achievement grew slower. When teachers participated in professional
development that focused on math content or instructional strategies in
mathematics (in Year 1), they were more likely to teach in ways
associated with student achievement growth (advanced concepts and
problem solving).

What conclusions In order to raise student achievement, professional development should


did the researcher be given to teachers focusing on instructional strategies and content
(s) make? especially in math - specifically advanced concepts and solving novel
problems.

#19 Name/Citation Yoon, K. S., Duncan, T., Lee, S. W., Scarloss, B., & Shapley, K. L.
(2007). Reviewing the evidence on how teacher professional
EFFECTIVENESS OF PD MODELS 24

development affects student achievement. issues & answers. REL 2007


no. 033 Regional Educational Laboratory Southwest.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=eric&AN=ED498548&site=eds-
live&scope=site

What was studied? This report reviews the research-based evidence on the effects of
professional development on student achievement. The focus is on
student achievement in three subjects: mathematics, science, and
reading and English/language arts.

What methodology The review process followed the protocol for finding relevant literature
was used? & screening from U.S. Department of Educations What Works
Clearinghouse.

How was the data The data was analyzed by narrowing down more than 1300 studies to 9
analyzed? studies focused on elementary school teachers and their students.

What did the This research finds that teachers who receive substantial professional
researcher (s) development an average of 49 hours in the nine studies can raise their
find? students achievement by about 21 percentile points.

What conclusions A larger number of rigorous studies on the link between professional
did the researcher development and student achievement might have made it possible to
(s) make? determine whether intensive, sustained, and content-focused
professional development is more effective.

#20 Name/Citation Desimone, L., Smith, T. M., & Phillips, K. J. R. (2013). Linking student
achievement growth to professional development participation and
changes in instruction: A longitudinal study of elementary students and
teachers in title I schools. Teachers College Record, 115(5), 1-14.
http://ezp.lndlibrary.org/login?url=http://search.ebscohost.com.ezp.lndli
brary.org/login.aspx?direct=true&db=a9h&AN=108455662&site=eds-
live&scope=site

What was studied? This study examined the influence of three year-long cohorts of
elementary school teachers participating in a learner-centered
mathematics professional development program.

What methodology All teacher-participants completed three pre-project instruments and


was used? post-project instruments. Three student achievement measures
EFFECTIVENESS OF PD MODELS 25

(curriculum-based unit tests) were administered to students as pre- and


post-tests.

How was the data The McNemar change test, which approximates a chi-square
analyzed? distribution, was used to see if the changes of the teacher beliefs and
practices were statistically significant before and after the professional
development.

What did the The findings of this study indicated that the CoDE:I project was
researcher (s) associated with statistically significant changes in teachers enacted
find? practices, as well as gains in teachers Mathematical Knowledge for
Teaching (MKT). There were also statistically significant relationships
between teachers mathematical knowledge for teaching post-test score
and student achievement as well teachers change in beliefs to
Discovery/Connectionist and student achievement.

What conclusions Future research should more closely examine the specific elements or
did the researcher components of professional development and how those components
(s) make? are empirically associated with the various teacher factors and their
students achievement.

Analysis, Synthesis, and Evaluation:

Professional Development for educators comes in many different forms, but two of the

main methods of PD that are prevalent in the field of education is traditional face to face learning

opportunities and virtual/online training. A variety of studies looked at the effectiveness and

limitations to both methods. Virtual or online training were in general more flexible and more

individualized to the needs of each teacher. Owen (2014) studied a virtual professional learning

and development (VPLD) program and concluded that the online community was successful in

sharing and receiving resources and suggestions. This program personalized the teacher training

by scaffolding the information so that skills were developed. The virtual mentors worked

individually with teachers to meet their professional learning needs and it benefited student

learning and achievement. One face to face model that also highlighted personalization and

choice were unconferences which are often known as EdCamps. These events, known for their
EFFECTIVENESS OF PD MODELS 26

unstructured nature, are energetic, interesting, and exciting according to participants (Carpenter,

2016). They also allow for personal choice in professional development which is key to success

and positive perceptions. These events have the potential to encourage and re-engage teachers

into their own learning and energize them to explore opportunities for their own group, but there

is no follow-up or structure for continued support so researchers are concerned that the

excitement and energy cannot be sustained.

Furthermore, the benefits of online PD were exemplified through online, interactive short

courses designed by NASA Office of Education according to Marrero, Woodruff, Schuster, &

Riccio (2010). They concluded that the online access to experts in the field of science through

the online platform was a major benefit to these online short courses that they would not

normally get in live trainings. These online courses broke down the barriers of location and

access to experts. It also established a community and network of support for other science

professionals that would be available and easy to access after the courses. Again, the flexibility

of the courses was important and the participants agreed that it contributed to their professional

growth and could apply what they learned to their current teaching. Content specific online

professional development resulted in improved pedagogical content knowledge and teaching

practices for math teachers based on a study of elementary math teachers (Dash, De Kramer,

ODwyer, Masters, & Russell, 2012). The use of Twitter for professional development is

expected to increase as educators are finding that being able to collaborate, network, and engage

in professional development through social media can be fulfilling as traditional professional

development (Ross, Maninger, Robert, LaPrairie, Sullivan, 2015). Lastly, Fishman, et al. (2013)

concluded that both face to face and online models increased the confidence of the teachers and
EFFECTIVENESS OF PD MODELS 27

improved students learning, so based on the circumstances and possible flexibility of the online

opportunities they are viable alternatives to traditional professional development.

Teacher evaluations are utilized in educational environments for two main purposes; for

accountability and professional development. Together observations and professional

development can be highly effective for student achievement. Shaha, Glassett, & Copas (2015)

suggested that a combination of observations and coordinated professional development had a

positive impact on students. The coordination between observed needs in teaching practices and

focused professional development was most productive for teacher growth. Teachers who

reported the greatest success with the evaluation system were teachers who had taught less than

five years. Delvaux, et al (2013) concluded that limited teaching experience, useful feedback,

and a positive attitude of the principal are the most important characteristics of the evaluation

system (p. 8). Teachers who viewed the evaluations as a formative experience had a most

positive impression of the evaluations as a tool for professional development.

In addition, peer partnerships and peer observations were studied for their effectiveness

for improvement on teaching skills and school culture. The study concluded that there were

benefits to the peer partnerships, specifically related to the focus on reflection, development of

new skills, and collegial support (Chester, 2012, p. 102). The non-threatening nature of the peer

partnerships allowed for vulnerability, reflection, and collaboration. Similarly, portfolios as a

form of teacher evaluation and professional development allow for reflection and personal

evaluation that leads to professional growth. Although there were no definitive results related to

the impact on professional development, teachers did value the self-reflection that the portfolios

allowed (Tucker, Stronge, Gareis, & Beers, 2003). Overall, teachers responded positively to non-
EFFECTIVENESS OF PD MODELS 28

threatening, reflective forms of evaluation and thought they would impact their professional

growth positively.

The findings in the research showed that most students showed progress in academics

when teachers were given professional development. When the professional development met

the needs of the teacher and observations done by administrators were focused on the areas the

professional development was targeting, student achievement in reading and math was improved.

Research also suggested that "higher utilizers (of professional development) achieved

significantly greater gains in student achievement" (Shaha & Ellsworth, 2013, p. 24). An

increase in students' metacognition skills was reported by Owen (2014) in a study on virtual

learning and professional development. One study by Yoon, Duncan, Lee, Scarloss, & Shapley

(2007), found that "rigorous professional development can increase student achievement by

about 21%". What this research represents is that quality professional development can have a

positive effect on all involved. Administrators who choose professional development that meet

the needs of their staff will see positive results in student achievement.

In conclusion, many articles that were evaluated were based on a combination of

quantitative and qualitative data. Much of both the quantitative and qualitative data was based

off of surveys conducted by teachers about their perceptions of different professional

development experiences. Although teacher perceptions and attitudes are important these are

subjective responses and therefore might not be the most valid data. More research is required to

determine the effectiveness of various forms of professional development on student

achievement and learning.


EFFECTIVENESS OF PD MODELS 29

Conclusion:

Choosing the best model of professional development for school success and student

achievement is important. Administrators need to decide if professional learning networks are

going to be effective for their staff and students. Administrators should also know if a face to

face or online professional development is more suitable for a given topic. Teacher evaluations

should be conducted to identify specific areas of improvement a staff member might need. The

professional development that the teacher participated in should impact student achievement in a

positive way.

Professional development, both online and face to face, has been shown to be popular

with teachers. Gains in teacher knowledge, increases in teacher confidence, and improvements in

student achievement are just a few of the many benefits of professional development. Face to

face professional development is a great way to network with other teachers and it opens doors

for future collaboration with staff. Online PD can allow for more flexibility and time for self-

reflection. What is clear is that all forms of professional development have shown to be effective

for school success and student learning.


EFFECTIVENESS OF PD MODELS 30

Matrix for Evaluating Research:

4-Exemplary 3-Accomplished 2-Developing 1-Beginning

Author (s): 1. 2. 3. 4. 5.
Kassner, Lin, Su- Pierson, Ross, Carrie Shaha, Steven
Laura ching; Melissa; R; Maninger, H.; Glassett,
Cheng, Wen- Borthwick, Robert, M: Kelly F.;
wen; Wu, Arlene LaPrairie, Copas, Aimee
Ming-sui Kimberly N:
Sullivan, Sam

Basic Information 3 2 3 2 3

Introduction 3 2 3 3 3

Literature Review 3 1 3 2 2

Methodology 1 3 1 1 2

Data Analysis 1 3 1 2 2

Conclusions/ 3 2 4 3 3
Recommendations

References 4 2 3 2 3

Impact 3 2 3 2 3

Total 21 17 21 17 21
EFFECTIVENESS OF PD MODELS 31

Author (s): 6. 7. 8. 9. 10.


Carpenter, Shaha, Owen, Hazel McMillan, Vu, Phu; Cao,
Jeffrey Paul Steven H.; Dorothy J.; Vien; Vu,
Ellsworth, McConnell, Lan; Cepero,
Heather Barbara; Jude
OSullivan,
Helen

Basic Information 3 3 3 3 2

Introduction 4 3 4 4 2

Literature Review 3 1 3 3 1

Methodology 3 2 2 3 2

Data Analysis 3 2 2 4 2

Conclusions/ 3 3 3 3 3
Recommendations

References 3 3 3 4 3

Impact 3 2 3 3 2

Total 25 19 23 27 17
EFFECTIVENESS OF PD MODELS 32

Author (s): 11. 12. 13. 14. 15.


Tucker, Marrero, Chester, Fishman, Delvaux,
Pamela D. Meghan E.; Andrea Barry; Eva;
Woodruff, Konstantopou Vanhoof, Jan;
Stronge,
Karen A.; los, Spyros; Tuytens,
James H. Schuster, Kubitskey, Melissa;
Gareis, Glen S.; Beth W.; Vekeman,
Christopher Riccio, Vath, Eva; Devos,
R. Beers, Jessica Richard; Geert; Van
Fitzsimons Park, Gina; Petegem,
Carol S.
Johnson, Peter
Heather;
Edelson,
Daniel C.

Basic Information 4 4 3 3 3

Introduction 4 3 4 3 4

Literature Review 4 2 4 3 4

Methodology 3 3 2 3 4

Data Analysis 3 3 2 4 4

Conclusions/ 4 4 3 4 4
Recommendations

References 3 3 3 4 4

Impact 3 3 3 3 3

Total 28 25 24 27 30
EFFECTIVENESS OF PD MODELS 33

Author (s): 16. 17. 18. 19. 20.


Raquel Dash, Desimone, Yoon, Kwang Polly, Drew;
Magidin de Sheralyn; Laura, Suk; Duncan, McGee,
Kramer , Teresa; Lee, Jennifer;
De Kramer, Smith, Thomas
Silvia Wen-Yu; Wang, Chuang;
Jessica Raquel; M., Scarloss, Beth; Martin,
Masters , Magidin Phillips, Shapley, Kathy Christie;
Laura M. Kristie J. R., L. Lambert,
O'Dwyer,
O'Dwyer , Richard;
Laura M.;
Sheralyn Pugalee, David
Masters,
Dash & K.
Michael Jessica;
Russell Russell,
Michael

Basic Information 3 3 3 3 3

Introduction 3 3 3 3 2

Literature Review 4 3 3 3 3

Methodology 2 3 4 3 3

Data Analysis 3 4 4 2 3

Conclusions/ 3 4 4 2 2
Recommendations

References 3 4 3 4 3

Impact 3 3 3 2 2

Total 24 27 27 22 21

You might also like