Evaluation Plan: Expertise Center Education Care (EC2-Saba)

Delroy Sinclair
Walden University

Professor Mitchell Pratt

EIDT-6130 Program Evaluation
October 18, 2015

Table of Content
Program Analysis .................................................................................................................. 2
The name and type of organization that runs the program. ................................................. 2
A description of the program ............................................................................................. 2
Goals/Objectives ............................................................................................................... 3
Operations ......................................................................................................................... 3
Internal Care ...................................................................................................................... 3
External Care ..................................................................................................................... 3
A brief history of the program ........................................................................................... 4
The stakeholders involved in the program and their interests. ......................................... 4
The contextual factors that impact the program .............................................................. 4
Potential ethical challenges involved in an evaluation of this program ............................... 5
Evaluation Model .................................................................................................................. 6
Model ................................................................................................................................ 6
Evaluation Criteria ................................................................................................................ 8
Data Collection Design and Sampling Strategy ................................................................... 10
Choice of Design ............................................................................................................. 10
Sampling Strategy ............................................................................................................... 10
Population ....................................................................................................................... 10
Sample ............................................................................................................................ 11
Limitation of Designs and Sampling Strategy...................................................................... 11
Choice of items in the survey .............................................................................................. 11
Report Strategy ................................................................................................................... 13
Reference ............................................................................................................................ 16
EC2 Info Guide. (2015). A Coherent System of Care Services. Retrieved from ................... 16
Appendix 1 ......................................................................................................................... 17
Appendix 2 ......................................................................................................................... 18
1|P a ge

Program Analysis

The name and type of organization that runs the program.

The name of the organization that I choose to evaluate is the Expertise Center
Education Care- Saba. The name of the organization is abbreviated with the acronym EC2,
which is derived from the double use of the letter E and C in the name of the
organization; thus the well known name EC2. This organization is located with its main
office in the Netherlands, having sub branches in the Dutch Caribbean islands of Bonaire, St.
Eustatius and Saba; know as the BES islands. These islands are municipalities of the
Netherlands; hence the funding of this organization is directly from the Dutch government
making a public entity within the kingdom of the Netherlands. It is a three (3) tier
organization, consisting of board-management, EC2 team and administrative staff (Expertise
Center Education Care, 2012).
A description of the program

In the Netherlands the procedures for referral for extra assistance can be timeconsuming and sometimes complicated. Several committees need to access the student before
a budget can be allocated. On Saba the EC2 has been established to alleviate that problem.
The EC2 works with all school boards on the island, and as such is involved with all students
in Preschool, Primary and Secondary Education as well as Social Youth Development
Program to offer assistance and professional advice on learning problem and challenges (EC2
Info Guide, 2015). The process of EC2-Saba starts when the homeroom teacher identifies an
issue in the classroom; the homeroom teacher then seeks help from the Internal Guidance
Office (IGO). If the IGO is unable to address the problem an Internal Care team of the school
is consulted; if the problem cannot be dealt with internally, the EC2 is informed and
2|P a ge

consulted. The EC2 has a wealth of professional and expertise who aid in the development of
Individual Education Plan (IEP), Remedial/Special Education assistance, Speech Therapist,
Psychiatrist, Physiotherapist among of a wide range of other specialists/expertise.
Goals/Objectives
To strengthened internal care capacity by establishing services and a multidisciplinary
team of ambulatory experts working in cooperation with all stakeholders.

To reach the students that have specific educational needs

To offer parental guidance

Build knowledge and skills in the school teams on learning and behavior problems

Develop local care infrastructure including a Care & Advice Team which also has the
authority for placement and referral of students.
To identify students with learning and behavior issues, in order to diagnose and start

treatment and remediation (Education Care Plan Saba, 2015).

Operations
Students on the little island of Saba can be referred for an evaluation by various
qualified professionals. The legal task of the EC2 is to strengthen internal care and organize
external care.
Internal Care
The internal care takes place in the classroom by Mentors/Homeroom teachers,
Special Education/PrO Teachers, Remedial Teachers, and Teacher Assistants. The Internal
Care is coordinated by the Internal Guidance Officers.
External Care
The external care is offered by professionals outside of the school; a Psychologist,
Child/Adolescent Psychiatrist, Occupational Therapist, and Young Child Pedagogue.

3|P a ge

External care is coordinated by the EC2 Director. An Education Care Plan has been
established to outline the individuals roles who are involved in the Care System.

A brief history of the program

The Expertise Center Education Care (EC2) was established on Saba in 2011. The
EC2 was set up according to Dutch guidelines to provide education care to all children and
youth aged 0-24 years old. The EC2 serves as a support center to meet the needs of students,
teachers and parents. The island's care system has been developed and will continue in the
future as the education laws for the Dutch Caribbean are implemented (EC2 Info Guide,
2015).
The stakeholders involved in the program and their interests.

Several groups of stakeholders (see appendix 1) are involved in the overall care
system of Saba. All have their own tasks and responsibilities towards the care students for
which they are accountable. All have their own competencies and experiences to offer in
either internal or external education care. With the care system as a binding factor,
stakeholders will gain in quality while giving shape and meaning to the system over the
upcoming years (Education Care Plan Saba, 2015 p. 21).
The contextual factors that impact the program

Saba is a small island within the Kingdom of the Netherlands; geographically it is

located far from Holland. Even though the citizens are by law, Dutch citizens, similar to
citizens of the Netherlands; they often times received sub-standard treatments. The political
structure of the Netherlands is different from the ones found on Saba. Hence, financial
distributions, organization structure, red-tape, bureaucracy and standards are a few factors
that will impact the program.
4|P a ge

Potential ethical challenges involved in an evaluation of this program

EC2 most times maintain a high patience confidence, information obtained from tests
and/or referrals are not given out to external parties. Conversely, sensitive information about
students, in term of learning problems and behaviors, are given to teachers in confidence; but
unfortunately the information is leak and obtained by individuals who should not know about
it. Saba is a small island as was mention in the introduction; hence most things are not kept in
confidence. Another challenge which poses both positive and negative effect is that most of
the specialists who work with the program are from overseas. This is positive in terms of the
confidentiality of records are maintained and unbiased and unfair referrals are obtained.
However, the negative aspect to this is that most of the specialists take a very long time to
adjust to the culture of a small island, and within a few months they leave. This result in a
high rate of turnover of specialists/expertise for the island.

5|P a ge

Evaluation Model

The Expertise Center Education Care (EC2) will be evaluated to determine its impact
on students, parents and the community of Saba; in terms of its provision of educational care.
This care system will be assessed to see to what extent its services and functions have made a
difference in the behavioral, social and emotional attitudes of the people of Saba.
To perform this evaluation, an evaluation model will be selected and evaluation
criteria will be identified and outlined. The advantages and disadvantages of various
evaluation models will also be highlighted after which, a specific model or combination of
models will be chosen for the evaluation. From the evaluation model(s) identified; specific
evaluation criteria will be outlined for Expertise Center Education Care (EC2).
Model
Evaluators today are exposed to many more evaluation approaches in comparison to a
century ago (Stufflebeam, 2001). There is a new paradigm shift in program evaluation. The
development and continued growth in evaluation methods and/or approaches have been
experiencing massive attention and are influenced by several seminars. Fitzpatrick, Sanders,
and Worthen (2011) identified four (4) commonly used evaluation models or approaches,
namely; participant-oriented, decision-oriented, expertise and consumer-oriented and
participant-oriented evaluation. There is not one model or approach that is more effective
than the other, in some situations and depending on the evaluator there may be a more
preferred model that is utilized.
The advantages and disadvantages listed below are from the textbook. After
reviewing the models, I will explain which method will be used for the EC2 evaluation.
Evaluation Models
Expertise and ConsumerOriented Approaches

Advantages
evaluation by experts in the
field
rely on evaluation logic and

Disadvantages
relies almost exclusively on
expert evaluation
neutrality or objectivity of
6|P a ge

quantitative methods
Program-Oriented
Evaluation Approaches

Decision-Oriented
Evaluation Approaches

Participant-Oriented
Evaluation Approaches

use logic models and/or

program theory
easily understood, easy to
follow and implement
produces relevant conclusions

designed for use by

stakeholders (i.e. upper
management, client)
multiple stage models provide
information that helps with
decision making
includes stakeholders which
can improve validity of the
evaluation
stakeholders can help provide
context, knowledge, and
perspectives
evaluation is always a
partnership

reviewers
lack of transparency
doesn't help with identifying
how the program can be
improved
single-minded focus
over simplification, can fail to
show complexity of the
problem
can neglect stakeholders with
less power or influence
inability to address questions
or issues that clash or do not
align with primary stakeholder
credibility of results to
stakeholders that do not
participate
the feasibility of implementing
a successful participative study
possible stakeholder bias
costly in terms of time and
resources

The evaluator will be using both an objectives-based and participant-oriented

evaluation to assess EC2. This mixed method will allow me to evaluate whether the
objectives of the program are being met but also allow for stakeholder knowledge,
experience, and insight to be heard. The strengths of these models, as shown in the table
above, outweigh the weaknesses. Stakeholder involvement will be important for gathering
the needed data to evaluate the objectives of the course and the overall impact of the program
on student performance/attitude. Stakeholder involvement will not cause any issues because
the program will continue regardless of the outcome of the evaluation. The purpose of the
evaluation will be to determine the extent to which the programs offered by EC2 changed
students behavioral, social, emotional and academics performance/attitudes.

7|P a ge

Evaluation Criteria

The following criteria will be used to evaluate the impact EC2-Saba has had:
1. How has the interventions impact students, parents and teachers attitude, after being
exposed to them?
2. How do students, parents and teachers feel about the programs/intervention offered by
EC2-Saba?
3. To what extent are the specific objectives of EC2 relevant to the needs of students,
parents and teachers?
4. Are the specified objectives being achieved? If not, why?
5. Do the objectives being evaluated actually help in improving students
performance/attitude?

The focus of these questions is to determine the impact that the programs of EC2 has
had on the persons involved in the program. EC2 has implemented several of their internal
and external educational care policies and programs. Several expertises have performed
diagnostics and treatments to students (subjects); most of which have not been shared.
Internally, EC2 has been keeping records and communicating with its key stakeholders. From
a more general perspective, this evaluation will publicly share some of the happenings of
EC2.
At this time the financial sector and other external stakeholders will not be evaluated.
EC2 has been in operations for quite a while now; they have not shown any signs of financial
challenges or seem to be experiencing any financial difficulties. Other external stakeholders
seem to be pretty contented with EC2s operations. However, from a educational perspective
and concern, the educational stakeholders, parents and Saban community would like to access
the impact EC2 has had on the environment.
The following are stakeholders that will be involved in determining the evaluation
questions.

8|P a ge

Educational Institution
o Laura Lindzey Day Care Center parents and teachers of this institution
o Sacred Heart School parents and teachers of this institution
o Saba Comprehensive School students, parents and teachers of this institution
Care and Referral Team (CART)
o School Care Coordinator
o EC2 Experts
These stakeholders should be instrumental in determining the evaluation questions,
criteria, and standards as well as ongoing throughout the evaluation. EC2 will not be
evaluated to assess whether it should be closed down or go out of existence; this evaluation
will be used to determine the effectiveness of the interventions EC2 offers and how it has
changed or not changed the attitudes of the students, parents and teachers of the island, Saba.
The evaluation questions represent mostly relative standards. It is hope that through this
evaluation, the programs offered could be identified as the best interventions for the subjects
exposed to them.

9|P a ge

Data Collection Design and Sampling Strategy

Choice of Design
This evaluation will be incorporating a mixed design; it will contain some aspect of
qualitative and quantitative design. Powell (2006) expressed that quantitative method of
evaluation can reach conclusions with a known degree of confidence about the extent and
distribution of the phenomenon. In the evaluators judgment this type of mix method would
provide sufficient data to reach a valued conclusion. First hand data needs to be collected,
hence the need to utilize surveys, questionnaire and observations.
The evaluation that will be carried out is not funded by all stakeholders; therefore
using questionnaires and survey will be very cost effective and will be able to reach a larger
sample group. These instruments will be easier to collect and analyze the data captured on
them. Surveys and questionnaires will be most suitable for the age group of the experimental
and exposed control group. The students will be eager to provide answers to questionnaires
and surveys rather than other instruments. Care will need to be taken into consideration when
designing the questions for both the questionnaire and the surveys; the age of the students, the
language spoken, ethnic background and the grade level. The same would need to be taken
into account for the surveys for the teachers and parents. Students and teachers will be
questioned and involved in in-person interview surveys. Parents and members of the EC2
could be asked to fill out online surveys.

Sampling Strategy
Population
The population for this evaluation would the EC2 department members, the entire
students, parents and teachers of the three (3) schools; Laura Day Care Center, Sacred Heart
School and Saba Comprehensive School. This population will fall in the approximation of

10 | P a g e

about five hundred (500) respondents. The population covers an extensive range that would
provide adequate data to be evaluated and to make a satisfied conclusion.
Sample
Ritter & Sue (2007) posited that the choice of respondents recruited to complete your
questionnaires and/or surveys should be based on your evaluation objectives and the
identified target audience. The evaluator feels it will be more effective to split the sample
group into strata of important categories; hence implement a stratified random sampling
technique to derive the sample. Since data collected for this evaluation will be obtained from
four (4) different entities which may have different experience with the intervention; this will
be the best sampling option. The sample will consist of about forty (40) respondents spanning
over the four entities in question.

Limitation of Designs and Sampling Strategy

One of the major limitations of the design and strategy is the limited resources; as was
stated this evaluation is not funded and supported by many of the stakeholders involved as it
is a voluntary evaluation undertaken by a concerned teacher. Another limitation will be the
availability of respondents from the EC2 teams; as was stated in the program analysis most of
these expertise members live off islands and sometimes only travels to the island when the
need arises. The evaluator is expecting some bias from parents and highly fancied teachers
who are Dutch native who supports just anything as long as it has a Netherlands symbol
attached to it.

Choice of items in the survey

The choice of item on the students questionnaire and/or survey will be closed ended
questions; this is properly guiding students response and control answers received. This type of item
is also suitable for the age groups of the student respondents in the sample. Teachers and parents
11 | P a g e

survey will contain both open and closed ended questions; this is to receive additional data which the
evaluator may have overlooked and to give these respondents a chance to better express themselves.
(See appendix 2 for an audio summary of the data collection design and sampling strategy)

12 | P a g e

Report Strategy
Stakeholder

Students
Educational
Institutions

Teachers

Internal Care
Coordinator

Parents

Reporting Strategy

Implications

No need. Students are a

category of stakeholders
in this because they
benefit from program.
However, I do not
believe that they need to
be recipients of the final
report. The student
body would not
comprehend this and
would do little or
nothing with this
information.
Teacher would get this
written report via e-mail
and faculty briefings
and/or meetings. This
report would contain
key findings of the
evaluation; and
conclusions reached and
proposed solutions, if
any.

This will lead to a None

better
intervention
programs that
would cater to the
needs of the
students locally.

Will received written

report via e-mail and
will also be invited to
the meeting of the
presentation.

The parents body would

also received a detailed
written report of the
findings and proposed
solutions via e-mail and
parents meetings.
Parents would also be

If the report
shows a positive
impact on
students care
needs; then
teachers would
have to utilize
this care system
some more and
start referring
students to the
program. If it is
the opposite then
teachers would
If the evaluation
give a positive
result, then the
job that the ICC
was doing was
effective; if not
they would need
to re-invent a new
program or
modify the
existing one.

Stakeholder
Involvement

None

Assist with
written report
and
communicate
findings to
other
stakeholders.

None

13 | P a g e

EC2 Experts

Government
al Bodies

Saba Local
Government

Ministry of
Education
(Netherlands)

informed of the step

forward and what are
futures plans and
interventions, if any.
Representative will be
invited to a meeting in
which a detailed
presentation will be
made of the findings of
the evaluation. A detail
written report will be
given to them on the
evaluation plan, the data
collected and the
findings of the
evaluator.
Representative will be
invited to a meeting in
which a detailed
presentation will be
made of the findings of
the evaluation. A detail
written report will be
given to them on the
evaluation plan, the data
collected and the
findings of the
evaluator.
Received detailed
written reports of
findings via e-mail and
teleconference meetings.

These members
will use the report
to see the
shortcomings of
the program, if
any, and to make
changes with
given suggestions
so as to make the
program better.

Use to
disseminate
archive data
to make
comparison
of students
performance
in the past
and present.

Depending on the
outcome, this
body may decide
to continue
funding or do a
complete
withdrawal of this
program. If
necessary
introduce a
completely new
cars system.
Depending on the
outcome, this
body may decide
to continue
funding or do a
complete
withdrawal of this
program. If
necessary
introduce a
completely new
cars system.

Have
conference/di
alogue with
the Ministry
and share
finding
within the
report with
them.

None

Values, Standards, and Criteria:

The major focus is the impact the program has had on students care. It is
imperative to see the extent to which this care system has made any difference
whatsoever in the lives of the students. Parents and teachers are concerned
especially for the slow learner and learner who have challenges. The burden
become unbearable when teachers hands are tied and nothing can be done to
assist these students. Most classroom teachers are not trained to deal
specifically with challenged students; hence a referral system that is
implemented should be working to assist in these areas.
14 | P a g e

Potential ethical issues:

Political context could be an ethical issue that this evaluation would face. This
care system was proposed by the Dutch government to survive in an English
speaking setting. Most documents are written in the Dutch language; not only
will there be a barrier to communication, but the bias would come into play
with this system being critique by a huge population of English teachers and
students. Bias in responses would be achieved from the Dutch respondents.

15 | P a g e

Reference

EC2 Info Guide. (2015). A Coherent System of Care Services. Retrieved from
http://www.ec2saba.org/Downloads/EC2_Info_Guide.pdf on September 12, 2015.
Expertise Center Education Care. (2012). Retrieved from http://www.ec2saba.org/ on
September 19, 2015.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2010) Program evaluation: Alternative
approaches and practical guidelines (4th Ed.). Boston, MA: Pearson.
Powell, R. (2006). Evaluation research: An overview. Library Trends, 55(1), 102-120.
Retrieved from the Education Research Complete database.
Ritter, L., & Sue, V. (2007). Selecting a sample. New Directions for Evaluation, 115, 2328.
Retrieved from the Education Research Complete database
Stufflebeam, D. (2001). Evaluation Models. Retrieved from
http://www.wmich.edu/evalphd/wp-content/uploads/2010/05/Evaluation_Models.pdf
on October 01, 2015.
The Education Care Plan. (2015). A coherent system of care facilities that guarantees
interrupted development for all students. Retrieved from
http://www.ec2saba.org/Downloads/Education_Care_Plan%20Saba_2015-2019.pdf
on September 11, 2015.

16 | P a g e

Appendix 1

17 | P a g e

Appendix 2

Please click on the image above to hear the recorded audio

18 | P a g e