Evaluation of the Determining Instructional Purposes (DIP) Training Program

A Proposal Submitted to Far West Laboratory for Educational Research and Development by GTS Educational Services
Introduction Far West Laboratory for Educational Research and Development (FWL) recently released a request for proposals (RFP) to evaluate its Determining Instructional Purposes (DIP) training program. This proposal response is respectfully submitted by Globe Trotting Scientists (GTS). FWL developed the DIP program to provide training in educational administration skills to school administrators and graduate students and is focused on the planning and development efforts of new school programs and initiatives. An evaluation to assess the value of the training package to FWL customers will be conducted. Information gleaned from the evaluation will facilitate both FWLs internal future planning and decision-making and that of their prospective customers. Determining Instructional Purposes training program Determining Instructional Purposes is a modular based training program that is categorized into units based on instructional objectives as follows: Unit 1 - Setting Goals Unit 2 - Analyzing Problems Unit 3 - Deriving Objectives The training units are flexible enough to be used independently and can be conducted as 1-2 day workshops or longer individual sessions. Each unit contains 4-6 print-based instruction modules featuring both required reading materials that discuss the new skills being introduced, and independent and group-based activities that are designed to put the skills into practice. An accompanying coordinators handbook is available and a training coordinator is recommended to administer the sessions. Team activities feature hypothetical planning teams that exercise their new skills as they work to solve problems and develop new programs in their simulated school system. The complete training time for all three units is estimated at 32-48 hours.

Evaluation Method Developing recommendations for the most effective dissemination of the training materials and an associated strategy to engage primary users (school administrators and graduate students) on its benefits will require direct engagement with the various stakeholders. Several procedures utilizing associated data sources that will comprise this evaluation of the DIP training program are reviewed below: - Purchasing History Review and Analysis Sales data on past customers of the DIP program will be provided by FWL. Purchasing trends and demographics will be measured from the three most recent years of available data. Purchase details, including specific training units purchased, usage patterns for instructional materials, training group characteristics, and presence or absence of a coordinator, will be measured with a survey tool and phone interviews by GTS staff. - Recent Customer Inquiry Data FWL will provide customer product information requests from non-purchasing school districts in the past year. These potential customers will be surveyed to identify the key factors in their decision making process. Data will be collected through telephone interviews conducted by GTS staff. - Opinion survey of training participants A training participant (administrators, graduate students, coordinators) opinion survey will be administered to a minimum of forty customers (obtained from historical sales data) by GTS. Feedback on instructional modules and associated activities will be sought. Participants will also be asked about the current format (print) of the training materials and other delivery format options. - Follow up opinion interviews with survey participants Interviews with at least fifteen selected survey respondents (participants or school district stakeholders) will be conducted. These interviews, either via telephone or webconferencing technology, will seek to measure the value and importance placed in the training content. The interviews will also query for detailed information on what delivery format they view as most desirable for their situation. - School district stakeholders opinion survey To better understand the perceived value and effectiveness of the instructional materials, a survey of at least twenty school district stakeholders (administrators and program coordinators) will be conducted. Survey participants will be identified from historical sales data.

-Expert Review GTS will employ a subject matter expert to review all DIP training materials. This review will help identify areas where the instruction may benefit from updates, especially in the flexibility of delivery methods. FLW will provide a full set of training materials to GTS for this review process.

Task Schedule Evaluation activities and milestones are detailed on the task schedule on page 4. Dates shown are approximate. The final completion date will be negotiated between Far West Laboratory for Educational Research and Development, and GTS.

Project Personnel Dr. Benton C. Quest- A senior evaluation specialist and owner of Globe Trotting Scientists Educational Services, Dr. Quest will oversee all aspects of the DIP evaluation, including the planning, implementation, and reporting. Dr. Quest has extensive education, including a PHD in Assessment of Research and Applied Archeology from the University of Arizona. He worked as a research scientist for the US Government for ten years, before the launch of GTS. Dr. Hadji Singh- A native of Calcutta, Dr. Singh studied psychology and mathematics at the University of Florida. After serving out his term with the U.S. Marines, Dr. Singh came to work as a measurement analyst at GTS. He will be developing the surveys, checklists and phone interview protocol. Data collection and data analysis will also be supervised by Dr. Singh. Additional evaluation specialists include Race Bannon from Intelligence One, who will oversee the expert review of instructional materials among other duties. GTS clerical staff will also provide support for the DIP evaluation.

TASK SCHEDULE Evaluation of the Determining Instructional Purposes (DIP) Training Program Task Respon- Completion sible Date* Party FWL August 1, 2011 August 15, 2011 September 1, 2011

1) Evaluation contract awarded to GTS.

2) Meet with FWL to review GTS evaluation proposal, receive requested changes, and finalize. 3) Provide data collection plan for target groups. Provide draft surveys to FWL for input. Provide interview questions/format to FWL for input. 4) Receive comments and requested changes to data collection plan, survey instruments, and interview questions/format. 5) Final revisions to collection plans, survey instruments, and interview protocol to FWL. 6) Provide GTS with historical sales data for past 3 years, including customer information. Provide GTS with contact information for potential customers from product information requests. Provide GTS with complete set of all program materials. 6) Survey data collection and interview sessions from participants, potential customers, and stakeholders completed. Attend DIP training session, if available. Complete expert review of all DIP training materials. 7) Compile, summarize and analyze survey and interview data. Present initial findings to FWL 8) Receive feedback, comments, from FWL on data results and analysis. 9) Final evaluation report submission to FWL

GTS

GTS

FWL

September 15, 2011 October 1, 2011 October 15, 2011

GTS

FWL

GTS

December 15, 2011

GTS

January 15, 2012 January 31, 2012 March 1, 2012

GTS

GTS

* Completion dates are approximate. Actual dates to be determined upon award of evaluation.

Budget and Payment Schedule We are proposing a budget of $43,000 for the Determining Instructional Purposes training program evaluation services. Budget details are outlined below:

Expense Personnel

Description Payment for evaluation director, measurement analyst, evaluation specialists, and secretarial staff includes all supplies, mail, phone and network charges, office supplies, and required travel fees.

Cost $40,000

Operations

$3,000

Total

$43,000

Requested Payment Schedule Payment Due Payment 1: Upon completion of evaluation task #4 Payment 2: Upon completion of evaluation task #6 Final payment: Upon completion of evaluation task #9 (project completion) Total Amount $14,000 $14,000 $15,000 $43,000