System of Systems Building Project: Data Work group meeting 11/16/12

The first data group meeting of System of Systems building project was held on 11/12/2012 between 10:00am to 11:30pm. We had a total of 24 participants including the projects 4 core members. The meeting started with a brief overview of System of Systems overview by Tom Sheaffer. Tom briefly talked about the projects intended goals and also went over the major components of Citys OST programs and their respective funding streams. This was followed by a round of introductions where participants also talked briefly of their respective roles in their agencies and their interests as it relates to this project. This was followed by Gulal leading a discussion around a variety of topics as listed below. Input was sought and graciously provided by participants and that is listed below in red.

1. Overview and benefits of Logic Model. Where are you going? How will you get there? What will show that youve arrived? Logic model defined*: Logic models are a concise way to show how a program is designed and will make a difference for a program's participants and community. On one sheet of paper, a logic model summarizes the key elements of your program, reveals the rationale behind your approach, articulates your intended outcomes and how they can be measured, and shows the cause-and-effect relationships between your program and its intended outcomes.
* from Harvard family research project

Benefits of logic model*: Focus on and be accountable for what matters OUTCOMES Provides common language Makes assumptions EXPLICIT Supports continuous improvement Promotes communications

* Child trends

2. Illustration of Logic model as it pertains to this project (This will be shared soon and we will continue to work on it as we move along this project). One of the components of logic model being data and its significance for good outcomes, the rest of the discussion lead to topics related to data. 3. How do you define data? According to Merriam-Webster: Factual information (as measurements or statistics) used as a basis for reasoning, discussion, or calculation
Defined instance of something Information Concrete recording of what happened History of happenings Number of regular events Things that are not visible are not data What is not data/what is not being measured Well defined categories To know what is going on at point in time. To see trendsover time data points To make decisions For funding support To see what services are provided To determine whether or not programmatic benchmarks are being met. Examples include: # of participants served, the amount of services received and so on. Learn about the characteristics of the population served and the type of services they are receiving (this requires individual level data as opposed to aggregate level data used to assess whether benchmarks are met) Advocacy work Accountability Outcomes Program Development Public policy Professional development

4. Why do we collect data?

5. Based on your own unique experiences what data do you think is not getting collected currently in your respective programs or at OST systems level?
Impact related data Subjective data like surveys Impact to family of the kids attending OST programs Costs to city for kids not part of After school

Client information to see if we are serving the right kids Data from places without means and or infrastructure to collect data Parent feedback (should collect emails to disseminate info) Means to determine which services impact which kids (no size fits all)

6. Not influenced by any particular type of MIS system (so that your thinking is not constrained by what a particular type of software can or cannot do), what kind of data and reports do you envision to get out of the MIS system assuming it has been running for few years?
It should be web-based Should have query tools so that anyone accessing the system can get the data and manipulate it in a way that fit their needs. Should have correlation ability Be able to track data over multiple periods of time Should allow input of data in context of the program Should have ability to generate reports based on user needs The system should have data elements (or field names) that are intuitive.

Data related topics that we will be encountering and may lead to creation of sub-groups:
7. Data Governance (Quality, Consistency, usability, security and availability of information)
Data Quality (can we attest that the data we have is accurate?) i. What are the characteristics of bad or poor data quality? 1. Story and data dont match 2. Not consistent ii. Impact of poor data quality. 1. Can be misleading and result in pursuing wrong goals 2. Lost opportunities (time spent on getting data that is not important means spending less time on tracking things that are important. 3. Poor data leads to collection of more poor quality data iii. What constitutes good Quality data? 1. Valid - the data accurately represents what you are trying to measure. For instance, the numbers of people that graduate don't necessarily represent good data on what has actually been learned. 2. Reliable - the data are reproducible. Repeated assessment yields the same data. 3. Authentic - the assessment simulates real-life circumstances.

4. Relevant - the data answers important questions and is not generated simply because it is easy to measure. 5. Effective - the data contributes to improving teaching and learning. 6. Meaningful data is an important tool for program staff 7. Data should be able to tell us what is happening and it should be not cumbersome to collect. iv. Tools to use for maintaining good data quality: 1. Auditing 2. Cleansing 3. Implement Business rules (includes data validation checks at the point of data entry ) v. Data synchronization challenges (especially when data is integrated across multiple systems)

Most of the above data areas fall under the general purview of data Governance: ( which includes Quality, Consistency, usability, security and availability)
b. Data Analysis (Internal and external) c. Data ownership (Includes developing or reviewing data sharing agreements) d. Data dictionary (defining a common dictionary is critical for clean data collection across the board) e. Online Data Directory a. Using data to map needs and supply Justin Ennis from ASAP gave an overview of their newly revised online directory and explained some of the features including its ability to be more interactive and have GIS capability. The new version is slated to be soft launched in January 2013.

Identified Work Groups and associated tasks: (Please choose a group or groups that you would like to be part of:)
1. Create a self-inventory of providers current data collection strategy(to gauge where everyone is as part of data collection effort) o I am in the midst of preparing a draft and will soon be sending that out. 2. Develop shared measures and outcomes (synchronization with the outcomes group) o Identify data needs by working closely with Outcomes group led by Lorraine Mcgirt 3. Development of data dictionary including exploring data systems integration. 4. Information sharing agreements 5. Features of the primary MIS (including looking at RFIQ responses from the vendors) 6. Universal Reports that appeal to all stakeholders

We are currently working on a group of reports that were recently identified for PCAPS and those will be shared with an intent to add more to that list.

Next data work group is tentatively scheduled for January 18th (Friday) 2013, between 10:00 AM thru 11:30 AM.