This document discusses different types and sources of data used in research. It covers primary and secondary data, methods of data collection including observation, surveys, experiments and qualitative methods. It also discusses the limitations of different data collection methods and sources. Some key points include:
- Primary data is collected specifically for the research purpose, while secondary data is collected for other purposes but relevant to the research.
- Common data collection methods include observation, surveys, experiments and qualitative methods. Each method has limitations such as inability to answer why questions.
- Sources of data can be internal company records or external published sources. Secondary data provides benefits but must be evaluated for accuracy and relevance.
- Qualitative and quantitative methods differ
This document discusses different types and sources of data used in research. It covers primary and secondary data, methods of data collection including observation, surveys, experiments and qualitative methods. It also discusses the limitations of different data collection methods and sources. Some key points include:
- Primary data is collected specifically for the research purpose, while secondary data is collected for other purposes but relevant to the research.
- Common data collection methods include observation, surveys, experiments and qualitative methods. Each method has limitations such as inability to answer why questions.
- Sources of data can be internal company records or external published sources. Secondary data provides benefits but must be evaluated for accuracy and relevance.
- Qualitative and quantitative methods differ
This document discusses different types and sources of data used in research. It covers primary and secondary data, methods of data collection including observation, surveys, experiments and qualitative methods. It also discusses the limitations of different data collection methods and sources. Some key points include:
- Primary data is collected specifically for the research purpose, while secondary data is collected for other purposes but relevant to the research.
- Common data collection methods include observation, surveys, experiments and qualitative methods. Each method has limitations such as inability to answer why questions.
- Sources of data can be internal company records or external published sources. Secondary data provides benefits but must be evaluated for accuracy and relevance.
- Qualitative and quantitative methods differ
To understand the primary and secondary types of data used in research
the various methods of data collection, their applicability, and limitations
how to anticipate and safeguard against the kind of errors that infect data
Data the quantification of tangible and intangible facts Data may be classified
- By the purpose of collection : 1. Primary Data : Data collected specifically for the purpose of providing information on the decision under question
2. Secondary Data: Data important for the decision being examined, but not collected specifically for questions related to it: socio-economic trends, industry trends, government policies or inc- company data including earlier research reports
Sources of Data cont. - By the nature of the variables studied : 1. Data related to past or future behaviour 2. Data related to attitudes, perceptions, beliefs, etc.
Both types of data required for understanding current market-trends and for predicting future trends
Sources of Data
Methods of Data Collection All research requires secondary data. Most require primary data in addition. Data collection methods depend on the research approach: exploratory, descriptive or causal. Extensive studies require more than one research approach at different stages. Method of collecting primary data:* Observation * Surveys * Experiments * Qualitative methods
Types of Surveys Unstructured(Open ended- qns.)
Degree of structure Semi- structure(some qns. open - ended )
Outdated Data Sampling Errors Missing-Response Response Errors Inappropriate Samples Errors Inaccuracy Ambiguity Guestimates Non-response Inaccuracy Ambiguity Twisting of Data Not-at-home Time Question Privacy Length Prestige Question Conflict Form with Question investigator wording Investigator Unwillingness Trans- lation
Limitations of Data Types of Data
Methods of Data Collection Limitations Secondary Data
Information may relate to a different period and may not be relevant any longer.
Sample ,or data base may be different Rigour in collecting data cannot be vouched for. contd.
Primary Data
Observation
Surveys
Experiments Cannot answer the questionwhy
Scope for rationalized responses
Quality of data will vary with the skill of the interviewer
Cannot tap into issues that are part of the subconscious mind.
Can study only one dependent variable at a time. Limitations of Data
Qualitative
Make assumptions that are not always sustainable in reality
Difficult to replicate and therefore to generalize from.
Results may be vitiated by environmental variables and competitive activity
Difficult to generalize
Quality of data highly dependent on the skill of the interviewer Limitations of Data TABLE 6.1: LIMITATION OF VARIOUS DATA COLLECTION METHODS contd. Benefits of secondary data Resource advantage Accessibility of data accuracy and stability of data
Assessment of data
Evaluation of secondary data Methodology check Sampling consideration Methodology of data Analytical tools used and subsequent reporting Accuracy check Topical check Classification of data Internal sources of data Company records Employee records Sales data (cash register receipt, salespersons call records ,sales invoices) Financial records External data sources Published data
Published data sources Government sources (census data , government publications) Other data sources Books and periodicals Guides Directories and indices Standard non-gov statistical data Computer stored data Reference database Syndicated data sources On-line database CD-ROM database Institutional syndicated data (Nielson ,Prowess IMRB) Qualitative methods of data collection Direct (non-disguised) Observation Focus group Depth interviews Content analysis Indirect (disguised- only disadvantage is privacy issue) Projective Association ,completion ,construction, expressive ,choice Sociometry new
Qualitative vs quantitative data method Research objective Need to explore and describe Study is quantified and needed suitable analysis Research design Exploratory or descriptive ,loosely structured and open ended Structured and measurable set of variables Sampling plan Manageable small sample vs large representative sample
Continued...... Data collection Data collection in depth and collected through more interactive and unstructured approach Data collected is formatted and structured Data analysis Textual and usually non statistical Interpretation through statistical methods Research deliverables Intial and ultimate objective to explain findings Findings are conclusive and demonstrate clear indications
Chapter Designing Questionnaires and Interview Guides
Objectives To Understand
the importance of a well-designed questionnaire
the funnel approach
the categories of questions and how to choose between them
mail, telephone, and Internet surveys how to collect data from in-depth interviews What to look out for when reproducing a questionnaire
Introduction A questionnaire acts as the means of communication between the respondent and the researcher. Clear and concise definition of the goals of the study make it easier to design the questionnaire. contd. The extent to which the subject matter as well as the structure of the questionnaire interests and involves the respondent. The funnel approach must be used for this purpose.
The length of a questionnaire
The appropriateness of a question: the flow chart approach and similar methods minimize errors in data collection.
The purpose of the study: this influences the format and structure of the questionnaire.
Factors Influencing Questionnaire Design
23 Questionnaires in Clinical Research Much of the data in clinical research is gathered using questionnaires or interviews. The validity of the results depends on the quality of these instruments. Good questionnaires are difficult to construct; bad questionnaires are difficult to analyze. Difficult to design for several reasons: Each question must provide a valid and reliable measure. The questions must clearly communicate the research intention to the survey respondent. The questions must be assembled into a logical, clear instrument that flows naturally and will keep the respondent sufficiently interested to continue to cooperate. 24 Quality aims in survey research Goal is to collect information that is: Valid: measures the quantity or concept that is supposed to be measured Reliable: measures the quantity or concept in a consistent or reproducible manner Unbiased: measures the quantity or concept in a way that does not systematically under- or overestimate the true value Discriminating: can distinguish adequately between respondents for whom the underlying level of the quantity or concept is different 25 Steps to design a questionnaire: 1. Write out the primary and secondary aims of your study. 2. Write out concepts/information to be collected that relates to these aims. 3. Review the current literature to identify already validated questionnaires that measure your specific area of interest. 4. Compose a draft of your questionnaire. 5. Revise the draft. 6. Assemble the final questionnaire.
26 Step 1: Define the aims of the study Write out the problem and primary and secondary aims using one sentence per aim. Formulate a plan for the statistical analysis of each aim. Make sure to define the target population in your aim(s).
27 Step 2: Define the variables to be collected Write a detailed list of the information to be collected and the concepts to be measured in the study. Are you trying to identify: Attitudes Needs Behavior Demographics Some combination of these concepts Translate these concepts into variables that can be measured. Define the role of each variable in the statistical analysis: Predictor Confounder Outcome 28 Step 3: Review the literature Review current literature to identify related surveys and data collection instruments that have measured concepts similar to those related to your studys aims. Saves development time and allows for comparison with other studies if used appropriately. Proceed with caution if using only a subset of an existing questionnaire as this may change the meaning of the scores. Contact the authors of the questionnaire to determine if a smaller version of the instrument exists that has also been validated. 29 Step 4: Compose a draft [1]: Determine the mode of survey administration: face-to-face interviews, telephone interviews, self-completed questionnaires, computer-assisted approaches. Write more questions than will be included in the final draft. Format the draft as if it were the final version with appropriate white space to get an accurate estimate as to its length longer questionnaires reduce the response rate. Place the most important items in the first half of the questionnaire to increase response on the important measures even in partially completed surveys. Make sure questions flow naturally from one to another. 30 Compose a draft [2]: Question: How many cups of coffee or tea do you drink in a day? Principle: Ask for an answer in only one dimension. Solution: Separate the question into two
(1) How many cups of coffee do you drink during a typical day? (2) How many cups of tea do you drink during a typical day? 31 Compose a draft [3]: Question: What brand of computer do you own? (A) IBM PC (B) Apple Principle: Avoid hidden assumptions. Make sure to accommodate all possible answers. Solution: (1) Make each response a separate dichotomous item Do you own an IBM PC? (Circle: Yes or No) Do you own an Apple computer? (Circle: Yes or No) (2) Add necessary response categories and allow for multiple responses. What brand of computer do you own? (Circle all that apply) Do not own computer IBM PC Apple Other
32 Compose a draft [4]: Question: Have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very often Principle: Make sure question and answer options match. Solution: Reword either question or answer to match. How often have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very Often
33 Compose a draft [5]: Question: Where did you grow up? Country Farm City Principle: Avoid questions having non- mutually exclusive answers. Solution: Design the question with mutually exclusive options. Where did you grow up? House in the country Farm in the country City 34 Compose a draft [6]: Question: Are you against drug abuse? (Circle: Yes or No) Principle: Write questions that will produce variability in the responses. Solution: Eliminate the question. 35 Compose a draft [7]: Question: Which one of the following do you think increases a persons chance of having a heart attack the most? (Check one.) [ ] Smoking [ ] Being overweight [ ] Stress Principle: Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item. Solution: Which of the following increases the chance of having a heart attack? Smoking: [ ] Yes [ ] No [ ] Dont know Being overweight: [ ] Yes [ ] No [ ] Dont know Stress: [ ] Yes [ ] No [ ] Dont know 36 Compose a draft [8]: Question: (1) Do you currently have a life insurance policy? (Circle: Yes or No) If no, go to question 3. (2) How much is your annual life insurance premium? Principle: Avoid branching as much as possible to avoid confusing respondents. Solution: If possible, write as one question. How much did you spend last year for life insurance? (Write 0 if none). 37 Step 5: Revise Shorten the set of questions for the study. If a question does not address one of your aims, discard it. Refine the questions included and their wording by testing them with a variety of respondents. Ensure the flow is natural. Verify that terms and concepts are familiar and easy to understand for your target audience. Keep recall to a minimum and focus on the recent past. 38 Step 6: Assemble the final questionnaire [1]: Decide whether you will format the questionnaire yourself or use computer-based programs for assistance: SurveyMonkey.com Adobe Live Cycle Designer 7.0 GCRC assistance At the top, clearly state: The purpose of the study How the data will be used Instructions on how to fill out the questionnaire Your policy on confidentiality Include identifying data on each page of a multi-page, paper- based questionnaire such as a respondent ID number in case the pages separate.
39 Assemble the final questionnaire [2]: Group questions concerning major subject areas together and introduce them by heading or short descriptive statements. Order questions in order to stimulate recall. Order and format questions to ensure unbiased and balanced results. 40 Assemble the final questionnaire [3]: Include white space to make answers clear and to help increase response rate. Space response scales widely enough so that it is easy to circle or check the correct answer without the mark accidentally including the answer above or below. Open-ended questions: the space for the response should be big enough to allow respondents with large handwriting to write comfortably in the space. Closed-ended questions: line up answers vertically and precede them with boxes or brackets to check, or by numbers to circle, rather than open blanks. Use larger font size (e.g., 14) and high contrast (black on white). 41 Enhance response rate When writing questions and assembling the final questionnaire, edit with a view towards saliency: apparent relevance, importance, and interest of the survey to the respondent Consider either pre-notifying those in your sample or sending reminders to those who received the survey (if self-administered). Studies have shown that making contact with the sampled individuals increases the response rate. If possible, offer an incentive. 42 Non-responders Understanding the characteristics of those who did not respond to the survey is important to quantify what, if any, bias exists in the results. To quantify the characteristics of the non- responders to postal surveys, Moser and Kalton suggest tracking the length of time it takes for surveys to be returned. Those who take the longest to return the survey are most like the non-responders. This result may be situation-dependent. 43 Conclusions You need plenty of time! Design your questionnaire from research hypotheses that have been carefully studied and thought out. Discuss the research problem with colleagues and subject matter experts is critical to developing good questions. Review, revise and test the questions on an iterative basis. Examine the questionnaire as a whole for flow and presentation. 44 References Designing Clinical Research, Stephen B. Hulley, et. al. www.statpac.com/surveys Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients, Health Technology Assessment, 2001. Vol.5, No. 31. Moser CA, Kalton G. Survey methods in social investigation. 2 nd ed. Aldershot: Gower; 1971. Questionnaire Design lecture, Theresa Scott
45 Answer key [1]: Question: How many cups of coffee or tea do you drink in a day? Principle: Ask for an answer in only one dimension. Solution: Separate the question into two
(1) How many cups of coffee do you drink during a typical day? (2) How many cups of tea do you drink during a typical day? 46 Answer key [2]: Question: What brand of computer do you own? (A) IBM PC (B) Apple Principle: Avoid hidden assumptions. Make sure to accommodate all possible answers. Solution: (1) Make each response a separate dichotomous item Do you own an IBM PC? (Circle: Yes or No) Do you own an Apple computer? (Circle: Yes or No) (2) Add necessary response categories and allow for multiple responses. What brand of computer do you own? (Circle all that apply) Do not own computer IBM PC Apple Other
47 Answer key [3]: Question: Have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very often Principle: Make sure question and answer options match. Solution: Reword either question or answer to match. How often have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very Often
48 Answer key [4]: Question: Where did you grow up? Country Farm City Principle: Avoid questions having non- mutually exclusive answers. Solution: Design the question with mutually exclusive options. Where did you grow up? House in the country Farm in the country City 49 Answer key [5]: Question: Are you against drug abuse? (Circle: Yes or No) Principle: Write questions that will produce variability in the responses. Solution: Eliminate the question. 50 Answer key [6]: Question: Which one of the following do you think increases a persons chance of having a heart attack the most? (Check one.) [ ] Smoking [ ] Being overweight [ ] Stress Principle: Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item. Solution: Which of the following increases the chance of having a heart attack? Smoking: [ ] Yes [ ] No [ ] Dont know Being overweight: [ ] Yes [ ] No [ ] Dont know Stress: [ ] Yes [ ] No [ ] Dont know 51 Answer key [7]: Question: (1) Do you currently have a life insurance policy? (Circle: Yes or No) If no, go to question 3. (2) How much is your annual life insurance premium? Principle: Avoid branching as much as possible to avoid confusing respondents. Solution: If possible, write as one question. How much did you spend last year for life insurance? (Write 0 if none).
Designing the Questionnaire
The process of questionnaire design depends on The management dilemma Definition of the marketing research problem
The research design: the sample, the specific interview approach and the medium of data collection including type of questions to be asked
The process of questionnaire design: Designing the Questionnaire
Contd. The process of questionnaire design:
Definition of MR Problem Hypotheses Objectives / Areas of Enquiry Definition of Sample Decision about Data-Collection Approach Flow -Chart for Questions Identification Questions Research Questions Classificatory Questions
Errors in Questionnaire Design and the Flow Chart Questionnaire design must consider:
Type of questions : structured vs. unstructured; closed- vs. open-ended; scales
Order of questions: the funnel approach from general to specific
The use of a flow-chart helpful for ensuring the order of questions
Content of questions: governed by
a) the purpose of the question, b) the extent to which it needs to be disguised, c) the scope of the question d) the wording of the question -- respondents ability and willingness to answer it.
Errors in Questionnaire Design and the Flow Chart
.contd. Pre-testing the questionnaire: - administration of the questionnaire to a small sample prior to its finalisation - purpose: to check for suitability of the type of questions, their order, content, and length - sample: 15 for a simple questionnaire; 25 for a complex questionnaire Questionnaires for Telephone and Internet Surveys
Factors to be taken into consideration: i. use structured questions as far as possible ii. frequent use of graphics, specially for attitudinal/ perception related questions iii. extensive use of multiple-choice or dichotomous questions iv. short questions v. limited use of why questions vi. brief questionnaire .contd. Questionnaires for Telephone and Internet Surveys Advantages of non-personal survey methods: i. Anonymity that provides privacy ii. The ability to answer at leisure iii. Lower per unit cost than personal interviews Disadvantages of non-personal survey methods: i. Low response rates in the absence of control ii. Biased sample because of non-response from those not interested
Projective Techniques and Interview Guides
Projective techniques useful when respondent unwilling or unable to share information. In-depth interviews require interview guidelines that do not include formal, structured, complete questions, but only a list of the areas of enquiry. The interview guideline used in projective techniques and in- depth interviews aims at obtaining information on all relevant issues, without being restricted to a predetermined order or structure of questions. The interview may at times require going back and forth on one issue, thus repeating inquiry through differently worded questions each time. Projective techniques particularly useful for interviewing children : retain their interest and encourage spontaneity.
Physical quality and visual appeal secondary but impact making characteristic of the interviewing instrument . Good production and paper quality important. Error free typing and design essential. Accurate and easy-to-understand translation crucial in multi- lingual countries like India.
Engendered Orange-Fleshed Sweetpotato Project Planning, Implementation, Monitoring and Evaluation: A Learning Kit. Volume 5: Workshop Evaluation, PAPA and Annexes