Running Head: BIAS IN TPC JOURNALS

Kenda McClimon ENGL 7702: Research Design Dr. Albers Quantitative Research Fall 2012

BIAS IN TPC JOURNALS

Introduction In the field of technical and professional communication, there are five popular journals that are used, which include: Technical Communication, Technical Communication Quarterly, IEEE Transactions on Professional Communication, Journal of Technical Writing and Communication, and Journal of Business and Technical Communication. These journals are used by both professionals working in the field and students in academia. Like many information sources, bias appears to exist behind what the journals produce. The research problem for this study is there is a chance that bias exists in technical and professional communication journals. While conducting research in the past, I noticed that quantitative research was much more difficult to come across than qualitative research in technical and professional communication journals. This was something worth putting to test. The research questions for this study are: are the journals listed above bias towards publishing qualitative research more than quantitative research, and do they focus more on professional studies rather than academic studies? There are two hypotheses for this quantitative study: 1. that technical and professional communication journals are bias towards publishing qualitative studies, and 2. that the journal articles publish more articles on professional topics than academic topics. The first hypothesis will be tested by measuring the ratio of qualitative to quantitative studies from every journal issue from 2007-2012 (five years). The second hypothesis will be tested using a within subjects design experiment that will be administered to both a technical writing department at a corporation and a technical and professional communication graduate department at a university. Along with the in subjects design study, the same journal issues will be counted to determine the ratio of professional to academic article topics. These hypotheses are important because these journals are among the largest influences in the field of technical and professional communication. It is vital to determine whether bias exists towards either qualitative or quantitative research, because the amounts of each should be equivalent to one another. Having a strong mix of both expands the research and learning opportunities of those who read and site the articles. Deciding whether and which journals focus on professional and academic targeted will help members of each environment narrow their focus on what they are looking for. Solving this study will supply readers with the knowledge to know where to turn for what they need. Literature Review Technical and professional communicators have long been innovators in the fields of writing and communication. Those in the field stay current with trends in technology and focus on communicating them to their audience. To learn about trends, technical and professional communicators attend conferences, undergo training, attend classes, and stay current with related literature. There are five popular journals that produce articles for this field, and each one offers the latest qualitative and quantitative studies for those in the field to read about. Technical Communication: Technical Communication is published through the Society for Technical Communication out of Fairfax, Virginia. This journal is published quarterly

BIAS IN TPC JOURNALS

and has a free subscription with membership. The typical article length is between 5,000 and 8,000 words. Technical Communication Quarterly: Technical Communication Quarterly is published quarterly with support from Taylor and Francis, the Association of Teachers of Technical Writing (ATTW), and Texas Tech University. A subscription is free when individuals join the ATTW. IEEE Transactions on Professional Communication: IEEE Transactions on Professional Communication is available through IEEE Explore Digital Library. The focus for this journal is to publish articles in the electronics fields. Journal of Technical Writing and Communication: The Journal of Technical Writing and Communication has been around for over thirty years and publishes articles on academia, industry, management, and government. The journal is produced through the Baywood Publishing Company, Inc., and subscriptions are sold by volume only. The journal publishes quarterly. Journal of Business and Technical Communication: The Journal of Business and Technical Communication is peer reviewed and published quarterly. It has been around since 1987 and covers written, oral, and electronic communication in all areas of business, science, and government.

Bias is something that is common in published literature. Some types of bias that exist are publication bias, citation bias, language bias, reference bias, and multiple publication bias (Beck-Bornholdt & Dubben, 2005). Bias can be both good and bad depending on the situation. For example, a beauty magazine would be bias towards hair and makeup information because that is what the reader expects to find. Bias in technical and professional communication journals can also be either good or bad. Bias is good when a journal immediately states who their target is and what they publish, much like IEEE Transactions on Professional Communication does by targeting the electronics fields. Bias is bad when this same journal would focus their publishing on either qualitative or quantitative research, when both are equally as important in the electronics fields. Qualitative research seems to be preferred by journals because it is open in the sense that the research is exploratory. In qualitative research, the researcher forms a central research question(s) and not hypotheses. Qualitative methods include case studies, ethnographies, and grounded theory studies. In quantitative studies, research questions and hypotheses are used to focus the purpose of the study. Quantitative methods include experiments and surveys. Quantitative illiteracy is an issue and one reason behind it could be the lack of published quantitative studies. By publishing less of one than the other, readers are left out on learning about studies that they can benefit and learn from. The majority of the article topics in these journals contain research that was conducted either in a professional or academic environment. The where(s) and how(s) of the article can effect who the journal applies to. For example, professionals may prefer research conducted for field purposes because that is what applies the best to them, and the same can be assumed for students. Methods and Survey Questions To find out whether the journals hold bias towards qualitative research, past issues were measured by categorizing each article in the journals. Since each of the journals are published

BIAS IN TPC JOURNALS

quarterly, there was 20 issues per journal making that a total of 100 issues reviewed from 20072012. Qualitative studies were categorized into Group L and quantitative studies were categorized into Group N. All other articles that were mixed methods, textual analysis, literature review, or opinion-based were categorized as Group O. The total number of articles would then be divided by the number of articles in each group to find the percentage of each study approach used. The final results will tell whether the journals are bias or not towards qualitative research. To find out whether the same journal articles were bias towards professional or academic research topics, a cross-sectional questionnaire was given to both 25 employees in a writing department and 25 graduate students. The experiment was a within-subjects design because each participant received the same two copies of each journal. The subjects were asked to look through the articles and article content and fill out a questionnaire. The questionnaire used a Likert Scale that asked the participants to rate each question on a scale of 1-5. A questionnaire was used for a one-shot case study to have both the students and the professionals rate how well the journal articles related to them and their work. The students were assigned to Group S and the writers were assigned to Group P. The independent variable was using the same two journals from the five journals for each participant to rate, and the dependent variable was how they rated them. The questionnaires would measure which group the articles apply to more. The questionnaires took place in a controlled setting in a hotel conference room on Saturday November 17, 2012 from noon- 3p.m., so that there were not any external validity threats. This experiment was done to see how well the two groups connected with the journal content. Lastly, the article topics were categorized and counted as either academic (Group S), professional (Group P), or other (Group T). Participant Selection Students and field professionals were chosen for this study so that the two groups could be compared and contrasted. Since one of the two hypotheses is that the journal articles are written more for professionals than students, it is necessary to see how the two groups perceive them by rating them. 25 students and 25 technical writers were used to represent the professional and college populations through single-stage sampling. The corporation chosen for the study was Advance Internet Technologies, Inc. (AIT), whom headquarters in Raleigh, North Carolina, and the graduate program that was chosen was the East Carolina University English Department in Greenville, North Carolina. 25 participants from each group were chosen at random using a random numbers table that was stratified to represent gender equally. Data Analysis Methods Categorizing the Articles The first step towards testing the hypotheses will be categorizing the data. The first hypothesis is that the five technical and professional communication journals are bias towards publishing qualitative research. In order to test this, the qualitative (Group L), quantitative (Group N), and other (Group O) groups of articles will be counted and divided by the total number of articles to analyze the journal articles. Next, to test the second hypothesis that the journal articles focus on

BIAS IN TPC JOURNALS

professional studies, the professional journal topics (Group P), academia/student topics (Group S), and other topics (Group T) will be counted and then divided by the total number of articles. For both hypotheses, an ANOVA test will be used to test the means of each group. The independent variable will be the article/study types, and the dependent variable used will be the article content. Within Subjects Design The last part of testing the second the hypothesis includes collecting the questionnaires and analyzing them. The data for the questionnaires will be analyzed in a series of five steps. The five steps are as follows: Step 1: To collect and make sure that all of the questions on the questionnaires have been completed. Step 2: Checking for response bias. To do this, members of each group who were not included in the study will be contacted via a short telephone conversation to see if their answers align with those who completed it. Step 3: To provide an analysis for how the answers measured in relation to each group. A t-test will be used to compare the two groups, and the null hypothesis will be that more professionals related to the articles than students. Step 4: To support hypothesis validity, the answers will be measured using a Likert Scale. A five-point Likert scale will be used to find the means of the ten questions. Internal consistency and reliability will be measured by making sure similar questions measure closely. Questions 12, 3-4, and 7-9 are all similar and should be answered almost the same. If the answers do not match up, then the questionnaire may be determined as void. Step 5: To obtain the statistics and patterns from the questionnaires. Univariate analysis of variance will be used to test the dependent variable of how the participants rated the journal content. The variable will divide the two groups of participants, and the Univariate analysis will use regression analysis that will either support or not support the hypothesis that the journals focus on professional studies.

BIAS IN TPC JOURNALS

References Access Board. IEEE Technical Transactions on Professional Communication. Retrieved November 22, 2012, from http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?reload=true&punumber=47 Access Board. Journal of Business and Technical Communication. Retrieved November 22, 2012, from http://jbt.sagepub.com/ Access Board. Journal of Technical Writing and Communication. Retrieved November 22, 2012, from http://www.baywood.com/journals/previewjournals.asp?id=0047-2816 Access Board. Technical Communication. Retrieved November 22, 2012, from http://techcomm.stc.org/instructions-for-authors/ Access Board. Technical Communication Quarterly. Retrieved November 22, 2012, from http://www.tandfonline.com/action/aboutThisJournal?show=abstractingIndexing&journal Code=htcq20 Beck-Bornholdt, H.P., & Dubben, H.H. (2005). Systematic Review of Publication Bias in Studies on Publication Bias. British Medical Journal, 331(7,517), 433-434. Creswell, J. W. (2003). Quantitative Methods. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (153-178). Thousand Oaks, CA: Sage Publications. Ograjensek, I., & Thyregod, P. (2004) Qualitative Vs. Quantitative Methods. Quality Progress, 37(1), 82.

BIAS IN TPC JOURNALS

Appendix Questionnaire: Once you have completed going through the journals, please complete this short questionnaire. Please respond to the questions according to the grading rubric listed below: 1. Strongly disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5. Strongly agree 1. The journals were interesting Strongly Agree 5 4 3 2. The article topics were interesting Strongly Agree 5 4 3

Strongly Disagree

Strongly Disagree

3. I could relate to the journal content Strongly Agree 5 4 3 2 4. The topics related to my day-to-day work Strongly Agree 5 4 3 2 5. These articles were helpful Strongly Agree 5 4

Strongly Disagree

Strongly Disagree

1 Strongly Disagree

6. The articles came across as bias to one or more groups Strongly Agree 5 4 3 2 1 Strongly Disagree 7. I would recommend these journals to colleagues Strongly Agree 5 4 3 2 1 Strongly Disagree 8. I would read these articles on my own time Strongly Agree 5 4 3 2 1 9. I would reference these journals for research Strongly Agree 5 4 3 2 1 10. On a scale of 1-5, I would rate these journals: Great 5 4 3 2 1 Awful

Strongly Disagree

Strongly Disagree