Audience perceptions of AI-generated content in sports journalism.

Audience perceptions of AI-generated content in sports journalism.

I. Introduction and research scope

Pavlik (2000) has noted that journalism has always been shaped by technology. This is not to
say that technology in and of itself drives change, but it is an intrinsic part of the mix of
economics, political, social, organizational factors (Boczkowski 2010). Using technology as a
part of the journalistic process is not a new phenomenon. The use of artificial intelligence
(A.I) to write journalistic content, however, is.

The advent of automated news stories has influenced journalism in terms of content, means
of production, and consumption. A.I promises to reap many big rewards for journalism.
Greater speed, accuracy, scale and diversity of coverage are just some of the results media
organisations are already seeing (Marconi 2017). The application of A.I in newsroom also
raises many questions, e.g. what are the implications for journalism and journalistic practice,
can journalists be taken out of the equation of journalism, how is this type of content
regarded by the readers?

Recent research have previously studied and discussed the impact of A.I in newsroom and
journalism practice. In the field of sports journalism, speed and accuracy are two important
advantages of automated content creators. A.I can create thousands of news stories for a
particular topic; they also do it more quickly, cheaply, and potentially with fewer errors than
any human journalist (Andreas 2016). Furthermore, sports journalism has been traditionally
viewed as an uncritical booster and promoter of sports and its culture rather than a field that
challenges and calls for the accountability of the powerful organizations and individuals in
the world of sports (Galily 2018).

The focus of current research has been generally on “the journalists” and “the media”. No one
has investigated how the readers perceive automated content in sports journalism. Therefore,
my research would focus on investigating how readers perceive A.I-generated content in
relation to similar content written by a journalist in sports journalism. My question is:

“How is A.I-generated content perceived by readers with regard to overall quality in

sports journalism?”

“Quality” is a concept with many facets. For content, “quality” can refer to an overall, but
somewhat vague. “Quality” can also be assessed by different standards depending on who is
the receiver and which platform the news is presented (Newhagen & Nass 1989).
Audience perceptions of AI-generated content in sports journalism.

In order to conduct research on content quality, Sundar (1999) suggests conducting a pre-
test where participants were asked to read news articles and then provided adjectival
descriptions of the story as a whole first and then the content of the story. As noted by Sundar,
this is a useful way to get the view of the respondents that is not influenced by the
researcher’s preconceptions.

One of the contributions from Sundar study (1999) is 21 factors of news story rating (see
Appendix 1). There are other criteria pertaining to the evaluation of the overall quality of a
news story. Slater and Rouner (1996) study suggest that message quality evaluation includes
consistency of tone, uniqueness of voice, level of creativity and formality, well written to
bring the reader closer emotionally and cognitively.

Based on the literature review above, there are 12 descriptors with respect to content
evaluation: objective, trustworthy, accurate, boring, interesting, clear, informative, well
written, useable, descriptive, coherent and pleasant to read.

II. Methodology
The study comprises of 2 stages: a pre-test and a focus group.

Participants would be recruited from Bachelor Communication and Media (BCM). Thanks to
the nature of this degree, BCM students have gradually cultivated their ‘news sense’ so they
know how to evaluate a text.

In the pre-test held on May 10, 5 participants are asked to read a text and are encouraged to
assign at least five words or phrases to evaluate the article content. This stage would verify
whether the content evaluation criteria are sufficient for the following step.

The focus group on May 13 utilises an experimental methodology. There will be two articles
that both report a football match between Whitman High School and Landon High School.
One will be written by a sports journalist and the counterpart is generated by Heliograf - The
Washington Post’s AI system (see Appendix). 10 participants are randomly assigned to read
one article, and then they would be asked to rate the text in a Likert scale questionnaire based
on provided criteria (see Appendix 2). After making this assessment, the respondents would
ask to assess whether they think the article is written by a journalist or generated by A.I.

III. Aims of research

Audience perceptions of AI-generated content in sports journalism.

The purpose of research is to develop a better standing on the influences of A.I in sports
journalism. This information would be used for further research in order to more thoroughly
study how the audience perceives automated content and how it may impact the practice of
journalism.

IV. Findings presentation:

My findings would be presented in a whiteboard animation video. First, illustrations can help
me to convey my findings in a compelling way, allowing the audiences to grasp ideas better.
Furthermore, it is time-saving thanks to some friendly-user Whiteboard Animation software
available such as VideoScribe.

Content structure for 5-minute video:

Introduction
of the
influences Research
Result
and process and
discussion
applications findings (2
(1.30 minute)
A.I in the minutes)
newsroom
(1.30 minute)
Audience perceptions of AI-generated content in sports journalism.

REFERENCES:

Andreas, G 2016, ‘Guide to Automated Journalism’, Tow Center for Digital Journalism,
<http://towcenter.org/research/guide-to-automated-journalism>.

Boczkowski, P 2010, ‘The mutual shaping of technology and society in videotex newspapers:
Beyong the diffusion and social shaping perspectives’, The Information Society,vol.20, no.4,
pp 255-267, <https://doi.org/10.1080/01972240490480947>.

Galily, Y 2018, ‘Artificial intelligence and sports journalism: Is it a sweeping change?’,

Technology in Society, vol.1, no 5, pp 23-28, <https://doi.org/10.1016/j.techsoc.2018.03.001>.

Marconi, F 2017, ‘Associated Press: Future of Journalism will be augmented thanks to AI’,
AI Business, viewed May 5, < https://aibusiness.com/ai-journalism-associated-press/>.

Newhagen, J, Nass, C 1989, ‘Differential Criteria for Evaluating Credibility of Newspapers

and TV News.’, Journalism Quarterly, vol.66, no.2, p277-284, <http://doi.org/10.1177/
107769908906600202>.

Pavlik, J 2000, ‘The Impact of Technology on Journalism’, Journalism Studies, vol.1, no.2,
pp 229-237, < https://doi.org/10.1080/14616700050028226>.

Slater, D Rouner, D 1996, ‘How Message Evaluation and Source Attributes May Influence
Credibility Assessment and Belief Change.’, Journalism & Mass Communication Quarterly,
no.73, vol.4, p.974-991, <http://doi.org/10.1177/107769909607300415>.

Sundar, S 1999, “Exploring Receivers’ Criteria for Perception of Print and Online News.”,
Journalism & Mass Communication Quarterly, no.76, vol.2, pp 373-386,
<http://journals.sagepub.com/doi/10.1177/107769908906600202 >.

APPENDIX:
Audience perceptions of AI-generated content in sports journalism.

Appendix 1: Factor Loadings of Print News Story Rating

Appendix 2: Content quality evaluation form