EFFECTS OF HUMAN AND HUMAN-MACHINE FAKE NEWS DETECTION APPROACHES ON USER DETECTION PERFORMANCE
Main Article Content
Abstract
Abstract: The detection of fake news is a relevant problem-solving mechanism. Literature presents us with three methods of detecting fake news; Human-based, Machine-based and the Human-Machine (hybrid) method. There are questions about which detection approach may be best in detecting fake news. In this study, we test the effects of using two existing methods to detect fake news: Human-based and the Human-Machine approach. Participants perform a small classification task where they were asked to determine if a news article was fake or not. The study used two levels of a within-subject design; where each participant used both the human-based approach and the human-machine approach. Performance and User experience were the dependent variables. Results of the study show that the Human-Machine approach improved user fake news detection effectiveness by 26%. The results suggest that augmenting human intelligence with machines has benefits in fake news detection. Â
Downloads
Article Details
COPYRIGHT
Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
- The journal allows the author(s) to retain publishing rights without restrictions.
- The journal allows the author(s) to hold the copyright without restrictions.
References
C. J. Bond and B. M. DePaulo, "Accuracy of deception judgments.," Personality and Social Psychology Review, vol. 10, no. 3, pp. 214-234, 2006.
D. Mrowca, E. Wang and A. Kosson, "Stance Detection for Fake News Identifcation," Stanford University, 2016.
E. Mustafaraj and P. T. Metaxas, "The Fake News Spreading Plague:Was it Preventable?," in Proceedings of ACM Web Science Conference, Troy, NY, USA, 2017.
A. A. Halimeh, P. Pourghomi and F. Safieddine, "The Impact of Facebook’s News Fact-Checking on Information Quality (IQ) Shared on Social Media," MIT International Conference on Information Quality, 2017.
N. J. Conroy, V. L. Rubin and Y. Chen, "Automatic Deception Detection: Methods for Finding Fake News," in ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community, St. Louis, MO, USA., 2015.
H. Allcott and M. Gentzkow, "Social Media and Fake News in the 2016," Journal of Economic Perspectives, vol. 31, no. 2, pp. 211-236, 2017.
G. L. Ciampaglia, P. Shiralkar, L. M. Rocha, J. Bollen, F. Menczer and A. Flammini, "Correction: Computational Fact Checking from Knowledge Networks," PLoS ONE, vol. 10, no. 6, 2015.
R. A. Baron and D. Bryne, Social Psychology, London: Allyn and Bacon, 2000.
M. Gabielkov, A. Ramachandran, A. Chaintreau and A. Legout, "Social Clicks: What and Who Gets Read on Twitter?," Proceedings of the 2016 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Science, pp. 179-192 , 2016.
M. Anderson, "Social media causes some users to rethink their views on an issue," 7 November 2016. [Online]. Available: http://www.pewresearch.org/fact-tank/2016/11/07/social-media-causes-some-users-to-rethink-their-views-on-an-issue/.
C. D. Goodwin-Ortiz, "Fake News on Social Media: Illusory Truth and the 2016 Presidential Election," Thomas Edison State University, New Jersey, 2017.
D. Maheswaran and S. Chalken, "Promoting Systematic Processing in Low Motivation Settings: Effect of Incongruent Information on Processing and Judgement," Journal of Personality and Social Psychology, pp. 1762 - 1776, 1991.
R. E. Petty and J. T. Cacioppo, "Involvement and Persuasion; Tradiction Versus Integration," Psychological Bulletin, pp. 367 - 374, 1990.
K. Shu, S. Amy, W. Suhang, T. Jiliang and L. Huan, "Fake News Detection on Social Media: A Date Mining Perspective," ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 22-36, 2017.
D. W. Hango, "University graduates with lower levels of literacy and numeracy skills," Statistics Canada: Insights on Canadian Society, 2014.
E. Okoro, B. Abara, A. Umagba, A. Ajonye and Z. Isa, "A Hybrid Approach to Fake News Detection on Social Media," Nigerian Journal of Technology (NIJOTECH), vol. 37, no. 2, April 2018.
D. Rus, "More Evidence That Humans and Machines Are Better When They Team Up," MIT Technology Review: Intelligent Machines, 8 November 2017.
S. Karodia, "Annotating theVeracity of Tweets through Mobile Crowdsourcing," University of Cape Town, Cape Town, 2017.
Facebook, "Tips to Spot False News," 2017. [Online]. Available: https://web.facebook.com/help/188118808357379?_rdc=1&_rdr.
S. Mwanza and H. Suleman, "News Credibility Checking System for Twitter," University of Cape Town, Cape Town, 2017.
S. Teufel, "AN OVERVIEW OF EVALUATION METHODS IN TREC AD HOC INFORMATION RETRIEVAL AND TREC QUESTION ANSWERING," in Evaluation of Text and Speech System, Dordrecht, The Netherlands, Springer, 2007, pp. 168-186.
R. A. Block, P. A. Hancock and D. Zakay, "How cognitive load affects duration," A meta-analytic review. Acta psychologica, vol. 134, no. 3, pp. 330 - 343, 2010.
M. Sucala, B. Scheckner and D. David, "Psychological Time: Interval Length Judgments and Subjective Passage of Time Judgments," Current psychology letters: Behaviour, brain & cognition, vol. 26, no. 2, 2010.
K. Finstad, "Response Interpolation and Scale Sensirivity: Evidence Aganist 5-Point Scales," Journal of Usability Studies, vol. 5, no. 3, pp. 104-110, 2010.
R. Peterson, "A Quantitative Analysis of Rating-scale Response Variability," Marketing Letters, vol. 8, pp. 9-21, 1997.
Snopes, "Snopes," Urban Legends Reference Pages, 2017. [Online]. Available: www.snopes.com. [Accessed 31 October 2017].
D. C. Engelbart, "Program On Human Effectiveness," Stanford Research Institute, December 1961.