Anna Sophie Kümpel, Nina Springer, Commenting quality in:

SCM Studies in Communication and Media, page 353 - 366

SCM, Volume 5 (2016), Issue 3, ISSN online: 2192-4007, https://doi.org/10.5771/2192-4007-2016-3-353

Browse Volumes and Issues: SCM Studies in Communication and Media

CC-BY-NC-ND

Bibliographic information
PDF download Citation download Share
353Studies in Communication | Media, 5. Jg., 3/2016, S. 353–366, DOI: 10.5771/2192-4007-2016-3-353 SC|M Studies in Communication | Media SC|M Studies in Communication | Media RESEARCH-IN-BRIEF Commenting quality Effects of user comments on perceptions of journalistic quality Qualität kommentieren Die Wirkung von Nutzerkommentaren auf die Wahrnehmung journalistischer Qualität Anna Sophie Kümpel & Nina Springer 354 SCM, 5. Jg., 3/2016 Anna Sophie Kümpel, Institut für Kommunikationswissenschaft und Medienforschung, Ludwig-Maximilians-Universität München, Oettingenstr. 67, 80538 München; Kontakt: kuempel(at)ifkw.lmu.de Nina Springer, Institut für Kommunikationswissenschaft und Medienforschung, Ludwig- Maximilians-Universität München, Oettingenstr. 67, 80538 München; Kontakt: nina.springer(at)ifkw.lmu.de 355 RESEARCH-IN-BRIEF Commenting quality Effects of user comments on perceptions of journalistic quality Qualität kommentieren Die Wirkung von Nutzerkommentaren auf die Wahrnehmung journalistischer Qualität Anna Sophie Kümpel & Nina Springer Abstract: The (participatory) opportunities provided by interactive features on news sites have been widely theorized and investigated in recent years. However, user comments’ effects on the perception of journalistic quality have only begun to be examined. To investigate those effects, the present study deployed two 2 × 2 experiments with a between-subject design, thereby exposing participants (N = 224) to a constructed online news article (covering a potential military intervention of the German armed forces against ISIS) and corresponding user comments. Comments varied in terms of (I) support for the issue described in the article as well as the (II) addressing of journalistic quality criteria (accuracy, impartiality). Results indicate that user comments indeed had considerable effects on readers’ quality assessments. If anonymous users praised the quality of the article in their comments, journalistic quality was perceived to be significantly higher. Implications for news media and media effects research are discussed. Keywords: Journalistic quality, user comments, media effects, quantitative experiment. Zusammenfassung: Die partizipativen Möglichkeiten von Nachrichtenseiten wurden in den letzten Jahren sowohl theoretisch als auch empirisch auf breiter Basis untersucht. Die Frage danach, ob und wie Kommentare von Nutzerinnen und Nutzern – als häufigste Form der Partizipation am Journalismus – auf die Wahrnehmung journalistischer Qualität wirken, wurde bislang jedoch nur unzureichend adressiert. Um diese Effekte zu untersuchen, führten wir zwei 2 × 2 Online-Experimente durch, bei denen wir 224 Teilnehmerinnen und Teilnehmer mit einem Nachrichtenartikel (über einen möglichen Einsatz der Bundeswehr gegen ISIS) sowie dazugehörigen Kommentaren konfrontierten. Diese wurden systematisch variiert und unterschieden sich mit Blick auf (I) die Zustimmung zum Thema des Artikels sowie (II) die Einschätzung journalistischer Qualität (Richtigkeit, Unparteilichkeit). Die Ergebnisse zeigen, dass die Kommentare einen deutlichen Einfluss auf die Qualitätswahrnehmung der Leserinnen und Leser hatten. Lobten die Kommentare die Qualität des Artikels, wurde die journalistische Qualität als signifikant besser wahrgenommen. Implikationen für Nachrichtenmedien sowie für die Medienwirkungsforschung werden diskutiert. Schlagwörter: Journalistische Qualität, Nutzerkommentare, Medienwirkungen, quantitatives Experiment. Commenting quality 356 SCM, 5. Jg., 3/2016 Research-in-brief 1. Introduction User comments are considered to be the most common form of user participation in online journalism (Hermida, 2011; Richardson & Stanyer, 2011), challenging our understanding of theorizing media effects in online environments (Springer, Engelmann, & Pfaffinger, 2015). Because news and user-generated information are nowadays inseparable, media effects research must take these interdependencies into account and focus on the wider embedding of media content instead of investigating it in isolation. The very same news article may have different effects on readers depending on the number of likes, shares, and – not least – the number and especially content and valence of user comments. This study examines the influence of user comments on readers’ assessments of journalistic quality. The relevance is twofold. First, if comments indeed affect quality assessments, this effect has implications for media images and, more generally, trust in media outlets. Second, these changes in media images and trust may affect selection decisions and thus determine whether readers are inclined to turn to certain media outlets after all (cf. Wolling, 2009). In this study, under a controlled experimental setting, we investigate how the positive and negative addressing of two journalistic quality criteria (accuracy respectively impartiality) as well as issue-related evaluations in user comments influence how people perceive the quality of a corresponding news article. 2. Effects of user comments on quality assessments Scholars who studied user comments from a media effects perspective focused on different dependent variables, ranging from risk perceptions (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014), third-person perceptions (Houston, Hansen, & Nisbett, 2011), and perceptions of media bias or the climate of opinion (Friemel & Dötsch, 2015; E.-J. Lee, 2012), to perceptions of credibility (J. Lee, 2014; J. Lee & Lim, 2014; Marchionni, 2014; Thorson, Vraga, & Ekdale, 2010). In general, all studies indicate that comments – at least for a short period of time – are able to shape attitudes and perceptions. However, to our knowledge, the effects on the assessments of journalistic quality have not been studied sufficiently. Experimental studies show that enabling comments does not lead readers to perceive journalism as more trustworthy (Marchionni, 2014), and that the sheer presence of comments can, on the contrary, even lead to a more negative assessment of journalistic quality (Ash, Hettinga, & Halpern, 2009; Prochazka, Weber, & Schweiger, 2016). But which factors are relevant with respect to journalistic quality – and are readers actually able to detect it? Through comparison of studies that develop catalogues of journalistic quality criteria, we have identified five dimensions that can – albeit sometimes differently labelled – be found in all of them: relevance, comprehensibility, diversity, impartiality, and accuracy (e.g., Jungnickel, 2011; Schatz & Schulz, 1992; Urban & Schweiger, 2014; Wellbrock & Klein, 2014). In the present study, we decided to solely examine the effects of accuracy and impartiality 357 Kümpel/Springer | Commenting quality due to a) the need for reduction in experimental research, b) the overall relevance of these criteria identified in previous studies on journalistic quality (e.g. Arnold, 2008; Wellbrock & Klein, 2014), and – most importantly – c) the fact that they were most often addressed in actual user comments on our chosen issue.1 Studies focusing on readers’ quality perceptions (e.g., Jungnickel, 2011; Urban & Schweiger, 2014) suggest that recipients are generally able to spot differences regarding these two quality criteria, although accuracy seems – due to the need for background knowledge – harder to evaluate than impartiality. Since none of the existing experimental studies focusing on quality assessments actually varied how quality criteria were addressed in user comments, we want to test whether comments with explicit reference to journalistic quality are able to affect readers’ quality assessments. Studies investigating the influence of positively and negatively valenced evaluations in user comments generally suggest that a positive evaluation in user comments is reflected in participants’ assessment of the subject or issue in question. Lee and Lim (2014), for example, found that positively valenced comments about a political candidate led participants to perceive said candidate as more trustworthy than those participants that were exposed to negatively valenced comments (see also J. Lee, 2014). Thus, we assume: 1 Rather than choosing the quality criteria for the study beforehand, we first decided upon a topic, examined the associated user comments and then selected the two quality criteria – impartiality and accuracy – that were most often addressed (cf. for the relevance of these criteria: Hackel-de Latour, 2015; Hagen, 2015). H1: A news article that is accompanied by user comments that positively address the accuracy/impartiality of the article is judged to be of higher quality than a news article that is accompanied by user comments that negatively address the accuracy/impartiality of the article. Besides, we are interested whether the sheer valence of comments addressing the issue of the article might influence quality assessments as well. We assume that the expression of consent or dissent in comments might induce a kind of spillover effect on perceptions of the article, simply through indicating (dis-) agreement. Due to people’s tendency to use heuristics when making judgments (Gigerenzer & Gaissmaier, 2011), the availability of valenced issue evaluations in the user comments might steer their quality assessments in the respective (positive or negative) direction. Hence, we propose: H2: A news article that is accompanied by user comments that endorse the issue of the article is judged to be of higher quality than a news article that is accompanied by user comments that do not endorse (= criticize) the issue of the article. 3. Method To investigate the effects of user comments on readers’ assessments of journalistic quality, we conducted two 2 × 2, post-test only experiments with a between-subject design. We exposed participants to a constructed online news article (same for all participants) as well as to corresponding user comments that varied in terms of (I) support for the issue described in the article (military intervention of the 358 SCM, 5. Jg., 3/2016 Research-in-brief German armed forces against ISIS) as well as the (II) addressing of journalistic quality criteria. Support was varied on two factor levels, differentiating between endorsement and refusal of the military intervention. The addressing of quality criteria was also varied on two factor levels, differentiating between praising (positive addressing) and criticizing (negative addressing). To keep the engagement time with the questionnaire as short as possible, one group of participants only saw comments focusing on accuracy (henceforth accuracy group), while the other group of participants only saw comments focusing on impartiality (henceforth impartiality group). In the experimental procedure, participants were randomly assigned to the experimental conditions and exposed to the article that was supposedly published on Spiegel Online. The article was written as balanced as possible, focusing on arguments for both perspectives – support for/opposition against the intervention – in equal depth.2 On the next page, five user comments were presented (see Table 1). Two of them addressed the issue of the article, both either endorsing or refusing it. Another two of them addressed the article’s impartiality (resp. its accuracy), both either praising or criticizing it. In all groups, an irrelevant comment (addressing the availability of the article on Twitter) was 2 The article was about 250 words long and composed of real online news media coverage on the issue. It reported a dispute between members of the Bundestag, in which some were strongly in favor of a military intervention against ISIS while some rejected it entirely. Both points of view were presented and illustrated by the quote of one representative which was recited by the journalist without further (evaluative) commentary. added to increase external validity as well as to fit the actual website design in which five comments are displayed. In December 2014, a total of 248 German participants took part in the two online experiments. They were recruited both by the authors, the participants in the seminar, and postings in Facebook university groups. The final sample consisted only of participants who had spent at least 15 seconds on both of the pages presenting the stimulus material (n = 224). Demographic characteristics of the sample included age (M = 25.09, SD = 9.57), gender (66.1% female), and educational level (34.8% with a university degree, 58.5% with a higher education entrance qualification and 6.7% with lower educational qualifications). Adapting operationalization by Jungnickel (2011) and Urban and Schweiger (2014), we used two statements to assess both perceptions of accuracy (“The article does not contain erroneous information” and “The journalists’ statements are without contradictions.”) and impartiality (“The journalist reports in a neutral and unbiased manner” and “All perspectives are covered in equal depth.”) on a Likert scale from 1 (strongly disagree) to 5 (strongly agree).3 Measures also included assessment of prior knowledge about the issue in question, Internet use and the use of commenting 3 The resulting means for assessments of impartiality were M = 2.97 (SD = 0.87, r = .392, p < .001) in the accuracy group and M  =  2.87 (SD = 0.98, r = .530, p < .001) in the impartiality group; the resulting means for assessments of accuracy were M = 3.72 (SD = 0.87, r = .601, p < .001) in the accuracy group and M = 3.93 (SD = 0.81, r = .405, p < .001) in the impartiality group. In both the accuracy (r = .497, p < .001) and the impartiality group (r = .397, p < .001), assessments of accuracy and impartiality are significantly correlated. 359 Kümpel/Springer | Commenting quality features (active, passive), as well as various sociodemographic variables. As a treatment check, we asked participants to rate statements that described the content of the user comments on the same Likert scale. Perception of accuracy in comments was measured with the statement “If the comments addressed the quality of the article, they praised it as well researched and factually correct” (TCA) and perception of impartiality in comments with the statement “If the comments addressed the quality of the article, they praised it as comprehensive and impartial” (TCIP). As predicted, participants in the accuracy group who read the comments that praised the articles’ accuracy agreed significantly more to TCA (M  =  4.21, SD = 0.87) than those who read comments that criticized its accuracy (M  =  1.28, SD = 0.64), t(106) = 20.03, p < .001. Accordingly, participants in the impartiality group who read comments that praised the articles’ impartiality agreed significantly more to TCIP (M = 4.19, SD = 0.78) than those who read comments that criticized its impartiality (M = 1.31, SD = 0.54), t(104) = 21.98, p < .001.4 4 We also conducted a treatment check to investigate whether participants perceived the issue endorsement in user comments as intended. We did so by assessing their agreement to the statement “If the comments addressed the military intervention against ISIS, they supported it”. As intended, participants who read the comments that endorsed the intervention agreed significantly more to the statement (M = 3.79, SD = 0.84) than those who read the comments that opposed the intervention (M = 1.71, SD = 0.82, t(212) = 18.269, p < .001). Table 1: Example of one of the comment threads used in the impartiality group Awesome! schmitt.schneider 17.11.2014 Finally, an article that integrates all perspectives on this important issue. So people are actually able to build their own (and independent!) opinion. Positive addressing of impartiality (+) Responsibility D.Stein 17.11.2014 Sure it’s hard to make the right decision when lives are affected. However, at the moment, prompt action is required. If you don’t want to give weapons to Kurds or other IS opponents, you are forced to act on your own! Hence, I totally support such a military intervention. Nevertheless, it is also necessary to involve the UN and develop concepts for the future rapidly. Support for issue (+) Well-made article Meine:Meinung 17.11.2014 An author who knows his craft and does not (!) take sides. Especially important considering the potential for conflict that arises from this issue. Positive addressing of impartiality (+) Twitter bgbg 17.11.2014 Why doesn’t the article appear on your twitter feed? Will you fix that soon? Irrelevant comment Moral duty incognito 17.11.2014 It is our moral duty to assist the persecuted people in Northern Iraq! And not only with the delivery of arms but with air strikes! How can we just look away when an ideologically distorted religion executes people of other religions en masse? Action is needed here! Support for issue (+) Note: The other three comment threads used in the impartiality group as well as the four comment threads used in the accuracy group can be requested from the SCM editorial staff or the authors (Original versions in German were used in the experiments and are readily available). 360 SCM, 5. Jg., 3/2016 Research-in-brief 4. Results To test our hypotheses, we conducted multifactorial analyses of covariance (ANCOVA). The dependent variables were participants’ accuracy and impartiality assessments; the factors were the addressing (positive, negative) of accuracy respectively impartiality and the endorsement of the issue (endorsing the issue, not endorsing the issue) in user comments. We controlled for participants’ gender, age, prior knowledge about the issue, usage of Spiegel Online, and their frequency of comment reading and posting. In the accuracy group, a significant main effect of the addressing of accuracy on both accuracy (AA) and impartiality assessments (IA) could be  observed, FAA(1, 102) = 38.89, p  <  .001, ηp2 = .28; FIA(1, 102) = 25.55, p < .001, ηp2 = .20 (see Table 2): Participants that read the news article that was accompanied by user comments that positively addressed the accuracy of the article judged it to be of higher quality in terms of both accuracy and impartiality than participants that read the news article that was accompanied by user comments that negatively addressed the accuracy of the article. Thus, H1 is supported. Additionally, for the assessment of accuracy, a significant main effect of issue endorsement (F(1,102) = 5.53, p = .021, Table 2: Main and interaction effects between the addressing of accuracy/ impartiality and issue endorsement in user comments Accuracy Group Accuracy + Accuracy - Accuracy addressing (AA) Issue endorsement (IE) AA*IE Issue + (n = 30) Issue - (n = 29) Issue + (n = 25) Issue - (n = 28) F ηp2 F ηp2 F ηp2 Accuracy assessment 4.2 4.1 3.6 3.0 38.9*** .28 5.5* .05 2.3 .02 Impartiality assessment 3.2 3.5 2.7 2.4 25.6*** .20 0.1 .00 5.0* .05 Impartiality Group Impartiality + Impartiality - Impartiality addressing (IA) Issue endorsement (IE) IA*IE Issue + (n = 27) Issue - (n = 29) Issue + (n = 26) Issue - (n = 30) F ηp2 F ηp2 F ηp2 Accuracy assessment 4.1 4.1 3.9 3.6 4.2* .04 2.2 .02 0.2 .00 Impartiality assessment 3.3 3.4 2.3 2.4 37.5*** .27 1.0 .01 0.1 .00 Notes: A plus (“+”) indicates that the quality criterion was positively addressed resp. that the issue was endorsed in user comments, while a minus (“-”) indicates that the quality criterion was negatively addressed resp. that the issue was not endorsed in user comments. Both accuracy and impartiality assessments are indices based on 2 items (scale from 1 to 5, the higher the value, the higher the perceived journalistic quality in terms of accuracy resp. impartiality). * p < .05; ** p < .01; *** p < .001 361 Kümpel/Springer | Commenting quality ηp2 = .05) could be observed, thereby supporting H2 for accuracy assessments. However, for assessments of impartiality, no main effect of issue endorsement was found. Instead, the ANCOVA showed a significant interaction effect between the addressing of impartiality and issue endorsement, F(1, 102) = 5.02, p = .027, ηp2 = .05. As Figure 1 shows, participants that were exposed to a mixed valence comment thread consisting of user comments that positively addressed the articles accuracy but evaluated the issue of the article negatively, perceived the article’s impartiality to be the highest. Furthermore, for assessments of accuracy, a significant effect of the covariate frequency of comment posting emerged, with further inspection of the data suggesting that participants, who indicated to regularly post own comments, perceived the article to be less accurate, F(1,102) = 4.33, p = .040, ηp2 = .04. In the impartiality group, a significant main effect of the addressing of impartiality on both accuracy (AA) and impartiality assessments (IA) could be observed, FAA(1, 102) = 4.18, p  =  .043, ηp2 = .04; FIA(1, 102) = 37.52, p < .001, ηp2 = .27 (see Table 2), thereby supporting H1. Again, participants that read the news article that was accompanied by user comments that positively addressed the impartiality of the article judged it to be of higher quality in terms of both impartiality and accuracy than participants that read the news article that was accompanied by user comments that negatively addressed the impartiality of Figure 1: Interaction effect between accuracy addressing and issue endorsement on impartiality assessments (accuracy group) Notes: A plus (“+”) indicates that the quality criterion was positively addressed resp. that the issue was endorsed in user comments, while a minus (“-”) indicates that the quality criterion was negatively addressed resp. that the issue was not endorsed in user comments. Impartiality assessment is an index based on 2 items (scale from 1 to 5, the higher the value, the higher the perceived journalistic quality in terms of impartiality). 362 SCM, 5. Jg., 3/2016 Research-in-brief the article. The predicted main effect of issue endorsement could not be observed for either AA or IA, thus contradicting H2 in the impartiality group. Apart from that, the ANCOVA showed a significant effect of the covariate prior knowledge about the issue for accuracy assessments in the impartiality group, F(1,102) = 4.08, p = .046, ηp2 = .04: Participants, who indicated to have more knowledge about the military intervention, perceived the article to be more accurate. 5. Discussion Our results show that user comments that addressed the journalistic quality criteria impartiality and accuracy had a systematic effect on readers’ quality assessments. If user comments praised the news article as being impartial and balanced, it actually led people to perceive the article to be of higher quality in terms of impartiality and – to a smaller degree – in terms of accuracy as well. The same effect could be observed for the assessment of accuracy. If user comments highlighted the article to be without erroneous information or contradictions, it was perceived as being more accurate and, once more, as being more impartial. Thus, it seems that compliments or acknowledgements about coverage in user comments generally lead readers to perceive a news article as being of higher quality, regardless of whether the comments explicitly address the article’s impartiality or its accuracy. In the context of credibility assessments, Yale and colleagues (2015) obtained similar results and found that people tend to evaluate news credibility in a heuristic manner, thus making nuanced distinctions between different types of credibility unlikely. Hence, our assumption that a news article with user comments positively addressing the accuracy/impartiality of the article is judged to be of higher quality than a news article with user comments negatively addressing the accuracy/impartiality of the article, proved to be correct in both groups under investigation and for both kinds of quality assessments. However, the assumption that the expression of consent or dissent in comments (in the form of issue endorsement) might induce a kind of spillover effect on perceptions of the article could only be confirmed for accuracy assessments in the accuracy group. The news article that was accompanied by user comments that endorsed a military intervention against ISIS was judged to be of higher accuracy than a news article that was accompanied by user comments that refused such an intervention. While it is not entirely clear why this effect was only observed for accuracy assessments, we suspect that this might be explained by the fact that “it is difficult for recipients to evaluate articles for accuracy because they need background knowledge on the topic to formulate adequate judgements” (Urban & Schweiger, 2014, p. 833). In our case, this background ‘knowledge’ might simply have been a cue derived from the valence of user comments. Thus, the positive valence (induced through expressing support/endorsement) in the comments might have been carried over to the perception of the news article’s accuracy, thereby positively influencing participants’ assessment. As our analysis has shown, accuracy assessments were also influenced by participants’ frequency of comment posting: Users who post own 363 Kümpel/Springer | Commenting quality comments more frequently, perceived the article to be less accurate. Motives connected to posting comments (cf. Springer et al., 2015; Ziegele et al., 2013) indicate that commenters are more critical and questioning in nature, further suggesting that they might judge an article’s accuracy more sceptically. However, this result has to be considered with caution since the distribution of comment posting frequency is highly positively skewed, in that 89.7% of our participants never post own comments and only 10.3% (at least rarely) do so. In the group of participants that read comments focusing on impartiality (impartiality group), participants’ accuracy assessment was, however, neither influenced by the addressing of the article’s issue nor participants’ frequency of comment posting. Instead, the accuracy assessment was influenced by participants’ prior knowledge about the issue. We assume that – due to the fact that no cues about the article’s accuracy were available in this group – participants had to rely more (or even solely) on their available knowledge. Because the article was composed, we were able to ensure that it was indeed without erroneous information or contradictions: Participants with more knowledge might actually have been able to recognize this accuracy. Besides, for assessments of impartiality in the accuracy group, a rather interesting interaction effect between accuracy addressing and issue endorsement emerged. Somewhat counterintuitively, participants that read a comment thread that consisted of user comments that positively addressed the articles accuracy but evaluated the issue of the article negatively, perceived the article’s impartiality to be the highest. Since impartiality involves the attempt to cover diverse ideas and opinions with professional detachment, the presentation of differently valenced and dissident user comments alongside the article probably spread to participants’ perception of the article’s impartiality. However, we certainly require more research to fully understand the mechanisms behind news users’ quality assessments. Overall, our study provides evidence that comments left by anonymous users on news websites can serve as meaningful cues for people’s own assessments. If journalistic work can be distorted or degraded by negative comments, or, on the other hand, appears more valuable, factual, or well researched due to positive comments, it seems increasingly crucial for online editorial departments to develop strategies for dealing with comments. One approach might be to actively participate in user discussions. Meyer and Carey (2014) have shown that having an active moderation in the comment section actually increases users’ willingness to post comments, and, supposedly, more thoughtful and respectful ones as well. At the same time, this feature offers the possibility for journalists to respond to criticism early on and to “take ‘emotional and physical ownership’ of the online environments they control” (p. 225). Of course, our study is not without limitations. Following recommendations for online surveys to minimize the need for vertical scrolling as well as to control the reading time for both the article and the user comments, we presented the article and the user comments on two sites of the questionnaire. The consequent highlighting of the user comments, however, might 364 SCM, 5. Jg., 3/2016 Research-in-brief have increased their influence (recency effect). Besides, trying to make the content of the manipulated comment threads as realistic as possible, we somewhat abandoned internal validity in favour of external validity. Hence, the wording and arguments differed between the experimental conditions— especially for the factor of issue endorsement. However, as the treatment check has shown, participants nevertheless perceived the comments as intended. To keep it simple, we furthermore disregarded the “interpersonal nature” (Ziegele, Breiner, & Quiring, 2014, p. 24) of user comments. The commenters in the stimulus do not refer to each other or even challenge or question certain opinions expressed by their fellow commenters. Future studies should thus investigate whether a comment thread with disputing and arguing commenters influences (quality) assessments differently. Additionally, it would be valuable to examine the behavioral consequences of the identified shifts in quality assessments. Do they actually affect future selection decisions or are perceptions of quality only a small element in the process of selecting media (e.g., Wolling, 2009)? It cannot be denied that user comments are meaningful cues in the environment of online journalism. Hence, they should not be treated as some kind of irrelevant attachment to news but rather as an inherent part of nowadays news production and consumption processes. Note: We would like to thank Gianna Banke, Tabea Eppelein, Michael Fischbacher, Markus Gropper, Corinna Hacker, Natascha Herr, Julia Herzer, Sebastian Huber, Laura Michalowski, David Rausch, Anna Schmitt, Leonie Teisner, and Julius zur Nedden, who participated in a seminar we conducted at LMU Munich and eagerly worked with us to prepare and conduct the experiments. Furthermore, we would like to thank the anonymous reviewers and the SCM editorial staff for their constructive comments and suggestions. References Anderson, A. A., Brossard, D., Scheufele, D. A., Xenos, M. A., & Ladwig, P. (2014). The “nasty effect:” Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19(3), 373–387. http://doi.org/10.1111/jcc4.12009 Arnold, K. (2008). Qualität im Journalismus – ein integratives Konzept [Quality in journalism – an integrative concept]. Publizistik, 53(4), 488–508. http://doi. org/10.1007/s11616-008-0012-y Ash, E., Hettinga, K., & Halpern, D. (2009). Effects of a trend: The influence of user comments on readers’ perceptions of online newspapers. Presented at the Annual meeting of the Association for Education in Journalism and Mass Communication, Boston. Friemel, T. N., & Dötsch, M. (2015). Online reader comments as indicator for perceived public opinion. In M. Emmer & C. Strippel (Eds.), Kommunikationspolitik für die digitale Gesellschaft (pp. 151–172). Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62(1), 451–482. http://doi.org/10.1146/annurev-psych- 120709-145346 Hackel-de Latour, R. (2015). „Lügenpresse“!? Über den Glaubwürdigkeitsverlust der Medien [„Lying press“!? About the loss of credibility in media]. Communicatio Socialis, 48(2), 123–125. 365 Kümpel/Springer | Commenting quality Hagen, L. (2015). Nachrichtenjournalismus in der Vertrauenskrise. “Lügenpresse” wissenschaftlich betrachtet [News journalism in a crisis of confidence. “Lying press” from a scientific point of view]. Communicatio Socialis, 48(2), 152–163. Hermida, A. (2011). Mechanisms of participation. How audience options shape the conversation. In J. B. Singer, A. Hermida, D. Domingo, A. Heinonen, S. Paulussen, T. Quandt, … M. Vujnovic (Eds.), Participatory journalism guarding open gates at online newspapers (pp. 11–33). Chichester, West Sussex, U.K.; Malden, MA: Wiley-Blackwell. Houston, J. B., Hansen, G. J., & Nisbett, G. S. (2011). Influence of user comments on perceptions of media bias and third-person effect in online news. Electronic News, 5(2), 79–92. http://doi. org/10.1177/1931243111407618 Jungnickel, K. (2011). Nachrichtenqualität aus Nutzersicht. Ein Vergleich zwischen Leserurteilen und wissenschaftlich-normativen Qualitätsansprüchen [News quality from the recipients‘ perspective. A comparison between readers’ assessments and scientific and normative quality standards]. Medien & Kommunikationswissenschaft, (3), 360–378. h t t p : / / d o i . o r g / 1 0 . 5 7 7 1 / 1 6 1 5 - 634x-2011-3-360 Lee, E.-J. (2012). That’s not the way it is: How user-generated comments on the news affect perceived media bias. Journal of Computer-Mediated Communication, 18(1), 32–45. http://doi.org/ 10.1111/j.1083-6101.2012.01597.x Lee, J. (2014). Are some people less influenced by others’ opinions? The role of internal political self-efficacy and need for cognition in impression formation on social networking sites. Cyberpsychology, Behavior, and Social Networking. http://doi.org/10.1089/cyber. 2013.0713 Lee, J., & Lim, Y. (2014). Who says what about whom: Young voters’ impression formation of political candidates on social networking sites. Mass Communication and Society, 17(4), 553–572. http://doi.org/10.1080/15205436.2013. 816743 Marchionni, D. (2014). Online story commenting. An experimental test of conversational journalism and trust. Journalism Practice. http://doi.org/10.1080/ 17512786.2014.938943 Meyer, H. K., & Carey, M. C. (2014). In moderation. Examining how journalists’ attitudes toward online comments affect the creation of community. Journalism Practice, 8(2), 213–228. http://doi.org/ 10.1080/17512786.2013.859838 Prochazka, F., Weber, P., & Schweiger, W. (2016). Effects of civility and reasoning in user comments on perceived journalistic quality. Journalism Studies, 0(0), 1–17. http://doi.org/10.1080/146167 0X.2016.1161497 Richardson, J. E., & Stanyer, J. (2011). Reader opinion in the digital age: Tabloid and broadsheet newspaper websites and the exercise of political voice. Journalism, 12(8), 983–1003. http:// doi.org/10.1177/1464884911415974 Schatz, H., & Schulz, W. (1992). Qualität von Fernsehprogrammen. Kriterien und Methoden zur Beurteilung von Programmqualität im dualen Fernsehsystem [Quality of TV programs. Criteria and methods for assessing program quality in the dual broadcasting system]. Media Perspektiven, (11), 690–712. Springer, N., Engelmann, I., & Pfaffinger, C. (2015). User comments: Motives and inhibitors to write and read. Information, Communication & Society, 0(0), 1–18. http://doi.org/10.1080/136911 8X.2014.997268 Thorson, K., Vraga, E., & Ekdale, B. (2010). Credibility in context: How un- 366 SCM, 5. Jg., 3/2016 Research-in-brief civil online commentary affects news credibility. Mass Communication and Society, 13(3), 289–313. http://doi.org/ 10.1080/15205430903225571 Urban, J., & Schweiger, W. (2014). News quality from the recipients’ perspective: Investigating recipients’ ability to judge the normative quality of news. Journalism Studies, 15(6), 821–840. http://doi. org/10.1080/1461670X.2013.856670 Wellbrock, C.-M., & Klein, K. (2014). Journalistische Qualität – eine empirische Untersuchung des Konstrukts mithilfe der Concept Map Methode [Journalistic quality – an empirical investigation of the construct employing the concept map method]. Publizistik, 59, 387–410. http:// doi.org/10.1007/s11616-014-0212-6 Wolling, J. (2009). The effect of subjective quality assessments on media selection. In T. Hartmann (Ed.), Media choice: A theoretical and empirical overview (pp. 84–101). New York, London: Routledge. Yale, R. N., Jensen, J. D., Carcioppolo, N., Sun, Y., & Liu, M. (2015). Examining first- and second-order factor structures for news credibility. Communication Methods and Measures, 9(3), 152–169. http://doi.org/10.1080/19312458.2015. 1061652 Ziegele, M., Breiner, T., & Quiring, O. (2014). What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on news items. Journal of Communication, 64(6), 1111–1138. http://doi.org/10.1111/jcom.12123 Ziegele, M., Johnen, M., Bickler, A., Jakobs, I., Setzer, T., & Schnauber, A. (2013). Männlich, rüstig, kommentiert? Einflussfaktoren auf die Aktivität kommentierender Nutzer auf Online-Nachrichtenseiten [Male, hale, comments? Factors influencing the activity of commenting users on online news websites]. Studies in Communication | Media, 2(1), 67–114.

Abstract

The (participatory) opportunities provided by interactive features on news sites have been widely theorized and investigated in recent years. However, user comments’ effects on the perception of journalistic quality have only begun to be examined. To investigate those effects, the present study deployed two 2 × 2 experiments with a between-subject design, thereby exposing participants (N = 224) to a constructed online news article (covering a potential military intervention of the German armed forces against ISIS) and corresponding user comments. Comments varied in terms of (I) support for the issue described in the article as well as the (II) addressing of journalistic quality criteria (accuracy, impartiality). Results indicate that user comments indeed had considerable effects on readers’ quality assessments. If anonymous users praised the quality of the article in their comments, journalistic quality was perceived to be significantly higher. Implications for news media and media effects research are discussed.

Zusammenfassung

Die partizipativen Möglichkeiten von Nachrichtenseiten wurden in den letzten Jahren sowohl theoretisch als auch empirisch auf breiter Basis untersucht. Die Frage danach, ob und wie Kommentare von Nutzerinnen und Nutzern - als häufigste Form der Partizipation am Journalismus - auf die Wahrnehmung journalistischer Qualität wirken, wurde bislang jedoch nur unzureichend adressiert. Um diese Effekte zu untersuchen, führten wir zwei 2 × 2 Online-Experimente durch, bei denen wir 224 Teilnehmerinnen und Teilnehmer mit einem Nachrichtenartikel (über einen möglichen Einsatz der Bundeswehr gegen ISIS) sowie dazugehörigen Kommentaren konfrontierten. Diese wurden systematisch variiert und unterschieden sich mit Blick auf (I) die Zustimmung zum Thema des Artikels sowie (II) die Einschätzung journalistischer Qualität (Richtigkeit, Unparteilichkeit). Die Ergebnisse zeigen, dass die Kommentare einen deutlichen Einfluss auf die Qualitätswahrnehmung der Leserinnen und Leser hatten. Lobten die Kommentare die Qualität des Artikels, wurde die journalistische Qualität als signifikant besser wahrgenommen. Implikationen für Nachrichtenmedien sowie für die Medienwirkungsforschung werden diskutiert.

References
Anderson, A. A., Brossard, D., Scheufele, D. A., Xenos, M. A., & Ladwig, P. (2014). The “nasty effect:” Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19(3), 373–387. http://doi.org/10.1111/jcc4.12009 https://doi.org/10.1111/jcc4.12009
Arnold, K. (2008). Qualität im Journalismus – ein integratives Konzept [Quality in journalism – an integrative concept]. Publizistik, 53(4), 488–508. http://doi. org/10.1007/s11616-008-0012-y
Ash, E., Hettinga, K., & Halpern, D. (2009). Effects of a trend: The influence of user comments on readers’ perceptions of online newspapers. Presented at the Annual meeting of the Association for Education in Journalism and Mass Communication, Boston.
Friemel, T. N., & Dötsch, M. (2015). On- line reader comments as indicator for perceived public opinion. In M. Emmer & C. Strippel (Eds.), Kommunikationspolitik für die digitale Gesellschaft (pp. 151–172).
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62(1), 451–482. http://doi.org/10.1146/annurev-psych-120709-145346 https://doi.org/10.1146/annurev-psych-120709-145346
Hackel-de Latour, R. (2015). „Lügenpresse“!? Über den Glaubwürdigkeitsverlust der Medien [„Lying press“!? About the loss of credibility in media]. Communicatio Socialis, 48(2), 123–125. https://doi.org/10.5771/0010-3497-2015-2-123
Hagen, L. (2015). Nachrichtenjournalismus in der Vertrauenskrise. “Lügenpresse” wissenschaftlich betrachtet [News journalism in a crisis of confidence. “Lying press” from a scientific point of view]. Communicatio Socialis, 48(2), 152–163. Hermida, A. (2011). Mechanisms of participation. How audience options shape the conversation. In J. B. Singer, A. Hermida, D. Domingo, A. Heinonen, S. Paulussen, T. Quandt, … M. Vujnovic (Eds.), Participatory journalism guard- ing open gates at online newspapers (pp. 11–33). Chichester, West Sussex, U.K.; Malden, MA: Wiley-Blackwell.
Houston, J. B., Hansen, G. J., & Nisbett, G.
S. (2011). Influence of user comments on perceptions of media bias and third-person effect in online news. Electronic News, 5(2), 79–92. http://doi. org/10.1177/1931243111407618 https://doi.org/10.1177/1931243111407618
Jungnickel, K. (2011). Nachrichtenqualität aus Nutzersicht. Ein Vergleich zwischen Leserurteilen und wissenschaftlich-normativen Qualitätsansprüchen [News quality from the recipients‘ perspective. A comparison between readers’ assessments and scientific and normative quality standards]. Medien & Kommunikationswissenschaft, (3), 360–378.
h t t p : / / d o i. o r g / 1 0 . 5 7 7 1 / 1 6 1 5 - 634x-2011-3-360
Lee, E.-J. (2012). That’s not the way it is: How user-generated comments on the news affect perceived media bias. Journal of Computer-Mediated Communication, 18(1), 32–45. http://doi.org/ 10.1111/j.1083-6101.2012.01597.x https://doi.org/10.1111/j.1083-6101.2012.01597.x
Lee, J. (2014). Are some people less influ- enced by others’ opinions? The role of internal political self-efficacy and need for cognition in impression formation on social networking sites. Cyberpsychology, Behavior, and Social Networking. http://doi.org/10.1089/cyber. 2013.0713
Lee, J., & Lim, Y. (2014). Who says what about whom: Young voters’ impression formation of political candidates on social networking sites. Mass Communication and Society, 17(4), 553–572. http://doi.org/10.1080/15205436.2013. 816743
Marchionni, D. (2014). Online story commenting. An experimental test of conversational journalism and trust. Journalism Practice. http://doi.org/10.1080/ 17512786.2014.938943
Meyer, H. K., & Carey, M. C. (2014). In moderation. Examining how journalists’ attitudes toward online comments affect the creation of community. Journalism Practice, 8(2), 213–228. http://doi.org/ 10.1080/17512786.2013.859838 https://doi.org/10.1080/17512786.2013.859838
Prochazka, F., Weber, P., & Schweiger, W. (2016). Effects of civility and reasoning in user comments on perceived journalistic quality. Journalism Studies, 0(0), 1–17. http://doi.org/10.1080/1461670X.2016.1161497 https://doi.org/10.1080/1461670X.2016.1161497
Richardson, J. E., & Stanyer, J. (2011). Reader opinion in the digital age: Tabloid and broadsheet newspaper websites and the exercise of political voice. Journalism, 12(8), 983–1003. http:// doi.org/10.1177/1464884911415974 https://doi.org/10.1177/1464884911415974
Schatz, H., & Schulz, W. (1992). Qualität von Fernsehprogrammen. Kriterien und Methoden zur Beurteilung von Program- mqualität im dualen Fernsehsystem [Quality of TV programs. Criteria and methods for assessing program quality in the dual broadcasting system]. Media Perspektiven, (11), 690–712.
Springer, N., Engelmann, I., & Pfaffinger,
C. (2015). User comments: Motives and inhibitors to write and read. Information, Communication & Society, 0(0), 1–18. http://doi.org/10.1080/136911 8X.2014.997268
Thorson, K., Vraga, E., & Ekdale, B. (2010). Credibility in context: How uncivil online commentary affects news credibility. Mass Communication and Society, 13(3), 289–313. http://doi.org/ 10.1080/15205430903225571 https://doi.org/10.1080/15205430903225571
Urban, J., & Schweiger, W. (2014). News quality from the recipients’ perspective: Investigating recipients’ ability to judge the normative quality of news. Journalism Studies, 15(6), 821–840. http://doi. org/10.1080/1461670X.2013.856670 https://doi.org/10.1080/1461670X.2013.856670
Wellbrock, C.-M., & Klein, K. (2014). Jour- nalistische Qualität – eine empirische Untersuchung des Konstrukts mithilfe der Concept Map Methode [Journalistic quality – an empirical investigation of the construct employing the concept map method]. Publizistik, 59, 387–410. http:// doi.org/10.1007/s11616-014-0212-6 https://doi.org/10.1007/s11616-014-0212-6
Wolling, J. (2009). The effect of subjective quality assessments on media selection. In T. Hartmann (Ed.), Media choice: A theoretical and empirical overview (pp. 84–101). New York, London: Routledge.
Yale, R. N., Jensen, J. D., Carcioppolo, N., Sun, Y., & Liu, M. (2015). Examining first- and second-order factor structures for news credibility. Communication Methods and Measures, 9(3), 152–169. http://doi.org/10.1080/19312458.2015. 1061652
Ziegele, M., Breiner, T., & Quiring, O. (2014). What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on news items. Journal of Communication, 64(6), 1111–1138. http://doi.org/10.1111/jcom.12123 https://doi.org/10.1111/jcom.12123
Ziegele, M., Johnen, M., Bickler, A., Ja- kobs, I., Setzer, T., & Schnauber, A. (2013). Männlich, rüstig, kommentiert? Einflussfaktoren auf die Aktivität kommentierender Nutzer auf Online-Nachrichtenseiten [Male, hale, comments? Factors influencing the activity of commenting users on online news websites]. Studies in Communication | Media, 2(1), 67–114. https://doi.org/10.5771/2192-4007-2013-1-67

Abstract

The E-Journal published by the German Communication Association (Deutsche Gesellschaft für Publizistik- und Kommunikationswissenschaft, DGPuK e.V.) aims to present a full range of studies in communications and media on a high level of quality. It therefore seeks to publish new research results in the field of media and communication sciences, gives a general overview of the international state of research and create a forum for interdisciplinary discussions.

The journal publishes top blind peer-reviewed  papers in German and English; German contributions will be accompanied by an extended, quotable abstract in English.

Website: www.scm.nomos.de

Zusammenfassung

Studies in Communication | Media ist das E-Journal der Deutschen Gesellschaft für Publizistik- und Kommunikationswissenschaft (DGPuK) e.V. Es präsentiert Beiträge und Analysen aus allen Forschungsrichtungen der Disziplin. Ziel ist es insbesondere, empirische wie theoretische Studien der Medien- und Kommunikationswissenschaft zu veröffentlichen und mit aktuellen Erkenntnissen Impulse zu setzen, den internationalen Forschungsstand zu reflektieren und über neue Entwicklungen und Kontroversen zu informieren. Durch interdisziplinäre Beiträge wird ein Forum geschaffen, das wissenschaftliche Perspektiven aufzeigt und den Austausch fördert.

Homepage: www.scm.nomos.de