Labeling the credibility of news sources does not distract news consumption from low-quality sources or reduce belief in widely circulated inaccurate claims among average internet users, but does provide an indicator of source quality can improve the quality of the diet of the biggest consumers of misinformation, shows a new study from the Center for Social Media and Politics at New York University.
Notably, the researchers also found that a majority of people rely on credible news sources, with two-thirds avoiding unreliable news sites altogether.
The study, published in the journal Scientists progresscentered on credibility ratings determined by NewsGuard, a browser extension that rates news and other informational sites to guide users in assessing the trustworthiness of content they encounter online.
“While it is encouraging that most of us rely on credible sources of information, many turn to sites of questionable reliability, raising concerns about misperceptions that people can have,” says Kevin Aslett, a postdoctoral fellow at the Center for Social Media and Politics (CSMaP) and the paper’s lead author. “However, although our study shows that, overall, credibility ratings have no discernible effect on the misperceptions or online news consumption behavior of the average user, our results suggest that biggest consumers of misinformation—those who rely on low-credibility sites—may move to higher-quality sources when presented with reliable news ratings.”
In the study, conducted in May and June 2020, researchers encouraged a random sample of more than 3,000 online participants to install the NewsGuard browser extension, which embeds source-level information reliability indicators into users’ search engine results pages, social feeds, and URLs visited. Various “shield” symbols are placed in the stream to provide visual summaries of source quality:
- Green shield = trusted source
- Red shield = unreliable source
- Gray shield = source with user-generated content
- Golden shield = satire
To measure the effect of these source labels, survey data was collected over two time periods (May 28 to June 9 and June 19 to June 30).
In addition to this panel survey, the researchers also collected anonymized digital trace data to characterize the quality news consumption of a subset of approximately 1,000 participants. These quality measures used the same NewsGuard ratings that were presented to study participants when they encountered news links in their browsers.
Previous studies have shown that exposure to misinformation and fake news online can increase belief in misperceptions, increase cynicism toward politics, reduce trust in the media, and increase affective polarization (i.e. i.e. contempt for supporters of the other political party). Additionally, previous research also suggests that ratings from expert sources, when displayed alongside a fake news article, can influence the perceived veracity of the article’s claim.
With these earlier findings in mind, the researchers tested whether food source reliability labels could counteract these effects by shifting the consumption of information from unreliable to more reliable sources, increasing trust in mainstream media and trusted sources, and/or mitigating political polarization and cynicism.
In order to measure whether information about the reliability of sources affects belief in misinformation as well as in accurate claims, respondents were asked to judge the veracity of five widely circulated statements about the Black Lives Matter movement and five statements as well. well aired on COVID-19–real and fake.
Combining panel survey data and individual-level web visit data, the results showed the following:
- A majority of people have reliable media diets: Most people (65%) have not visited any unreliable news sites before the study began, a finding consistent with the authors’ previous research. In fact, only 1.5% of respondents relied heavily on unreliable sources of information.
- Source credibility labels had no impact, on average: In-browser credibility labels failed to measurably shift online consumption from unreliable to more trusted sources, failed to significantly change misperceptions of widely circulated inaccurate claims on COVID-19 and the Black Lives Matter movement, and have not damaged trust in the media in general.
- However, source tags have made a difference for those who consume less low-quality news: According to NewsGuard scores, there was a noticeable improvement in the overall credibility of news sources visited by those who started the study with the lowest diet quality.
“In our partisan age, where attitudes toward news sources are strongly correlated with partisanship, relatively subtle cues like source credibility labels may not be powerful enough to change news habits and counter public misperceptions,” observes Andrew M. Guess, a professor. researcher affiliated with CSMaP and assistant professor at Princeton University. “However, a key indicator of the success of this intervention is how it changes the behavior of those who consume the least amount of poor quality information. The fact that it does not work for the general population does not mean that the tool is ineffective. It means it needs to be part of a much larger toolkit to combat the spread of misinformation online.”
Co-authors of the paper were Joshua A. Tucker and Jonathan Nagler, professors in NYU’s Department of Politics, and Richard Bonneau, professor in NYU’s Department of Biology and the Courant Institute of Mathematical Sciences. Tucker, Nagler and Bonneau are co-directors of CSMaP.
NewsGuard was not involved in the design or funding of this research.