Fact-Checking AI: Who can we trust?

Bo Brusco | June 12, 2021 (8-minute read)

Photo by Unsplash user @thenewmalcolm.

Trust is vital when it comes to journalism which is why, despite their success, news outlets like CNN and Fox News have a strong following but are also widely distrusted by specific demographics. If a news story is only as good as how trustworthy the name behind the story is, then artificial intelligence will have to earn the unanimous trust of the American people. 

Trustworthy Sources of News

In January of 2020, Pew Research conducted a study wherein participants ranked how much they trusted different news outlets. A total of 30 news sources were used in the study, with various names, including ABC News, Breitbart, BuzzFeed, CNN, Fox News, The Washington Post, and the New York Times. 

The study found that “More Democrats [...] trust than distrust most of the 30 outlets in the study, but the reverse is true among Republicans and GOP leaners.” The study also noted that “while Democrats’ trust in many of these outlets has remained stable or in some cases increased since 2014, Republicans have become more alienated from some of them, widening an already substantial partisan gap.”

While the study’s results were more or less predictable, with CNN losing more trust from Republicans and Breitbart being less trustworthy for Democrats, the study, in general, provides an interesting insight, tapping into the importance of trust. 

Instead of asking citizens which news outlets they think report the news most accurately, Pew Research specifically focused on the trust factor. During the study, Pew asked respondents first if they have heard of one of the 30 news outlets listed in the survey, next if they trusted that outlet for “for political and election-related news.” Lastly, if they expressed that they did not trust said news outlet, respondents were asked if they distrusted it. 

When deciphering whether or not one can trust a report published by any odd news company, there are a couple of steps to take that can ensure whether or not the said report is factual. These steps include fact-checking, cross-checking, and exposing biases. 

Fact-Check

The easiest way to fact-check something is to go to sites that are engineered to be just that: fact-checking destinations. These sites include FactCheck.org, Snopes.com, and Politifact.com—to name the more prominent ones. Of course, sometimes even the fact-checkers get it wrong. That is why it is important to cross-check. 

Cross-Check

Fact-checking is arguably the easiest place to start when questioning the validity of claims made in the news. It’s a convenient way to be sure that Americans are not getting bamboozled. If one wants to be very sure that they are dealing with factual claims, cross-checking with different sources is an additional step that can be taken. Of course, even cross-checking is not a surefire way to decipher the factual nature of a news report, but cross-checking a fact from multiple credible sources can be very useful when determining what to believe. 

Triangulation is another term for cross-checking and is a technique used primarily for qualitative or exploratory research. It “facilitates validation of data through cross verification from more than two sources.” This is a powerful analytical approach because “triangulation is not just about validation but about deepening and widening one’s understanding.” With a deepened understanding of the information being consumed by the general public, one can begin to decipher facts from the falsehoods used to promote the narratives of political tribes. 

Expose Biases

Of course, the surest anyone can possibly be is by confirming facts by cross-checking them from two opposing sources. It stands to reason that if two rival news publishers are broadcasting the same facts, those facts theoretically check out. For example, if CNN and Fox News both agree on something, then that is most assuring. People who expose themselves to competing biases are likely to get the most complete picture of a story. 

The Truth is All About Trust

While these steps may aid those in their quest for facts, the truth is that it all boils down to the basic principle of trust. Maintaining journalistic integrity is crucial for news outlets because once they misstep, or more appropriately, misprint, their readers’ confidence has been jeopardized. The Pew Research study mentioned earlier is a testament to how paramount trust is in the news industry. It is everything. 

Even if CNN were to publish a story right now that was factually accurate in every way, Republicans will still likely distrust it, and the same can be said about Democrats and Fox News. And understandably so, it is apparent that both news companies know their target audience and produce content to appease that audience’s biases. So, given the track records of CNN and Fox, it is no wonder that only specific demographics trust their word. 

The reason why trust overrules factual accuracy is due to the simple nature of the principle. A single line from a philosophy book written by Fabrice Midal perfectly depicts trust’s practical but vital function. “If I trust in what you tell me,” Midal writes, “it’s precisely because I can’t be absolutely certain of it.” At the end of the day, it is effectively impossible to know for certain how accurate that report is, and that is why the American public has to trust in the word of these news corporations. 

Another Case for AI Fact-Checking

It is tempting to chalk this whole issue up to “journalist integrity,” which has become a sort of buzzword in today’s political sector. But, as Matt Taibbi, who is a journalist himself, said in 2015, “Opinion can’t be extracted from reporting...Everything journalists do is a subjective editorial choice, from the size of headlines to the placement of quotes and illustrations.” It might sound insignificant, but even small details such as diction and syntax are susceptible to biases. Humans will always be subjects to subjectivity. 

Noting the reality of human nature, it may be prudent to share the responsibility of objectively reporting facts with non-human entities, namely artificial intelligence. Nathan Lambert, a graduate student in UC Berkley’s Robot Learning program, believes that AI can play a crucial role in helping distill fact from fiction. 

In an article he wrote for towardsdatascience.com, Lambert notes the difference between localized truth and global truth. As an example, he says, “A local truth for me and many of my closest friends is that rowing is a special sport that is unmatched due to its cooperation and high limits, but it’s definitely not true to everyone. A global truth is the Earth is round.” 

Attempting to explain the nuances of the world and its various truths, Lambert notes a potential kink with fact-checking AI. “The problem I see with internet fact-checkers is that users will want their local truths to be checked too,” he explains. “How do we collect a dataset of true versus untrue statements?”

Boiling the pot of potential obstacles down, Lambert points out two facts of the matter regarding AI’s current ability to fact-check. First, he acknowledges that AI won’t be able to fact-check everything down to fundamental scientific truths. Then, in a follow-up question, he asks, “who moderates the database?” People, who have within them their own implicit biases, will still have to form the data foundation from which AI will have to operate.

All of this means that even though AI will be a non-human way of human-made fact-checking reports, a group of humans will first have to build a foundation for the algorithms to work. It is crucial that a great majority of Americans trust the individuals who fulfill this role because for AI to be successful in its fact-checking position, it can’t appear bias or untrustworthy in any minute way.