NewsGuard wins Pentagon-State Department contest for detecting COVID-19 misinformation and disinformation
Company Known for “Red” and “Green” Website Ratings Demonstrates How Its Human Intelligence Solution Can Work with AI Tools Through Its Misinformation Fingerprints™
(August 17, 2020—New York City) A company founded on the notion that human intelligence can be the vital ingredient in rooting out online misinformation and disinformation has bested 15 technology companies in a contest run by the Pentagon and Department of State to offer solutions to hoaxes related to the COVID-19 pandemic.
NewsGuard, which deploys journalists to issue trust ratings for thousands of news and information websites across the U.S. and Europe, flagging misinformation and disinformation narratives and sources in the process, was named a winner of the Countering Disinformation Challenge, a contest offered jointly by State Department and the Department of Defense (DoD) as a part of the DoD’s National Security Innovation Network (NSIN).
The contest sought solutions that would help the State Department and DoD “evaluate disinformation narrative themes in near real time” by identifying online sources spreading COVID-19 disinformation or misinformation narratives, understanding the nature and possible motives of those sources, and flagging hoaxes, narratives, and sources of disinformation as they emerge.
To meet this challenge, NewsGuard offered a “human intelligence” solution with two key components:
- Access to NewsGuard’s constantly updated database of journalist-produced ratings and “Nutrition Labels” for thousands of news and information websites in the U.S. and Europe. The database, available through an API, a browser extension and mobile apps, covers all the sites accounting for 95% of engagement in each market and includes numerous examples of state-sponsored misinformation sites, sites with unclear funding or ownership that publish disinformation narratives, and other sources of misinformation.
- NewsGuard’s growing database of Misinformation Fingerprints™, a catalogue of known hoaxes, falsehoods and misinformation narratives that are spreading online. The dataset for each narrative includes a description of each hoax and why it is false, examples of stories, social media posts, videos, and other content containing the hoax, and related keywords and hashtags. Because these elements—language, hashtags, etc.—are unique to each hoax, each “fingerprint” becomes a unique identifier that can be combined with AI and social listening tools to identify the initial source of the hoax and instances in which the hoax is repeated or amplified on social networks or the open web.
As a winner, NewsGuard will receive $25,000 to conduct a pilot and will work with the State Department’s Global Engagement Center to scope and develop a test in support of the DoD’s Cyber National Mission Force. Two other companies, PeakMetrics, which offers a dashboard for tracking mentions of a topic across multiple media channels, and Omelas, which offers a product for visually mapping online information, were also named winners of the contest.
With the cybersecurity industry populated largely by AI and machine-learning tools, NewsGuard’s human intelligence approach proved a unique and valuable offering. “We were surprised to see that a solution like NewsGuard’s using human intelligence could work so well,” said Dan Madden, who was one of the judges and is the Director for the Mid-Atlantic Region of the Pentagon’s National Security Innovation Network (NSIN).
“We’ve always believed that any effective solution for countering misinformation and disinformation would have to start with a substantial ‘human intelligence’ component because, unlike pornography or hate speech, it takes humans effectively to root out misinformation and disinformation,” said NewsGuard Co-CEO Gordon Crovitz. “This contest has helped affirm that idea. With our Misinformation Fingerprints database, we’ve created a human intelligence product that can work seamlessly with the wide range of powerful AI and social listening tools that now exist in the market.”
“The technology to search content across platforms using powerful AI models has existed for some time,” added NewsGuard Co-CEO Steven Brill. “What we’ve provided is the missing piece of the puzzle: A detailed ‘fingerprint’ for each misinformation or disinformation narrative that can serve as the unique identifier for that narrative.”
Given its process for monitoring and evaluating the credibility of thousands of news and information websites worldwide, NewsGuard is uniquely positioned to provide such a dataset.
In describing the background for this content, the State Department and Pentagon explained, “Both Russia and China appear to be engaged in COVID19 related disinformation campaigns. Though these particular campaigns are new, the problem of disinformation and misinformation in national security affairs is much broader and more enduring.”
The competition stressed the need for identifying hoaxes and misinformation in advance—what NewsGuard calls its “prebunking” of hoaxes from its already identified sources of misinformation—or in “near real time,” as NewsGuard’s Rapid-Response Team of analysts are able to do when misinformation emanates from a source the analysts have not previously rated. Until now, analysts have only had what the State Department and Pentagon called “after the fact” identification of hoaxes, which has limited the effectiveness of AI and other tools.
The background published by the State Department and Pentagon for this contest explained:“Competition in the information environment includes physical action and verbal activities in support of overarching and sub-narratives. In some cases, narratives are enduring while others are limited in duration. Current analysis of activity in the information environment is retrospective and focused on discrete events. The final product is an after the fact assessment and typically based on analysis of limited and select social media. This discrete analysis inhibits practitioner’s ability to understand broader trends in the information environment and evaluate effectiveness of friendly and adversary competing narratives.”
It further explained: “Effective operations in the information environment require the scalable ability to recognize and visualize malign and subversive narratives in near real time. Relevant evaluation includes recognition that state and non-state actors communicate and reinforce narratives through physical activity, physical meetings, public statements, social, and traditional media. Effective identification and evaluation of these narratives include integration of quantitative and qualitative analysis. Evaluation should also involve an understanding of narrative variation by region and target audience. This includes analysis and differentiation between misinformation and disinformation and recognition of persistent, emerging, opportunistic, and declining narratives. Further details include understanding of target audiences where select narratives are resonating, the role of select audiences in amplification or diminishment of narratives and connectivity between audiences.”
“When a new hoax pops up claiming that COVID-19 was planned or that it can be cured through a dangerous, unproven treatment, there is an extremely high likelihood that it will either originate from or quickly appear on websites our team already has rated and flagged as publishing misinformation,” explained NewsGuard general manager Matt Skibinski.” In this way, we ‘prebunk’ many of the leading hoaxes by having already signaled that the source is suspect. And if the hoax appears on a new site and begins to trend, our Rapid Response Team will see it immediately in our monitoring and alert systems. This gives us a powerful bird’s eye view of the misinformation landscape—and sets us up to provide a powerful, up-to-date catalogue of misinformation narratives that are spreading across the internet.”
NewsGuard has prebunked all the top online hoaxes, including those related to COVID-19 and many other healthcare hoaxes, often linking them to disinformation perpetrated by elements of the governments of Russia, China, Iran and others.
NewsGuard’s ratings and Nutrition Labels as well as its Misinformation Fingerprints™ product are available for licensing by governments, cybersecurity companies involved in threat intelligence, social media and other digital platforms seeking to identify misinformation and inauthentic accounts on their platforms and other entities seeking to restore trust to content on the internet.
Launched in March 2018 by award-winning journalist and media entrepreneur Steven Brill and former Wall Street Journal publisher Gordon Crovitz, NewsGuard provides credibility ratings and detailed “Nutrition Labels” for thousands of news and information websites. NewsGuard rates all the news and information websites that account for 95% of online engagement across the U.S., U.K., Germany, France, and Italy.
NewsGuard rates each site based on nine apolitical criteria of journalistic practice, including whether a site repeatedly publishes false content, whether it regularly corrects or clarifies errors, and whether it avoids deceptive headlines. It awards weighted points for each criterion and sums them up; a score of less than 60 earns a “Red” rating, while 60 and above earns a “Green” rating, which indicates it is generally reliable. In the process, it catalogues the most prominent falsehoods and misinformation narratives spreading across these sites, capturing data and examples about each hoax that can be used as a “fingerprint” for the hoax by AI and machine learning tools.
NewsGuard’s ratings and Nutrition Labels can be licensed by cybersecurity companies, social media and search companies, internet service providers, browsers, news aggregators, and others in order to make NewsGuard’s information about news websites available to their users. These ratings are made available to consumers through its browser extension, which is available on Chrome, Safari, Edge, and Firefox browsers, on Android devices through a mobile app, and on iOS or Android devices through the Edge mobile browser. Hundreds of libraries globally use NewsGuard’s free media literacy browser extension on their public-access computers to give their patrons more context for the news they encounter online. For more information, including to download the browser extension and review the ratings process, visit newsguardtech.com.
- Steven Brill, Co-CEO, firstname.lastname@example.org, +1 212-332-6301
- Gordon Crovitz, Co-CEO, email@example.com, +1 212-332-6407
- Matt Skibinski, General Manager, firstname.lastname@example.org, +1 973.818.4698