Using Cosine Similarity to compare week to week coverage in Russian media
What are the differences in how various Russian media outlets – traditional and web native – cover events? How does coverage differ between sources during the same time period? Does coverage overlap – or do different outlets highlight different events? What do these choices tell us about media outlet priorities and preferences?
With these questions in mind, we used Russian Media Cloud to determine the levels of similarity between four Russian media sets – Russian government websites, Russian television news, Russian mainstream media websites and popular Russian blogs - over two weeks in April 2011. We use cosine similarity to determine how similar the content of the media sets are to one another, a method described in this earlier post by Ethan Zuckerman.
Comparing the four media sets lead to six possible pairings – TV vs. MSM, MSM vs. Popular Blogs, TV vs. Popular Blogs, MSM vs. Government, TV vs. Government and Government vs. Popular Blogs. In the table below, we have listed these six pairings by similarity scores from most similar to least similar over the two weeks (where 1 would equal perfect similarity between two media sets, and 0 would equal perfect dissimilarity).
We selected the weeks of April 4th and April 18th in particular because of a difference in breadth of coverage – the word clouds on April 4th had relatively high similarity scores whereas on April 18th, the scores were comparatively low. We were curious to see what this distinction in similarity scores between sources would tell us about the sources themselves, and the events they chose to cover.
|1||TV vs MSM||0.85||0.484|
|2||MSM vs Pop Blogs||0.712||0.44|
|3||TV vs Pop Blogs||0.698||0.442|
|4||MSM vs Gov||0.629||0.177|
|5||TV vs Gov||0.578||0.268|
|6||Gov vs Pop Blogs||0.492||0.144|
Our hypothesis was that during one time period a set of major news stories may dominate all the media outlets, where as at other times, there are particular stories that are of interest to a specific media set.
A closer examination of the word clouds from each media set provides insight into which events and angles were featured by which type of source. Below, we have listed word clouds for each of the four media sets (TV, MSM, Popular Blogs and Government) during each of the two weeks studied. We’ve provided English translations of leading key words. Note that we are still refining the stoplist, extractor and parsor, so that unusual text may occasionally appear.
As Bruce Etling qualified in an earlier post – “Media Cloud outputs alone do not say anything about the meaning behind the differences between different sources. However, with additional context about what we know of the political situation and media ownership in Russia, as well qualitative analysis of sentences within queries, we can begin to hypothesize about the possible meaning behind similarity scores, word clouds, polar maps and other automated outputs.”
In this case, our findings support our initial theory that similarity scores are higher when there is a prominent news story or event, and lower when no one set of news events dominates the agenda. Specifically, on April 4th, all media sets focused on four key stories, while on April 18th, the sources each highlighted separate, more minor, stories. In addition, this initial experiment found that despite the distinction in events, coverage or similarity ratios between weeks, the media outlets themselves appear to exhibit a specific pattern of preferences in terms of stories, news agenda and style.
During the two weeks studied, we found that television news, for example, was drawn to coverage of patriotic items and pageantry – such as celebrations of the 50th anniversary of Russia (and the world’s) first human space flight, and the church ceremonies surrounding the Easter holidays. During the same time period, the online mainstream media provided news of leading international current events such as those occurring in Japan or Libya, as well as more domestically focused tragedies and scandals, such as bombings in Minsk, or prominent kidnappings in Russia. Russian government sites are mostly focused on official discussions of protocol, procedures and regulations. And finally, leading Russian blogs were generally full of discussions of personal and lifestyle topics, as well as personal perspectives on major national events such as holidays. The blogosphere is also a source of topics and discussions that were not featured elsewhere – such as discussions of the DDOS attacks on LiveJournal, and various debates surrounding the rights and statuses of minority ethnic groups.
April 4, 2011
This week, the similarity scores of all four key media sets were relatively high, indicating a high level of cosine similarity. This could be attributed to the fact that four key stories dominated the Russian media this week, although each set of sources covered a different specific set:
|1||Upcoming 50th anniversary of Yuri Gagarin’s space flight on April 12, 2011||Russian TV
|2||Gadaffi and Libya||Russian TV
(State of Fukushima Nuclear reactor – Russian TV only)
|4||DDOS attacks on Live Journal||Blogs only|
The distribution of these stories within Russia’s media landscape is unsurprising. For instance, a “patriotic” item such as the upcoming anniversary of the first human/ Russian in space and the Russian space program in general, was covered by both the Russian government sites as well as Russian television. This was the only “current event” prominently featured on the government websites. MSM and the blogosphere were less interested in the topic – but maybe because their coverage is more current, and the actual anniversary was not until April 12.
Prominent international current events surrounding Libya, Gadaffi, and Japan occupied both Russian television and the MSM. Russian TV also discussed on the Fukushima reactor, although MSM did not.
On the blogosphere, personal and lifestyle topics related to children and family were more common. In terms of “current events” the blogosphere was mostly occupied with the DDOS attack on LiveJournal – a story that did not appear in any of the other outlets – neither the MSM, nor Television nor Government sites.
APRIL 18, 2011
This week, the similarity scores among the four key media sets were comparatively low, indicating a low level of cosine similarity. The two most similar media outlets this week are still farther apart than least similar from the week of April 4th. While one term/event – Easter – was appeared in all the word clouds, it was accompanied by different words for each media set. In addition, there were at least six other topics that were featured in some media sets and not others.
|1||Easter Sunday – April 24, 2011
- Coverage of Easter dominated TV, which was focused on “official celebration” of Easter, in Moscow’s churches etc, citing Patriarch Cyrill, official celebrations etc. Bloggers were more focused on the holiday itself and family, personal meanings.
- MSM discussed Easter, but were full of a number of other current events.
- Government sites also mentioned the holiday.
|ALL – Mostly TV and Blogs, but also some MSM and Gov|
|2||Kidnapping of Russian “Cyber security mogul” Kaspersky’s son (later freed)||MSM|
|3||April 11, 2011 Minsk Metro bombing/explosion||MSM, some TV|
|4||“Mistral” – The complex politics surrounding the negotiations of Russian purchase of Mistral class helicopter carriers from France||MSM|
|5||Customs, border, protocol etc – Newly organized Customs Union of Russia, Ukraine, Kazakhstan and Belarus||Government|
|6||Caucasus – two topics occurring at the same time led this word to rise to prominence – ethno-nationalist protests against people from the Caucasus and a recent law exempting Caucasus residents from Russian military service||Blogosphere|
The upcoming Easter celebrations were covered by all four sources, although each one addressed the topic from different angles and using different language. Television in particular was dominated by coverage of official Easter celebrations and related pageantry, to the detriment of other news or current events.
The blogosphere was also caught up in discussions of the Easter holiday – although from a more personal perspective. In addition, bloggers discussed two separate events involving the term “Caucuses,” which cause the word to appear in the cloud – ethno-nationalist protests against people from the Caucasus and a recent law exempting Caucasus residents from Russian military service. These topics were not covered by either Television or MSM – at least not to a significant enough degree to allow the term to appear in the cloud.
The Mainstream Media did mention the Easter Holiday, but they were more focused on current events. Three stories in particular were prominent on the MSM, but did not appear to receive significant coverage elsewhere. These included the kidnapping (and subsequent release) of the son of “Cyber security mogul” Kaspersky, the metro bombing in Minsk and the negotiations surrounding the Russian purchase of Mistral class helicopter carriers from France.
Russian government sites mentioned Easter, but were more focused on the newly organized Customs Union involving Russia, the Ukraine, Kazakhstan and Belarus.
The process described above demonstrates approaches where comparing cosine similarity scores between media sets can be useful. An examination of the similarity scores and word clouds during two highly divergent weeks indicates that sources may be more similar when a set of stories or events dominates the agenda, and more distinct when there is no one overwhelming set of events or perspectives. It would be interesting to repeat this approach over future weeks to see if the initial theory continues to hold.
In addition, during the two weeks highlighted above, we found that each media set or media type appears to exhibit stylistic tendencies and news agenda preferences particular to its needs and audience. Our preliminary results found Russian government sites to be mostly focused on official discussions of protocol, procedures and regulations, while television news highlighted patriotic events and pageantry. Online mainstream media covered leading international current events, as well as more domestically focused tragedies and scandals, while leading Russian blogs focused on personal and lifestyle topics, and also featured topics and discussions that did not appear elsewhere.
APRIL 4, 2011
Politics & Finance – Russia, Russian, Country, President, Moscow, Government, Regions, USA, Rubles, Banks, million, business, Military, Service, Dmitri, Medevedev, Vladimir, Segei, Alexander, Gadaffi, Libya, Japan
2) RUSSIAN TV
Politics – Russia, Russian, Country, President, Moscow, USA, Government, Region, International, Minister, Medvedev, Dmitri, Military, Service
Finance – Rubles, Market, Bank, Dollars
Misc – automobiles, cars, roads
Gagarin 50th anniversary – Cosmos/Space, Program, Gagarin, Flight, Astronaut
Intl Current Events – Gaddafi, Libya; Japan, Fukushima
3. POP Blogs
General Politics/Finance – Russia, Russian, Country, Moscow, President, Wars, Government, Dollars, Rubles
Lifestyle - Film, Kids, Roads, Schools
Technology – File, Program, Users, Free (as in download), Site, Internet
Specific – Attacks, Live Journal
Official & Political – Federation, Federal, Government, Russia, Russian, President, Regions, Putin Medvedev, Mayor, International, Administration, safety, internal, military, general, ministry, service, organs (as in “of the government”), Rubles, Economics, workers, organizations, education, Cosmic/Space
April 18, 2011
1. RUSSIAN TV
Easter Holiday – Easter, Patriarch Cyrill, Church, Christ, Service, Saint, Holiday, believers, eggs
General Politics – Moscow, President, Moscow Mayor Sobyanin
Minsk bombing – explosion
Easter – Easter, Services
Minsk bombing – explosion, Lukashenko, killed, suffering
Kaspersky kidnapping – Kaspersky’s, son
Helicopter carrier negotiations – Mistral
Libya – Gaddaffi
3. Pop Blogs
Easter – Easter, holiday, Christ, rose (verb), egg, beliefs,
Politics/Events – economics, Russian, Caucaus
Technology – file, torrent, users, client, free, automatic, download
Official – Government, Federation, Russian, administration
New Customs Union – customs, changes, border, ratification, protocol, points
Easter – Patriarch, Cyrill, Church