Media Cloud

While the researchers behind Media Cloud are using the tools developed to address a set of research questions, the system has a broader purpose as a platform for open, collaborative research by scholars around the world. Key elements of the Media Cloud project include:

• Developing an open database of the topics of all news stories from an increasing number (eventually, thousands) of sources;
• Building lightweight, interactive tools that allow users to ask the database questions;
• Publishing open application programming interfaces (APIs; click here for an explanation) that give other researchers full access to the database;
• Publishing the code for the system under a free software license; and
• Publishing research generated from use of the database, including studies on media signatures, meme propagation, and geographic attention profiles.
  

This information and communication technology (ICT)-based system automatically codes a high volume of articles in a generalised fashion; new stories are automatically added, and the archive is maintained indefinitely. The Media Cloud system provides the results as a web service. This includes downloading and processing terabytes of news content. The service can generate visual or textual results based on user queries. For example, it can chart the terms that appear most frequently in the New York Times as compared to leading blogs, or it can generate a world map showing which countries generate the most media attention from any source. For each story from a given news source, the system automatically assigns relevant terms to that article. These terms, and the stories they describe, are then explored in relation to the rest of the interconnected network of media sources. The fact that sources cluster together around specific topics, or diverge can, Media Cloud developers explain, inform a diversity of research questions.

Specifically, Media Cloud code runs a web application that allows the user to manage a set of media sources and their feeds. It periodically crawls the feeds setup within the web app and downloads any new stories found within the downloaded feeds. Then, it extracts the substantive text from the downloaded story content (minus the ads, navigation, comments, etc.) and associates a set of tags with each story based on that extracted text. By visiting the Media Cloud website, one can choose a chart type, show the top 10 most mentioned terms for each media source, and show the top 10 most mentioned terms for each media source that occur in stories along with the specified term. A world map function enables the user to see a world map of each media source, with darker colors indicating more coverage of those countries.

In short, the Media Cloud system identifies relevant terms from news stories and allows analysis of that data. In order to understand how others might like to use the system, Media Cloud developers created an interactive process - accessible here. In this sense, the project is intended to be participatory.

Research, Technology.

The idea for Media Cloud emerged after a series of discussions in which Berkman Center researchers posed various questions about the media landscape, such as: Do bloggers introduce storylines into mainstream media or the other way around? What parts of the world are being covered or ignored by different media sources? Where do stories begin? How are competing terms for the same event used in different publications? Can we characterise the overall mix of coverage for a given source? How do patterns differ between local and national news coverage? Can we track news cycles for specific issues? Do online comments shape the news?

Media Cloud is intended to offer a way to quantitatively examine these and other research questions. "Traditional media content analysis involves time-consuming manual coding. This covers only a small subset of content, and it categorically rules out certain types of analysis. Such methods typically cannot support new research questions without re-coding the content. The Media Cloud approach is fundamentally different in nature and scope..."

As of this writing, the system is tracking many national (US) news sources, and the catalog continues to build. As the process evolves, Media Cloud should be able to simultaneously track thousands of international and local news sources, providing fine-grained data about how they are evolving. "By combining massive data collection with novel clustering techniques, we may be able to identify thousands of instances of new ideas emerging from one corner the media ecosystem and spreading to other parts - or failing to spread. Media Cloud will eventually allow researchers to more clearly see the relationships between media, and pierce through the cloud of data overload."