Media Sustainability Index (MSI) Methodology

[6]IREX prepared the Media Sustainability Index (MSI) in cooperation with the United States Agency for International Development (USAID) as a tool to assess the development of media systems over time and across countries. IREX staff, USAID, and other media-development professionals contributed to the development of this assessment tool.

The MSI assesses five "objectives" in shaping a successful media system:

1. Legal and social norms protect and promote free speech and access to public information.
2. Journalism meets professional standards of quality.
3. Multiple news sources provide citizens with reliable, objective news.
4. Media are well-managed enterprises, allowing editorial independence.
5. Supporting institutions function in the professional interests of independent media.

These objectives were judged to be the most important aspects of a sustainable and professional independent media system, and serve as the criteria against which countries are rated. A score is attained for each objective by rating between seven and nine indicators, which determine how well a country meets that objective. The objectives, indicators, and scoring system are presented below.

The scoring is done in two parts. First, a panel of local experts is assembled in each country, drawn from the country’s media outlets, nongovernmental organizations (NGOs), professional associations, and academic institutions. Panelists may be editors, reporters, media managers or owners, advertising and marketing specialists, lawyers, professors or teachers, or human rights observers. Additionally, panels comprise the various types of media represented in a country. The panels also include representatives from the capital city and other geographic regions, and they reflect gender, ethnic, and religious diversity as appropriate. For consistency from year to year, at least half of the previous year’s participants are included on the following year’s panel. IREX identifies and works with a local or regional organization or individual to oversee the process.

Panel participants are provided with a questionnaire that explains the objectives, indicators, and scoring system. Each panelist individually reviews the questionnaire and scores each indicator. Descriptions of each indicator explain their meaning and help organize the panelist’s thoughts. For example, the questionnaire asks the panelist to consider not only the letter of the legal framework, but its practical implementation, too. A country without a formal freedom-of-information law that enjoys customary government openness may well outperform a country that has a strong law on the books that is frequently ignored. Furthermore, the questionnaire does not single out any one type of media as more important than another; rather it directs the panelist to consider the salient types of media and to determine if an underrepresentation, if applicable, of one media type impacts the sustainability of the media sector as a whole. In this way, we capture the influence of public, private, national, local, community, and new media.

The panelists then assemble to analyze and discuss the objectives and indicators. While panelists may choose to change their scores based upon discussions, IREX does not promote consensus on scores among panelists. The panel moderator, in most cases a representative of the host-country institutional partner or a local individual, prepares a written analysis of the discussion, which is subsequently edited by IREX editorial staff. Names of the individual panelists and the partner organization or individual appear at the end of each country chapter.

IREX editorial staff review the panelists’ scores, and then score the country independently of the MSI panel. This score carries the same weight as an individual panelist. The average of individual indicator scores within each objective determines the objective score, and the average of the five objectives determines the overall country score.

In some cases where conditions on the ground are such that panelists might suffer legal retribution or physical threats as a result of their participation, IREX will opt to allow some or all of the panelists and the moderator/author to remain anonymous. In severe situations, IREX does not engage panelists as such; rather the study is conducted through research and interviews with those knowledgeable of the media situation in that country. Such cases are appropriately noted in relevant chapters.

Changes and Additions in 2011

Between 2001 and 2010 IREX used the same objectives and indicators without any changes. In the MSI's tenth year, IREX drew on our experience using this methodology in three regions, Africa, Europe and Eurasia, and the Middle East, to refine our methodology. Based upon the comments from our panelists during panel discussions, IREX felt that certain concepts required clarification and amplification. Changes in technology over time required more direct language to show that our studies have captured its impact on the media sector and ensure that panelists continue to consider this in their deliberations. Finally, IREX intended from the beginning that the MSI not discriminate in favor of a country with a preponderance of public media or private media; regardless of ownership, the underpinnings of an effective media system are the same. Therefore, IREX clarified some language to ensure our intentions are clear in that regard.

Highlights of the Changes

Close inspection of the new objectives and indicators will reveal some subtle changes, and we invite users of the MSI to review these at their convenience. However, below is a summary of the key amendments and additions, with a short explanation.

Objective 1, indicator 2: Although international norms of media freedom frown upon licensing and/or registration of print media or online media, this nonetheless occurs in many countries. The original wording of this indicator singled out broadcast media to reflect IREX's belief that only media making use of a public good—the broadcast frequency spectrum—should be subject to licensing. The changed wording broadens the scope, yet the guiding questions in the questionnaire ask panelists to consider if any licensing or registration serves to protect a compelling public interest.

Objective 3, indicators 1 and 2: Changes made to these two indicators are intended to clarify the meaning of each and make each more distinctive. Indicator 1 covers the availability of different sources of news on different platforms and the diversity of viewpoints represented therein. Indicator 2 assesses any obstacles faced by citizens when trying to access domestic and foreign media, be those obstacles legal, socioeconomic, and/or infrastructural (e.g., inconsistent electrical supplies) in nature.

Objective 3, indicator 8: Ideally, citizens have access to news about their immediate area, neighboring communities, national developments, and international events. Further, such reporting should be contextualized: the media should analyze the impact of such developments for their audience in a way, for example, that an international satellite news channel cannot do. IREX felt that this concept was not receiving adequate consideration in panel discussions and added an indicator dedicated to it.

Objective 4: By changing the wording of this indicator, IREX intended to remove a perception of bias against public or non-profit media. IREX's intention from the beginning was to focus on good management and solid financial sustainability that encourages editorial independence of media outlets.

Objective 4, indicator 5: By broadening the language of this indicator to specifically include government advertising, which in some countries is the largest source of advertising revenue, IREX aims to more fully assess the government's impact on the media marketplace and how fairly it spreads public funds amongst the media. Previously, guiding language in the questionnaire asked panelists to consider government advertising, but this change makes IREX's intention more explicit.

Objective 5, indicator 8: Information and communications infrastructure is increasingly important to allow media to reach citizens and for citizens to serve as reporters or otherwise interact with the media. Citizens whose countries have poor resources in this area face disadvantages in this regard. IREX added an indicator to assess how well this infrastructure serves both the media and citizens. Indicator 7 under this objective, which previously also tried to cover this concept, is now solely dedicated to the control of these resources and the ability of media to access them without undue restrictions.

 

Impact on Scores of the Methodology Changes

In considering changes, IREX wanted to be sure that historic scores would maintain comparability to future sources. IREX did not see the need for radical additions;rather the intention of the changes was to ensure that MSI panelists properly assess the concepts already incorporated. However, adding indicators or changing language has had some minor impact on scores.

For example, adding an additional indicator in Objective 5 (previously seven indicators, now eight) to cover the information and communications infrastructure does allow for a relatively wealthy country with an advanced infrastructure but otherwise lackluster supporting institutions to perform better than in the past without any apparent change. However, the ability of one additional indicator to significantly change the average of seven other indicators is limited. Where the new indicators noticeably impact scores from previous years is noted in the introductory paragraph of the relevant objective in each country chapter.

Further, changes to the wording of the indicators had modest impact. For example, in years past panelists sometimes provided better scores than expected for Objective 4, indicator 5, which covered government subsidies for private media, if the government provided no such subsidies. Guiding text also asked panelists to consider government advertising, but experience showed that they probably did not do so to the extent IREX desired. Changing the wording of the indicator to also specify advertising has had a noticeable impact on the scores for this indicator, although it has not been enough to drastically impact Objective 4 scores.

The changes made to the methodology will result in more accurate reflections of the sustainability of a country's media sector and its ability to function as the "fourth estate." While IREX recognizes that scores are affected by these changes, the magnitude of the impact has been minimal in this first year of using the revised methodology and does not discredit comparisons to scores from past years.

I. OBJECTIVES AND INDICATORS

Objective #1: Legal and social norms protect and promote free speech and access to public information.

Indicators

1. Legal and social protections of free speech exist and are enforced.

2. Licensing or registration of broadcast media protects a public interest and is fair, competitive, and apolitical.

3. Market entry and tax structure for media are fair and comparable to other industries.

4. Crimes against media professionals, citizen reporters, and media outlets are prosecuted vigorously, but occurrences of such crimes are rare.

5. The law protects the editorial independence of state or public media.

6. Libel is a civil law issue; public officials are held to higher standards, and offended parties must prove falsity and malice.

7. Public information is easily available; right of access to information is equally enforced for all media, journalists, and citizens.

8. Media outlets' access to and use of local and international news and news sources is not restricted by law.

9. Entry into the journalism profession is free and government imposes no licensing, restrictions, or special rights for journalists.

Objective #2: Journalism meets professional standards of quality.

Indicators

1. Reporting is fair, objective, and well sourced.

2. Journalists follow recognized and accepted ethical standards.

3. Journalists and editors do not practice self-censorship.

4. Journalists cover key events and issues.

5. Pay levels for journalists and other media professionals are sufficiently high to discourage corruption and retain qualified personnel within the media profession.

6. Entertainment programming does not eclipse news and information programming.

7. Technical facilities and equipment for gathering, producing, and distributing news are modern and efficient.

8. Quality niche reporting and programming exists (investigative, economics/business, local, political).

Objective #3: Multiple news sources provide citizens with reliable, objective news.

Indicators

1. Plurality of public and private news sources (e.g., print, broadcast, Internet, mobile) exist and offer multiple viewpoints.

2. Citizens' access to domestic or international media is not restricted by law, economics, or other means.

3. State of public media reflect the views of the political spectrum, are nonpartisan, and serve the public interest.

4. Independent news agencies gather and distribute news for media outlets.

5.Private media produce their own news.

6.Transparency of media ownership allows consumers to judge the objectivity of news; media ownership is not concentrated in a few conglomerates.

7. A broad spectrum of social interests are reflected and represented in the media, including minority-language information sources.

8. The media provide news coverage and information about local, national, and international issues.

Objective #4: Media are well-managed enterprises, allowing editorial independence.

Indicators

1.Media outlets operate as efficient and self-sustaining enterprises.

2. Media receive revenue from a multitude of sources.

3. Advertising agencies and related industries support an advertising market.

4.Advertising revenue as a percentage of total revenue is in line with accepted standards.

5. Government subsidies and advertising are distributed fairly, governed by law, and neither subvert editorial independence nor distort the market.

6.Market research is used to formulate strategic plans, enhance advertising revenue, and tailor the product to the needs and interests of the audience.

7. Broadcast ratings, circulation figures, and Internet statistics are reliably and independently produced.

Objective #5: Supporting Institutions function in the professional interests of independent media

Indicators

1. Trade associations represent the interests of media owners and managers and provide member services.

2. Professional associations work to protect journalists' rights and promote quality journalism.

3. NGOs support free speech and independent media.

4. Quality journalism degree programs exist providing substantial practical experience.

5. Short-term training and in-service training institutions and programs allow journalists to upgrade skills or acquire new skills.

6. Sources of media equipment, newsprint, and printing facilities are apolitical, not monopolized, and not restricted.

7. Channels of media distribution (kiosks, transmitters, cable, Internet, mobile) are apolitical, not monopolized, and not restricted.

8. Information and communication technology infrastructure sufficiently meets the needs of media and citizens.

A. Indicator Scoring

Each indicator is scored using the following system:

0 = Country does not meet the indicator; government or social forces may actively oppose its implementation

1 = Country minimally meets aspects of the indicator; forces may not actively oppose its implementation, but business environment may not support it and government or profession do no fully and actively support change.

2 = Country has begun to meet many aspects of the indicator, but progress may be too recent to judge or still dependent on current government or political forces.

3 = Country meets most aspects of indicator; implementation of the indicator has occurred over several years and/or through changes in government, indicating likely sustainability.

4 = Country meets the aspects of the indicator; implementation has remained intact over multiple changes in government, economic fluctuations, changes in public opinion, and/or changing social conventions. 

 

B. Objective and Overal Scoring

The averages of all the indicators are then averaged to obtain a single, overall score for each objective. Objective scores are averaged to provide an overall score for the country. IREX interprets the overall score as follows:

Unsustainable, Anti-Free Press (0-1): Country does not meet or only minimally meets objectives. Government and laws actively hinder free media development, professionalism is low, and media-industry activity is minimal.

Unsustainable Mixed System (1-2): Country minimally meets objectives, with segments of the legal system and government opposed to a free media system. Evident progress in free-press advocacy, increased professionalism, and new media businesses may be too recent to judge sustainability.

Near sustainability (2-3): Country has progressed in meeting multiple objectives, with legal norms, professionalism and the business environment supportive of independent media. Advances have survived changes in government and have been codified in law and practice. However, more time may be needed to ensure that change is enduring and that increased professionalism and the media business environment are sustainable.

Sustainable (3-4): Country has media that are considered generally professional, free, and sustainable, or to be approaching these objectives. Systems supporting independent media have survived multiple governments, economic fluctuations, and changes in public opinion or social conventions.