Validity and Compatibility of the SAM and KLD Screening Instruments

• 1. Introduction

  To date corporate sustainability performance¹xIn this paper the term corporate sustainability performance throughout is used as a generic term expressing the business-society relationship. Corporate sustainability performance relates to business’ performance in areas like corporate governance, social relations, contributions to a healthy environment and the creation of economic value. is a very popular concept and a key issue in many companies. Hardly a day goes by without newspapers or magazines devoting attention to this subject, along with many websites, blogs and other social media informing us about all kinds of developments related to corporate sustainable performance.
  However, if you would ask an average entrepreneur or corporate manager to give a description of corporate sustainability performance, you may find that he or she may not be able to answer the question. Others may be able to tell you that corporate sustainability performance is about companies contributing to a sustainable society. Again others may refer to John Elkington’s²xJ. Elkington, Cannibals with Forks. The Triple Bottom Line of the 21st Century Business, New Society Publishers, Gabriola Island, BC 1999. PPP-alliteration (People, Planet and Profit), implying that a company has to balance its social, ecological and economic performances. Others may offer an entirely different perspective.
  It may even be more difficult for an average entrepreneur to connect corporate sustainability performance to his or her concrete business context. In itself this is quite remarkable since the corporate sustainability performance debate already has a long and rich tradition.³xA.B. Carroll, “The Pyramid of Corporate Social Responsibility: Toward the Moral Management of Organizational Stakeholders”, 34 Business Horizons 4, 1991, pp. 39-48. To date, a multitude of corporate sustainability performance constructs exist.⁴xN. Dentchev, To What extent is Business and Society Literature Idealistic?, Working Paper 2004/245, Ghent University. However, it is not clear whether these are indicative of a definitional chaos or an evolutionary process, as we have seen signs of both. The overwhelming amount of constructs combined with a disparity of definitions and interpretations has undoubtedly contributed to the corporate sustainability performance confusion among entrepreneurs and managers.

  On the other hand, we have also seen the emergence of guidelines concerning how sustainability should be reported. These guidelines may be helpful to entrepreneurs and managers to get a better understanding of what corporate sustainability performance might look like in their business or trade. In 1999, Ranganathan counted as many as 48 of these guidelines, and this number has steadily increased since.⁵xJ. Ranganathan, “Signs of Sustainability, Measuring Corporate Environmental and Social Performance”, in M. Bennet, P. James & L. Klinkers (eds.), Sustainable Measures, Evaluation and Reporting of Environmental and Social Performance, Greenleaf Publishing, Sheffield 1999, pp. 475-495. To date there are probably more than 100 of these guidelines,⁶xTo get an impression of the guidelines presently available, see: <www.oesorichtlijnen/navigator>. which can be generic or sector specific. However, the multitude of guidelines does not make it any easier for an average entrepreneur or corporate manager to select the (most) suitable one. Besides, if we take a closer look at the generic guidelines, we will come across substantial differences in foci. This does not contribute to a better understanding and applicability of the corporate sustainability performance notion in the business community.

  Companies are no longer solely judged on their financial performance, but they also have to account for their sustainability performance to a variety of stakeholders. Along with the growing focus on corporate sustainability performance, the number of organizations assessing companies’ governance have increased, and so has social, ecological and economic performance. In this paper these organizations will be referred to as sustainability rating agencies or SRAs. To measure corporate sustainability performance SRAs use a sustainability assessment methodology. Usually, such a methodology comprises a screening instrument and an assessment procedure. A screening instrument measures or evaluates corporate sustainability performance, while an assessment procedure describes how the results of the screening instrument are processed and synthesized. For SRAs, screening instruments are the basis for rating.

  The aim of this article is to analyze and compare the contents of the screening instruments of two sustainability rating agencies: the Zürich (Switzerland) based Sustainable Asset Management Group (SAM) and the Boston (USA) based KLD⁷xAt the time this research was conducted, KLD Research & Analytics was still an independent SRA. As of November 2009 KLD Research & Analytics became part of the RiskMetricsGroup, which is now part of MSCI, Inc. In this paper we will continue to use the abbreviation KLD. Analytics, Inc. (KLD). Both are widely respected corporate sustainability performance research organizations. SAM and KLD have also launched and still support prominent and well-known sustainability indexes. In cooperation with Dow Jones & Company, SAM launched and still supports a whole range of indexes and index families, such as the Dow Jones Sustainability Index family. KLD launched ethical indexes, such as the Domini Social 400 Index.⁸xThis index is presently known as the MSCI KLD 400 Social Index Content analysis methodology is used to analyze the screening instruments of both SRAs.

  To my knowledge, this study is unique, because it is a first attempt to systematically analyze the contents of screening instruments of SRAs. We have seen studies⁹xMistra, “Values for Money”, 2004. Available at: <www.sustainability.com/publications/latest/values-for-money.pdf>; SustainAbility, Rate the Rater Phase One, Look Back and Current State. May 2010 (2010a). Available at: <www.sustainability.com/library/rate-the-raters-phase-one>; SustainAbility, Rate the Rater Phase Three, Uncovering Best Practices, February 2011 (2011a). Available at: <www.sustainability.com/library/rate-the-raters-phase-three>; SustainAbility, Rate the Rater Phase Four, The Necessary Future of Ratings, July 2011 (2011b). Available at: <www.sustainability.com/library/rate-the-raters-phase-four>. particularly focusing on a wide range of topics that are of concern to SRAs, however these studies do not report in detail about the contents of the screening instruments used, or the compatibility of these instruments among them.
  The results of this study suggest that the SAM and KLD instruments are imperfect measures of corporate sustainability performance, implying that the validity of these measures is questionable. The results also show that the screening instruments have limitations to their compatibility and cannot be used interchangeably because of differences in the underlying conceptions of corporate sustainability performance. So unfortunately this adds to the confusion surrounding corporate sustainability performance.

  One of the goals of this study is to contribute to the practise and theory of corporate sustainability performance measurement. This study contributes to the practise of corporate sustainability measurement, because it provides insight into the screening instruments of well known SRAs and the underlying corporate sustainability concepts. Getting more insight into these screening instruments is particularly important for companies which performance are rated and ranked by these SRAs. It is equally important for stakeholders, such as (ethical) investors, who use corporate sustainability information provided by them for investment purposes, or for reshuffling their portfolios.¹⁰xP.D. Kinder & A.L. Domini, “Social Screening: Paradigms Old and New”, 4 Journal of Investing 6, 1997, pp. 12-19.

  The theoretical contribution of this study lies in the efforts to gain a better understanding of corporate sustainability performance and measurement. In this study we use a research format designed and developed by Dommerholt¹¹xE. Dommerholt, Corporate Sustainability Performance: Constructs, Measures and Investors’ Responses, Doctoral dissertation, VU, September 2009. that is based on stakeholder theory: the Corporate Sustainability Analysis Framework (CSAF). This framework is based on the assumption that the extent to which companies’ contribute to sustainable development is the result of stakeholder pressures and expectations, and that these are codified in sustainability guidelines. The CSAF can also be used as a tool for assessing corporate sustainability performance or for evaluating the content of corporate documents, such as sustainability reports, annual reports and other forms of stakeholder communication. But the CSAF can also be used for corporate sustainability rating purposes.
  To date, a plenitude of corporate sustainability performance frameworks exist. All of these instruments commonly comprise of an array of topical areas. However, we do not know whether these reflect either stakeholder pressures or expectations. They could be considered as issues that companies are held responsible for. The CSAF comprises more than 200 of these issues that were derived from a set of 24 sustainability (reporting) guidelines.
  We also have very little insight into the relevance of corporate sustainability performance related issues. Although of great interest, this has not been a focal issue in the corporate sustainability performance arena. In the CSAF weights have been assigned to all 200+ sustainability items. These weights were derived from the set of 24 sustainability (reporting) guidelines.
  Hence, the CSAF not only provides insight into issues companies are considered responsible or accountable for, but it also provides insight into the relevance of these issues.
  Another goal of this study is to test whether the CSAF is a suitable framework for analyzing the screening instruments of SRAs.

  This article is organized as follows. In section 2 a literature review on corporate sustainability performance measurement will be presented. Section 3 is devoted to sustainability rating agencies, especially SAM and KLD. Section 4 focuses on validity and compatibility in relation to corporate sustainability performance measurement, and at the end of this section the research questions that are central to this study will be phrased. Subsequently, in sections 5 and 6 the data selection and data analysis method will be explained, and in section 7 the results and conclusions of the study will be presented. In the last section some implications of, and limitations to this study will be discussed.

• 2. Literature Review on Corporate Sustainability Performance Measurement

  Corporate sustainability performance measurement already has a longstanding tradition, but the level of attention for the subject soared in the early seventies of the previous century along with the increased attention for the link between the corporate sustainability performance-financial performance. These measures can be roughly divided into two categories: Uni- and multidimensional measures

  2.1 Uni- and Multi-Dimensional Measures

  A uni-dimensional measure of corporate sustainability performance measures consists of only one issue, implying that corporate sustainability performance can be captured into a single measure. We have seen quite some examples of these measures, such as pollution control,¹²xE.g. E.H. Bowman & M. Haire, “A Strategic Posture Toward Corporate Social Responsibility”, XVII California Management Review 2, 1975, pp. 49-58; H.R. Fogler & F. Nutt , “A Note on Social Responsibility and Stock Valuation”, 18 Academy of Management Journal 1, 1975, pp. 155-160; B.H. Spicer, “Investors, Corporate Social Performance and Information Disclosure: An Empirical Study”, 53 The Accounting Review 1, 1978, pp. 94-111. hazardous waste disposal,¹³xE.g. J. Rockness, P. Schlachter & H.O. Rockness, “Hazardous Waste Disposal, Corporate Disclosure, and Financial Performance in the Chemical Industry”, 1 Advances in Public Interest Accounting 1986, pp. 167-191. product recalls,¹⁴xG. Jarrell & S. Pelzman, “The Impact of Product Recalls on the Wealth of Sellers”, 93 Journal of Political Economy 3, 1985, pp. 512-536; S.W. Pruitt & D.R. Peterson, “Security Price Reactions Around Product Recall Announcements”, IX The Journal of Financial Research 2, 1986, pp. 113-122; P. Bromiley & A. Marcus, “The Deterrent to Dubious Corporate Behaviour: Profitability and Safety Recalls”, 10 Strategic Management Journal 1986, pp. 233-250; W.N. Davidson & D.N.Worrell, “Research Notes and Communications: The Effect of Product Recall Announcements on Shareholder Wealth”, 3 Strategic Management Journal 1992, pp. 467-473. illicit behaviour,¹⁵xW.N. Davidson III & D.L. Worrell, “The Impact of Announcements of Corporate Illegalities on Shareholder Returns”, 31 Academy of Management Journal 1, 1988, pp. 195-200; J. Frooman, “Socially Irresponsible and Illegal Behavior and Shareholder Wealth, A Meta Analysis of Event Studies”, 36 Business & Society 3, 1997, pp. 221-249. signing of the Sullivan Principles,¹⁶xD.M. Patten, “The Market Reaction to Social Responsibility Disclosures: The Case of the Sullivan Principles Signings”, 15 Accounting, Organizations and Society 6, 1990, pp. 575-587; McMillan, “Corporate Social Investments: Do they Pay?”, 15 Journal of Business Ethics 1996, pp. 309-314. winning an award,¹⁷xR.D. Klassen & C.P. McLaughlin, “The Impact of Environmental Management on Firm Performance”, 42 Management Science 8, 1996, pp. 1199-1214; K.B. Hendricks & V.R. Singhal, “Quality Awards and the Market Value of the Firm: An Empirical Investigation”, 42 Management Science 3, 1996, pp. 415-436. environmental management system improvements,¹⁸xS.J. Feldman, P.A. Soyka & P.G. Ameer, “Does Improving a Firm's Environmental Management System and Environmental Performance Result in a Higher Stock Price?”, 6 Journal of Investing 4, 1997, pp. 87-97. a corporate code of ethics;¹⁹xS. Webley & E. More, Does Business Ethics Pay?, Institute of Business Ethics, London 2003. addition to a sustainability index.²⁰xL. Becchetti, R. Ciciretti & I. Hasan, Corporate Social Responsibility and Shareholder Value: An Event-study, Federal Reserve Bank of Atlanta, Working Paper 2007-6, April 2007. Available at: <www.frbatlanta.org/filelegacydocs/wp0607.pdf>; Dommerholt 2009.

  Corporate sustainability performance is commonly believed to be a multi-dimensional construct and complex phenomenon.²¹xA.B. Carroll, “A Commentary and an Overview of Key Questions on Corporate Social Performance Measurement”, 39 Business and Society 4, 2002, pp. 466-478; T. Rowley & S. Berman, “A Brand New Brand of Corporate Social Performance”, 39 Business & Society 4, 2000, pp. 397-418. Using a uni-dimensional measure to capture or reflect a multi-dimensional phenomenon is likely to cause validity problems.²²xJ. Griffin & J. Mahon, “The Corporate Social Performance and Corporate Financial Performance Debate, Twenty-Five Years of Incomparable Research”, 36 Business & Society 1, 1997, pp. 5-31.
  Multi-dimensional measures of corporate sustainability performance comprise a number of sustainability related issues that are indicative of a company’s performance.²³xS. Zyglidopoulos, “The Impact of Accidents on Firms’ Reputation for Social Performance”, 40 Business & Society 4, 2001, pp. 416-441. To date a plethora of such measures exist. Three types of these measures will now be explained.

  2.2 Reputational Scales

  Generally speaking, reputational scales gauge perceptions of corporate sustainable performance as reported by relevant stakeholder groups. However, perceptions of corporate sustainability performance not necessarily measure corporate sustainability performance.
  One of the first scholars to use reputational measures was Milton Moskowitz, one of the first scholars to study the corporate sustainability performance – corporate financial performance link. He created samples of companies based on – according to his opinion – good social responsibility credentials.²⁴xG.J. Alexander & R.A. Buchholz, “Corporate Social Responsibility and Stock Market Performance”, 21 Academy of Management Journal 3, 1978, pp. 479-486.
  A very well-known example of a reputational scale is the Fortune survey. These measures comprise eight attributes: Overall quality of management; Quality of products and services; Financial soundness; Value as a long-term investment; Use of corporate assets; Innovativeness; Ability to attract, develop and keep talented people; Community or environmental responsibility.²⁵xR. Wokutch & B. Spencer, “Corporate Saints and Sinners: The Effect of Philanthropic and Illegal Activity on Organizational Performance”, 29 California Management Review 2, 1987, pp. 63-77; S.A. Waddock & S.B. Graves, “The Corporate Social Performance-Financial Performance Link”, 18 Strategic Management Journal 4, 1997, pp. 303-319; Zyglidopoulos 2001; P. Dev Sinha & T. Salas, The Relationship Between Corporate Social Responsibility and Profitability of Hospitality Firms: Do Firms that Do Good Also Do Well?, School of Hotel Administration, Cornell University, Ithaca NY 2002.
  Reputational scales have been criticized for lack of sufficient theoretical underpinning in the selection of attributes. Furthermore, the selected attributes are (implicitly) considered equally important as reputational scales although a sound underpinning for doing so does not exist.²⁶xD.J. Wood & R.E. Jones, “Stakeholder Mismatching: A Theoretical Problem in Empirical Research on Corporate Social Performance”, 3 The International Journal of Organizational Analysis 3, 1995, pp. 229-267.

  2.3 Content Analysis

  Content analysis can be described as “a technique for gathering data that consists of codifying qualitative information in anecdotal and literary form into categories in order to derive quantitative scales of varying levels of complexity”.²⁷xW.F. Abbott & R.J. Monsen, “On the Measurement of Corporate Social Responsibility: Self Reported Disclosures as a Method of Measuring Corporate Social Involvement”, 22 Academy of Management Journal 3, 1979, pp. 501-515. Content analysis is a generally accepted methodology for evaluating corporate sustainability performance.²⁸xP.L. Cochran & R.A. Wood, “Corporate Social Responsibility and Financial Performance’, 27 Academy of Management Journal 1, 1984, pp. 42-56. This methodology is mainly, although not exclusively, used for analyzing social and environmental disclosures in annual reports. Classification schemes used for analyzing these reports vary widely in terms of range and attributes, as shows the following overview: Prose in annual reports devoted to social responsibility,²⁹xBowman & Haire 1975. Social responsibility disclosures in relation to five categories: environment; fair business; personnel; community; product,³⁰xR.W. Ingram, “An Investigation of the Information Content of (Certain) Social Responsibility Disclosures”, 16 Journal of Accounting Research 2, 1978, pp. 270-285. Environmental contingencies; Environmental expenditures and investments; “Pollution abatement”; Environmental preservation; “Other environmentally related information”,³¹xD.W. Walden & B.N. Schwartz , “Environmental Disclosures and Public Policy Pressures”, 16 Journal of Accounting and Public Policy 1997, pp. 125-154. Environmental disclosures and other social disclosures;³²xD. Neu, H. Warsame & K. Pedwell, “Managing Public Impressions: Environmental Disclosures in Annual Reports”, 23 Accounting, Organizations and Society 3, 1998, pp. 265-282. Environment; Energy; Fair business practice; Human resources; Community involvement; Products; Other disclosures.³³xE.g., D.M. Patten, “Variability in Social Disclosure: A Legitimacy Based Analysis”, 6 Advances in Public Interest Accounting 1995, pp. 273-285.

  The validity of content analysis depends on the classification scheme adopted by the researcher. The more questionable the scheme is, the more criticism the research can be expected to receive.³⁴xM. Sharfman, “The Construct Validity of the Kinder, Lydenberg & Domini Social Performance Ratings Data”, 5 Journal of Business Ethics 1996, pp. 287-296. However, we do not know if, and the extent to which, the schemes that have been used for content analysis of annual reports and other documents, capture the full breadth of the complex and multi-dimensional corporate sustainability performance construct.

• 3. Sustainability Rating Agencies

  Rating the corporate sustainability performance of companies already has a longstanding tradition. Usually, authors mark the 17th century as the starting point of corporate sustainability performance. The reason for this is that during that time, the Quakers adopted the thought that the slave trade could not be reconciled with their belief in God.³⁵xKinder & Domini 1997. As a consequence they refused to do business with or invest in companies that were somehow connected with these practices.
  In the mid 1920s Methodists and Baptists rejected the use of alcohol and tobacco.³⁶xIbid. In the decades that followed, alcohol and tobacco screens were supplemented with nuclear power and gambling screens, thus voicing the criticism of Christian and secular groups on these practices.³⁷xIbid. These screens are usually referred to as exclusionary – or negative – screens. That is, companies that produce products and services that are criticized by societal groups, or that have business relationships with such companies, are excluded from investment portfolios.
  In the 1970s and 1980s so-called qualitative (or positive) screens, comprising a range of indicators for assessing companies’ social and environmental performance, were developed. These indicators enable investors to create investment portfolios specifically reflecting their values, and allow for longitudinal evaluations. These screens also facilitate benchmarking of companies against peer companies and industries, thus enabling the selection of ‘best-in-class’ performers.

  To date companies are no longer solely judged on their financial performance. They also have to account for their sustainability performance to a variety of stakeholders. Assessing companies’ sustainability performance is the core activity of Sustainability Rating Agencies (SRAs). These agencies are an important link between companies and their stakeholders, especially financially orientated ones.³⁸xH. Schäfer, “International Corporate Social Responsibility Rating Systems”, 20 Journal of Corporate Citizenship 2005, pp. 107-120.
  SRAs come in all sorts, depending on their target audience, geographic orientation and/or issue focus. Some SRAs focus on specific stakeholder groups, such as consumers, investors, companies or the general public. However, we witness vast differences among these SRAs in terms of issue coverage. Some SRAs assess a company’s sustainability performance on the basis of a combination of social (people), environmental (planet) and economic criteria (profit). Other SRAs take only one “leg” of this so-called “triple P” approach into account. Again, others narrow corporate sustainability performance down to only one issue or item, such as CO₂ emissions or human rights.

  Accurate estimates of the number of SRAs presently active in the corporate sustainability performance arena are not available. In a compendium, Van den Brink describes 29 of these agencies, whereas a study conducted by Schäfer³⁹xSchäfer 2005. includes 40 of these SRAs. In the “Rate the Raters” study, conducted by Sustainability, we find that the number of raters presently active in the field is well above 100, whereby the number of SRAs quintupled between 2000 to 2010.⁴⁰xSustainability, 2011b But we have also seen mergers of SRAs, and SRAs having been taken over by others. An example of the dynamic in the rating industry is the purchase of two prominent players in the sustainability arena (KLD and Innovest) by RiskMetrics, which in turn was acquired by MSCI.

  Not all firms in the sustainability rating, research and screening business were specifically founded for this purpose. In some cases these activities were mere by-products of initial activities. For example: banks, insurers and asset management firms created their own sustainability departments.⁴¹xSchäfer 2005. Other organizations, presently active as SRAs were already actively researching corporate sustainability performance as non-governmental organization (NGOs). Examples of former NGOs that became SRAs are Ethibel (Belgium), Co-op America (USA) and the Ethical Consumer Association (UK). On the other hand, Oekom Research, an SRA based in Germany, has its roots in circles of critical journalists.⁴²xIbid.
  Some SRAs, such as the Sustainable Asset Management Group (SAM) and USA based Innovest, were established specifically for assessing corporate sustainability.⁴³xSchäfer 2005; T. van den Brink, Guide Screening and Rating Sustainability, Triple P Performance Centre, Vrije Universiteit Amsterdam 2002. Other firms, such as the not for profit UK-based Ethical Investment Research Service (EIRIS), were founded or supported by churches and charities organizations.⁴⁴xSchäfer 2005.

  Screening instruments and rating methodologies of SRAs are proprietary, because these instruments and methodologies are seen to be ‘trademarks’ of SRAs that determine their (competitive) position in the corporate sustainability rating arena. Therefore rating agencies are quite reluctant to provide in-depth information on their rating schemes and assessment methodologies.⁴⁵xSustainability, 2011a On the basis of publicly available information it is almost impossible to get clear view on what these screens and methodologies represent. However, from the (limited) information that is available we know that the foci of screening methods differ substantially. A generally accepted standard or benchmark for evaluating the quality of screening instruments does not exist and screening instruments usually also lack a sound theoretical foundation.⁴⁶xVan den Brink 2002.
  Although most screening methodologies are proprietary, the outline of the rating procedures of SRAs are largely similar.⁴⁷xWBCSD, World Business Council for Sustainable Development. Sustainable Development Reporting: Striking the Balance, Geneva, World Business Council for Sustainable Development 2003; Van den Brink 2002 In this context, the term “proprietary” relates to the fact that most SRAs have developed screening methodologies that have become their trademark and raison d’être. Sometimes rating schemes are made public, but how the results of the screening phase are processed is often shrouded in mysteriousness.
  Usually rating procedures comprise three phases. In the first phase, SRAs collect the required data using a variety of techniques (questionnaires, interviews, company information, published material in newspapers and magazines, company visits etc.). The second phase consists of data verification and analysis, while in the last phase the ranking or rating of companies takes place.⁴⁸xVan den Brink 2002.
  Little is known about the SRAs’ verification and analysis phase. “Many companies perceive this phase as a black box in which the weighting of the issues relative to one another takes place.”⁴⁹xS. de Hoo & H. King, “Company Sustainability Reporting and Sustainability Rating”, in van den Brink 2002.

  In the next section two SRAs come up for discussion: KLD Analytics Inc. (KLD) and the Sustainable Asset Management Group (SAM). As mentioned earlier, to date a multitude of SRAs have entered the corporate sustainability scene. As may be expected, KLD and SAM are not selected randomly. Both are very respected SRAs, and have dominated the corporate sustainability performance rating scene for many years. Both launched and still support very well-known and prominent sustainability indexes.⁵⁰xAnother, more practical reason for selecting these SRAs is they were the only SRAs that positively responded to my request to provide additional information about the (composition of) sustainability indexes they launched and still support: the Dow Jones Sustainability World Index family (SAM) and the Domini Social 400 Index (KLD).

  3.1 KLD Analytics Inc.⁵¹xSee note 3. (KLD)

  The KLD research methodology lies at the basis of various products and indexes, such as the SOCRATES-database and the Domini Social 400 Index (DSI).⁵²xSee note 4. The methodology includes a research and rating approach. The rating approach involves sixteen dimensions, which are divided into two groups: Social Issues and Controversial Business Issues. The following social issues are distinguished: Community, Corporate Governance, Diversity, Employee Relations, Environment, Human Rights and Product. Strengths and concerns have been defined for all social issues. Controversial Business Issues comprise the following items: Adult Entertainment, Firearms, Military Weapons, Tobacco, Alcohol, Gambling, Nuclear Power and Abortion. As is the case with many screening instruments, KLD does not provide a theoretical underpinning for the selection of issues and attributes that constitute the screen.⁵³xSharfman 1996; S.B. Graves & S.A. Waddock, “A Look at the Financial-Social Performance Nexus when Quality of Management is Held Constant”, 12 International Journal of Value-Based Management 1999, pp. 87-99.

  Once all the required information is gathered, KLD rates companies on a scale ranging from “major concern” to “major strength” with a middle neutral rating⁵⁴xS.A. Waddock & S.B. Graves, “The Corporate Social Performance-Financial Performance Link”, 18 Strategic Management Journal 4, 1997, pp. 303-319. using a proprietary framework of positive and negative indicators. Controversial Business Issues are rated on a scale of “absence of controversy” to “major controversy”.⁵⁵xB. Ruf, K. Muralidhar, R. Brown, J. Janney & K. Paul, “An Empirical Investigation of the Relationship Between Change in Corporate Social Performance and Financial Performance: A Stakeholder Theory Perspective”, 32 Journal of Business Ethics 2001, pp. 143-156.
  Since the rating method is proprietary, the weighting process and the weights assigned to the various areas and attributes are not disclosed. Also, how flows of information converge into a rating is not disclosed. Ruf et al. provide some marginal insight into the evaluation process. They state that:

  “To enhance consistency in the evaluations, a research staff member evaluates each company using pre-specified criteria. Unclear judgements are discussed and made by a research team. Furthermore evaluations are conducted at the same time each year for companies within an industry. This improves consistency of intra-industry assessments and over-time assessments.”⁵⁶xRuf et al. 2001, p. 149

  Furthermore, corporate sustainability performance information provided by KLD has been and still is being used by many scholars for scientific research purposes and is considered an excellent source of information.⁵⁷xJ. Harrison & R. Freeman, “Stakeholders, Social Responsibility and Performance: Empirical Evidence and Theoretical Perspectives”, 42 Academy of Management Journal 5, 1999, pp. 479-485.

  3.2 Sustainable Asset Management Group (SAM)

  The SAM research methodology lies at the heart of a number of products and indexes, such as the Dow Jones Sustainability World Index family. The starting point of the methodology is this definition of a set of sustainability criteria. SAM distinguishes between industry-specific and general criteria. According to SAM, these criteria are based on widely accepted standards, best practices and audit procedures, as well as industry specialists and consultants. However, SAM does not disclose precise information as to which standards, practices, procedures, etc. are being used. In the overall assessment, general criteria, which apply to all industries, are assigned a weight of 60%, while the remaining 40% are allocated to industry specific criteria. The latter cover issues that reflect the economic, environmental, social, political, and technological forces that drive sustainability performance of that specific industry group.
  Except for the pharmaceutical industry, detailed industry-specific criteria are not publicly disclosed. SAM distinguishes some sixty industries for which industry-specific criteria have been developed.
  The general criteria comprise the following economic, environmental and social issues: Economic (Corporate Governance; Risk & Crisis Management; Codes of Conduct/Compliance/Corruption & Bribery; Industry Specific Criteria); Environment (Environmental Reporting, Industry Specific Criteria); Social (Human Capital Development, Talent Attraction & Retention, Labor Practice Indicators, Corporate Citizenship and Philanthropy, Social Reporting, Industry Specific).

  The rating procedure followed by SAM is highly untransparent and consists of a complex system of issue scores (e.g., – adjusted – answer scores; (adjusted) question scores; question verification score) and weights (e.g., criteria weight; class weight; question weight; weight of quality/public availability) that ultimately result in an overall company score. Dommerholt tried to unravel the rating procedure on the basis of publicly available information.⁵⁸xDommerholt 2009. How successful his efforts may have been is something we will never know, because the results cannot be verified.

• 4. Validity and Compatibility of Screening Instruments

  In this section the validity and compatibility concepts will be discussed. The concept of validity is dealt with in most research handbooks and it refers to whether a measure measures what it is supposed to. In this study the validity concept specifically refers to the validity of screening instruments. Comparability, on the other hand, is a new concept and this refers to the exchangeability of screening instruments.

  4.1 Validity

  A screening instrument is called valid when it measures what it is supposed to,⁵⁹xP. Brownell , Research Methods in Management Accounting, Coopers & Lybrand Accounting Research Methodology, Monograph no. 2, Coopers & Lybrand Australia 1995. or as Chatterji and Levine put it: validity is about whether a measure identifies performance that is important to society.⁶⁰xA. Chatterji & D. Levine, “Breaking Down the Wall of Codes: Evaluating Non-Financial Performance Measurement”, 48 California Management Review 2, 2006, pp. 29-51. In the context of this paper we call a screening instrument a valid measure of corporate sustainability performance if it measures just that.
  To date a multitude of measures have been developed to tap into the corporate sustainability performance construct. Since corporate sustainability performance is considered a multifaceted construct that cannot simply be captured by a uni-dimensional of single issue measure, this implies that uni-dimensional measures challenge the very concept of validity.
  Multi-dimensional corporate sustainability performance measures, on the other hand, usually capture a variety of sustainability related issues. However, we do not know if these gauges measure what they are supposed to measure. In other words, we do not know whether these measures are valid measures of corporate sustainability performance.
  If we would like to assess the validity of a measure, we need a – preferably generally accepted – framework against which these measures can be benchmarked. Unfortunately, such a framework does not yet exist. But even if such a framework did exist, then determining the validity of a corporate sustainability performance measure is not at all an easy task, because “the assessment of validity is not a binary phenomenon. It is not a situation where a measure is either valid or it’s not. Rather, since validity is a continuous phenomenon, we can only say the degree to which a measure is valid”. This statement made by Sharfman very well describes the challenges we are facing when assessing the validity of composite measures such as the SAM and KLD screening instruments. Following Sharfman, we can only give an estimation of the extent to which these instruments are considered valid.

  Some have reasons to believe that the KLD screening instrument is a valid measure of corporate sustainability performance.⁶¹xSharfman 1996. He benchmarked the KLD screening instrument against a selection of other bases for rating (e.g., the fortune reputation score) and corporate sustainability performance indexes. A major point of concern in his research methodology is that he believes that the selected bases for rating sufficiently or perhaps even comprehensively represent or reflect corporate sustainability performance. To my knowledge, there is no evidence that supports this idea. We do know, however, that the KLD screening instrument lacks a theoretical basis, although Ruf et al. state that the KLD screening instrument reflects the concerns historically held by social investors and include those identified as important in surveys of social fund managers.

  Chatterji and Levine express some serious doubts about the validity of the SAM screening instrument, because they suggest that this instrument may not capture the kind of information desired by stakeholders, or that information is not provided in a manner desired by stakeholders. Interestingly, they implicitly suggest that the validity concept concerning measures of corporate sustainability performance should de rooted in stakeholder theory. This implies that if we want to capture the full breadth of what corporate sustainability performance might look like, we should take stakeholder desires, demands and expectations as the starting point.
  SAM claims that the screening instrument it uses for assessing corporate sustainability performance is based on widely accepted standards, best practices and audit procedures as well as input from industry specialists and consultants. However, SAM does not give further information regarding these standards, practices, etc., to back their screening instrument.

  4.2 Compatibility

  In the context of this paper, screening instruments are considered compatible if these instruments can be used interchangeably. Chatterji and Levine⁶²xA.K. Chatterji & D.I.I. Levine, Imitate or differentiate? Evaluating the Validity of Corporate Social Responsibility Ratings, Working Paper Series, University of California, Berkely 2008. Available at: <http://repositories.cdlib.org/crb/wps/37>. speak of convergent validity, that is whether social ratings agree with one another, after adjustments have been made for purposeful differences. They find major social ratings to have a fairly low correlation with each other, indicating that convergent validity, or compatibility, is low and that as a consequence these ratings cannot simply be used interchangeably.
  This conclusion is supported by findings of De Hoo and King. They found an overlap of only 15% when comparing the questionnaires of more than five SRAs.⁶³xDe Hoo & King 2002; WBCSD 2003. However, we do not have a detailed insight into the differences between screening instruments.

  Just as is the case with validity, assessing compatibility is not a binary phenomenon. Compatibility comes in degrees. If two screening instruments are for 99.9% identical, should we then reject the idea of compatibility of these instruments? This implies that we can only give an estimation of the extent to which screening instruments are compatible.
  Besides, validity (as discussed in the previous paragraph) and compatibility are to a certain extent related notions. Compatibility of two (or more) highly valid screening instruments is likely to be high, whereas compatibility of highly and poorly valid instruments is likely to be poor. On the other hand, two screening instruments that poorly reflect corporate sustainability performance may nevertheless be highly compatible.

  Since this article takes the KLD screening instruments as a starting point, we want to find an answer to the following questions:

  1. To what extent can the SAM and KLD screening instruments be called valid measures of corporate sustainability performance?

  2. To what extent are the SAM and KLD screening instruments compatible?

  Figure 1SAM and KLD Screening Instruments

  Documents	Analyzed sections	Available at:
  SAM Research Corporate Sustainability Assessment Questionnaire 2006 (without industry specific criteria)	All questions on pages 4-63, the following question excluded: 11; 22; 28; 37; 43; 58; 63; 66 and 71.64xThese questions are excluded from the analyzed sections because they focus on the verification of a performance score by the responsible industry analyst.	<www.sustainability-indexes.com>
  KLD Research, Social Ratings Criteria, SOCRATES Company Profiles 2006	The following pages were analyzed: 4-11.	<www.kld.com>

• 5. Data Selection

  In figure 1 the SAM and KLD screening instruments, the analyzed sections of these documents as well as internet locations from which these documents can be downloaded are presented. The SAM Research Corporate Sustainability Assessment Questionnaire 2006 (without industry specific criteria) comprises 71 questions concerning economic, environmental and social issues that were derived from general (i.e. non-industry specific) corporate sustainability assessment criteria. For reasons of completeness and comprehensiveness, it would have been better to also include industry specific criteria in the data set. However, SAM has publicly disclosed only the supplement of the pharmaceutical industry and it is uncertain if an analysis of this document can be generalized with regard to other industries.

  The KLD Research, Social Ratings Criteria, SOCRATES Company Profiles, 2006, describes KLD’s social ratings criteria and controversial business issues ratings. Because our focus is on contribution of businesses to sustainable development, we will only take into account positive screens, which are aimed at selecting best-in-class performers. This means that for the KLD instrument, only the part describing the social issue ratings will be analyzed.
  Controversial business issues are commonly regarded as related to a company’s ethical performance. Some audiences consider the production of tobacco and alcohol unethical, because of the negative public health effects of these commodities. However, tobacco and alcohol producing companies not necessarily perform poorly on governance, social, environmental and economic criteria. In other words, we ought to make a distinction between corporate sustainability performance and ethical performance.

• 6. Data Analysis Method

  6.1 Content Analysis

  The SAM and KLD instruments will be analyzed using the content analysis methodology. Content analysis usually involves two activities: (1) the construction of a classification scheme and (2) devising a set of rules about “what” and “how” to code.⁶⁵xJ.M. Milne & R.W. Adler, “Exploring the Reliability of Social and Environmental Disclosures in Content Analysis”, 12 Accounting, Auditing & Accountability Journal 2, 1999, pp. 237-256. In its simplest form, content analysis uses a dichotomous classification, which means that to each category a score of zero or one is assigned, indicating the presence or absence of an information category. However, such a dichotomous classification does not take the extensiveness of the presented information into account and is therefore a relatively poor indicator of information content.⁶⁶xS. Parsa & R. Kouhy, Disclosure of Information By UK Companies – A Case of Legitimacy Theory, London, Middlesex University Business School, 2001. Available at: <www.mubs.mdx.ac.uk/research/Discussion_Papers/Accounting_and_Finance>.
  The key assumption underlying the content-analysis methodology is that by analyzing texts researchers get to understand other people’s cognitive schemes.⁶⁷xV.J. Duriau, R.K. Reger & M.D. Pfarrer, “A Content Analysis of the Content Analysis Literature in Organization Studies”, 10 Organizational Research Methods 1, 2007, pp. 5-34.
  To be able to properly and accurately apply the content-analysis methodology, a classification scheme and a coding procedure are required. In the following section, the classification scheme and coding procedure used in the present study will be explained.
  When applying the content-analysis methodology, (a combination of) words, sentences or pages is likely to be the unit of analysis. However, a distinction should be made between the basis for coding and the basis for measuring; these two do not necessarily coincide. Sentences may be used as the basis for coding, while words are counted to measure the extensiveness of the information disclosed.⁶⁸xMilne & Adler 1999. Or as Duriau et al.⁶⁹xDuriau et al. 2007, p. 6. state “at its most basic, word frequency has been considered to be an indicator of cognitive centrality (..) or importance”.

  6.2 Classification Scheme

  As we have seen, both the SAM and KLD instruments are multi-dimensional measures of corporate sustainability performance. However, since these are different measures of corporate sustainability performance, they also may measure corporate sustainability differently. If so, then obviously these measures are built on different perceptions of corporate sustainability performance.
  In order to get an idea of the corporate sustainability performance notions underlying these instruments and to get a clearer view on what these instruments actually measure, a benchmark model is required. However, a universally accepted multi-dimensional framework for measuring corporate sustainability performance framework does not yet exist.
  In this study the Corporate Sustainability Analysis Framework (CSAF) developed by Dommerholt⁷⁰xDommerholt 2009. will be used as a classification scheme or benchmark against which the SAM and KLD instruments will be evaluated. This framework is rooted in stakeholder theory and rests on the assumption that corporate sustainability performance is based on stakeholder expectations and demands, since companies typically are responsible for meeting the interests and expectations of their stakeholders.⁷¹xM.B.E. Clarkso, “A Stakeholder Framework for Analyzing and Evaluating Corporate Social Performance”, 20 Academy of Management Review 1, 1995, pp. 92-117; P. Bansal, “The Corporate Challenges of Sustainable Development”, 16 Academy of Management Executive 2, 2002, pp. 122-131; R.K. Mitchell, B.R. Agle & D.J. Wood, “Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts”, 22 Academy of Management Review 4, 1997, pp. 853-886; C. Oliver, “Strategic Responses to Institutional Processes”, 16 Academy of Management Review 1, 1991, pp. 145-179. This is also in line with suggestions of Chatterji and Levine, who – as noted earlier – believe that the validity of corporate sustainability performance screening instrument is associated with desires of stakeholders.
  Depending on the goals and interests of stakeholders, expectations, demands or desires may vary among them. It is obvious that demands of human rights groups are likely to differ substantially from expectations of environmentalists or investors. However, by bringing all these expectations and desires together, we may get a pretty good idea of the responsibilities the corporate community has to live up to. These responsibilities are tied to issues, or “topical areas” as Carroll⁷²xA.B. Carroll, “A Three-Dimensional Conceptual Model of Corporate Performance”, 4 Academy of Management 4, 1979, pp. 497-505. calls them.
  In fact, the discussion about corporate sustainability performance bears great resemblance with the fable of the six blind men and the elephant⁷³xThe fable is adopted from: <www.jainworld.com/literature/story25.htm>.. This old Indian fable is about six blind villagers who were told that an elephant had come to visit their village. These men were quite anxious, as they had no idea what an elephant was. They were guided to the animal and they all started to touch the creature. The first man, who touched the leg thought that an elephant was something like a pillar. The second man touched the tail and said that an elephant “looks” like a rope. The third person touched the elephant’s trunk and believed an elephant to be some kind of a thick tree branch. After having touched the ear, the fourth man thought an elephant was like a big hand fan. The man who touched the belly, thought an elephant might be something like a huge wall. Finally, the sixth man who touched the elephants tusk, believed an elephant to resemble a solid pipe.
  All of them began to argue about the creature they just touched and everyone insisted that he was right. It looked like they were getting agitated with each other. A wise man who passed by asked the six men what the discussion was all about. They told him they could not agree to what an elephant is like. The six men told the wise man what they had just experienced. Then the wise man explained them that all of them were right. The reason why their stories differed was that the six blind men touched different parts of the elephant. What they did not know was that the features they mentioned to the wise man belonged to one and the same creature.
  The parallel between this fable and the corporate sustainability performance debate is that in this debate also a variety of stakeholders also believe or claim to know what corporate sustainability performance looks like, or what the features of corporate sustainability performance are or should be. However, what they may not know is that their perception of corporate sustainability performance is just a part, or a feature, of a major jig-saw puzzle. By putting the parts of the puzzle together, an image of what corporate sustainability performance might look like emerges. At first the image may look rather dim and vague, but the more pieces of the puzzle are connected, the clearer the image becomes. There is only one problem and that is that we do not know the ultimate size of the puzzle, or how many pieces are needed to complete the puzzle.
  As time passes by, some of the pieces of the puzzle may fade and need to be replaced. Other pieces turn out to be far more color-proof and need not be replaced by new or brighter ones, thus giving expression to the idea that corporate sustainability performance is a time-bound construct, or a moving target.⁷⁴xS.P. Sethi, “Dimensions of Corporate Social Performance: An Analytical Framework”, California Management Review 1975, pp. 58-64.

  But how to start, and where to find the parts of the corporate sustainability performance jig-saw puzzle? Or in other words, where to find sustainable development related items that reflect stakeholder expectations and pressures, or items that companies are considered responsible or accountable for?
  These items are “locked in” in so-called sustainability (reporting) guidelines, which capture expectations of a plenitude of stakeholders concerning business’ contribution to sustainable development. The CSAF is based on a set of 24 of such sustainability (reporting) guidelines, an overview of which is presented in Appendix 1. An overview of topical areas that have been derived from these guidelines is included in Appendix 2. This Appendix shows that the CSAF comprises four aggregation levels: Sustainability Items, Issues, Aspects and Dimensions. In parenthesis the weights assigned to these Sustainability Items, Issues, Aspects and Dimensions are mentioned. These weights are a reflection of the relative importance of an Item, Issue, Aspect or Dimension. The higher the assigned weight, the higher the relative importance. The weight distribution of the CSAF was tested and it was found to be robust.

  • Sustainability Items represent the lowest aggregation level. The CSAF comprises more than two hundred of these items. The single most important sustainability item is the ‘Non-discrimination’ item with a weight of more than 6.5%.

  • The second aggregation level contains Sustainability Issues. Issues are comprised of Sustainability Items sharing common characteristics. Of all Issues, with a weight of 11.1%. The Human Rights Issue has the highest weight.

  • The third aggregation level are referred to as to Sustainability Aspects. These are clusters of related Sustainability Issues. “Employees” is by far the largest Aspect with a weight of almost 27%.

  • The fourth and highest aggregation level is the Sustainability Dimension, which contains clusters of related Sustainability Aspects. With a weight of more than 54% the Social Dimension is clearly dominating.

  When combining the desires and expectations of a plenitude of stakeholders, we can say that socially oriented issues are prevalent and determine the color of the corporate sustainability jig-saw puzzle. Interestingly, environmentally oriented issues are not as prevalent as one might expect in the light of ongoing global warming discussion. Economic issues hardly color the corporate sustainability performance landscape.

  However, corporate sustainability performance is not only about topical areas. It is also about performance. But what do we actually mean by performance in this context? This question is somewhat difficult to answer since in the business-society literature, the term performance is not unambiguously defined. Following Mitnick⁷⁵xB.M. Mitnick, “Commitment, Revelation, and the Testament of Belief: The Metrics of Measurement of Corporate Social Performance”, 39 Business & Society 4, 2000, pp. 419-465. and Steg et al.⁷⁶xL. Steg, C.H. Vlek, S. Lindenberg, T. Groot, H. Moll, T. Schoot-Uiterkamp & A. van Witteloostuijn, Towards a Comprehensive Model of Sustainable Corporate Performance. Three dimensional modelling and practical measurement (Second Interim Report), Groningen, Departments of Economics, Environmental Sciences, Management Science, Psychology and Sociology, University of Groningen, Vrije Universiteit Amsterdam, University College Windesheim 2003. the performance dimension of the CSAF comprises four so-called performance domains: Principles, Practices, Outcomes and Stakeholder perceptions. Principles are at the basis of practices, which in turn result in intended outcomes. Moreover, stakeholders should be engaged in shaping and evaluating these steps. If performance domains are linked, doing good works (practices/outcomes) for bad reasons (principles) can be distinguished from good works (practices/outcomes) for good reasons (principles).

  6.3 Rules about “What” and “How” to the Code

  Sustainability Items (the lowest aggregation level) are used as the basis for coding, while performance measures in combination with Sustainability Items are used as the basis for measuring. Each time a Sustainability Item is mentioned, a score of 1 is assigned. If the Sustainability Item is accompanied by one or more performance domain measures, then for each of these measure an additional score of 1 is assigned.
  The assumption underlying this measurement method is that the more often a Sustainability Item is addressed and accompanied by performance domain measures, the more important that particular Sustainability Item is considered to be. Hence, the higher the score assigned to a Sustainability Item, Issue, Aspect or Dimension, the more elaborately that Item, Issue, Aspect of Dimension is addressed in the selection of (reporting) guidelines.

• 7. Results

  The results of the content-analysis of the SAM and KLD instruments are presented in figure 2 and are for reasons of clarity provided only for the Dimension and Aspect levels, the highest aggregation levels, of the CSAF. Figure 2 provides insight into the (accentuation of) topical areas covered by the SAM and KLD screening instruments.
  The first column covers the Dimension and Aspect names. In the second and third columns so-called representation coefficients for the SAM and KLD instruments are presented. These coefficients provide information about the relative over- or underrepresentation of a Dimension or Aspect, and are calculated by dividing the SAM and KLD Dimension and Aspect weights by the CSAF Dimension and Aspect weights. If a representation coefficient is greater/lower than 1, the concerning Dimension or Aspect is considered over/under represented.
  The third and fourth column of figure 2 cover the weights that have been assigned to the Dimensions and Aspects. The higher the weight, the more attention is paid to a specific Dimension or Aspect.
  These weights have been computed by dividing the respective Aspect and Dimensions scores by the instrument scores. In the last column the weight distribution of the CSAF is presented.
  Furthermore, the last row contains the number Sustainability Items that are addressed in both instruments. The last but one row captures the total or instrument scores of both screening instruments.

  

  Figure 2 Accentuation of Topical Areas in SAM and KLD Screening Instrument

  As can be seen the SAM instrument covers 85 (or 38%) of the maximum 222 Sustainability Items of the CSAF. With weights of 36 and 5% the Governance and Economic Dimensions are clearly overrepresented when compared with the CSAF. On the other hand the Environment Dimension is severely underrepresented with a weight of little over 5%. The reason for this is that in this study SAM’s generic Sustainability Assessment Questionnaire was analyzed. SAM typically covers environment related issues in industry specific supplements.
  At the Aspect level, the SAM instrument typically accentuates Overarching Processes, Employees and Community Aspects. Together these Aspects account for a weight of more than 65%.
  Two other Aspects that are severely overrepresented are The Board and Economic performance drivers, while the Aspects Emissions, Products and Services and Environmental conformance are severely underrepresented. The term Environmental conformance refers to the environmental impact companies have on the natural and social environment. The Overarching Processes Aspect especially inclines towards severe overrepresentation.

  The KLD instrument covers 59 Sustainability Items (or 26%) and all four Dimensions of the CSAF. With weights of 23 and 57% the Social and Governance Dimensions are slightly overrepresented and with a weight of 6% the Economic Dimension is modestly overrepresented. With a weight of 14% the Environment Dimension is modestly underrepresented.

  The following Aspects are severely overrepresented in the instrument: The Board, Competitors, Economic market value and Economic performance drivers. Severely underrepresented are the Principles, Shareholders, Business partners, and Environmental conformance Aspects. Figure 2 further shows that almost all environmental Aspects are either absent or underrepresented.

  In figures 3A and 3B the degree of overlap between the SAM and KLD instruments, and the overlap of both screening instruments with the CSAF is presented. The degree of overlap is expressed in absolute and relative terms. The absolute overlap refers to the number of overlapping Sustainability Items, Issues, Aspect and Dimensions. The relative overlap is calculated as the number of Sustainability Items, Issues, Aspects or Dimensions as a percentage of the total number of jointly addressed Sustainability Items, Issues, Aspect or Dimensions.
  Figure 3A captures the overlap per aggregation level, whereas figure 3B covers the overlap for each of the four Dimensions at the Sustainability Item level.

  

  Figure 3A Overlap between the SAM and KLD Instruments and the CSA

  The top row of figure 3A captures information about the overlap of the SAM and KLD instruments. As can be seen, both instruments have 28 Sustainability Items in common, while the relative overlap is 24%. Furthermore, the instruments have 28 Sustainability Issues, 16 Aspects and 4 Dimensions in common, subsequently corresponding with a relative overlap of 36%, 85% and 100%. Thus, the higher the aggregation level, the higher the relative overlap.
  The bottom row contains information on topical areas of the SAM and KLD instruments and partly corresponds with figure 2.

  

  Figure 3B Overlap between the SAM and KLD Instruments and the CSAF per Dimension

  As we saw earlier, the SAM instrument covers 85 Sustainability Items. According to Figure 3B these Sustainability Items are distributed among the four Dimensions as follows: Governance and Social Dimensions: 27 and 42 Items respectively, while the Environment and Economic Dimension both cover 8 Sustainability Items.
  The distribution of the 59 Sustainability Items of the KLD instrument among the four Dimensions is as follows: Governance and Social Dimensions: 9 and 29 Items respectively, whereas the Environment and Economic Dimensions capture 16 and 5 Items.
  Judged by the relative overlap, the SAM instrument is typically governance oriented, while the KLD-instrument is slightly biased towards environmental dimension.

  Furthermore, Figure 3B teaches us that, with 32%, overlap of the SAM and KLD screening instruments is the highest at the Social Dimension level. The overlap between both instruments is smallest for environment related issues: the relative overlap at the Environment Dimension level is only 14%.

  

  Figure 4A Correlations between the SAM and KLD Instruments and the CSAF

  Absolute or relative overlap figures inform us about the absolute or relative number of Sustainability Items, Issues, Aspects and Dimensions the SAM and KLD instruments have in common. However, these figures do not inform us about differences in ranking of topical areas enclosed in the screening instruments.
  Figures 4A and 4B contain Spearman coefficients pertaining to the correlations of the weight distributions of the SAM and KLD screening instruments, and the correlations of the weight distributions of both screening instruments with the CSAF’s weight distribution. The Spearman correlations inform us about the similarity of weight distributions or the ranking of topical areas. These correlations can vary from -1 to 1. In case the Spearman coefficient equals 1, then weight distributions match perfectly, which means that rankings of topical areas are identical. If the coefficient equals -1, the weight distributions are reciprocal, meaning that topical areas that rank highest in one instrument rank lowest in the benchmark instrument and vise versa. In case the Spearman correlation coefficient equals 0, then the rankings of topical areas are not correlated at all.

  At the Sustainability Items level, correlations are statistically significant (p<0.05), but fairly low (0.151). At the Issues and Aspect levels, correlations are also statistically significant. Especially at the Aspect level, correlations are substantial (0.631). The weight distributions of the Governance, Social, Environment and Economic Dimensions of both instruments are identical. In both instruments the Social Dimension ranks highest, followed by the Governance, Environment and Economic Dimension.
  Also correlations of the SAM and KLD instruments with the CSAF at the Sustainability Item, Issues and Aspect level are high and statistically significant (p<0001), indicating that the ranking of these topical areas are fairly similar.

  

  Figure 4B Correlations between the SAM and KLD Instruments and the CSAF per Dimension

  Figure 4B contains Spearman correlation coefficients of the weight distributions or ranking of Sustainability Items per Dimension. As we can see, Spearman correlations pertaining to the ranking of Sustainability Items are statistically significant (p<0.05) but not overwhelmingly at the Social Dimension level. For all other Dimensions correlations are not statistically significant.

  Furthermore, correlations of the SAM and KLD instrument and the CSAF are statistically significant (p<0.05) for the Governance and Social Dimensions. The SAM instrument also correlates statistically significant with the Environment Dimension of the CSAF.

• 8. Conclusion

  In this paper validity and compatibility of the SAM and KLD screening instruments are the central themes. A screening instrument can be called valid if it identifies performance that is important to society. A screening instrument is called compatible if these instruments can be used interchangeably.

  8.1 Validity

  The outcome of the content-analysis approach suggests that there clearly is not a perfect match between the SAM and KLD instruments and the CSAF. This indicated that the SAM en KLD screening instruments cannot be called valid measures of corporate sustainability performance when the CSAF is taken as the standard. However, as discussed earlier, assessing validity is not a binary phenomenon. A simple “yes” or “no” will not do. To make things more complicated, the validity of a measure also depends on the aggregation level under observation. It makes a big difference if validity is assessed at a high (i.e. Dimension) or low aggregation (i.e. Sustainability Item) level.
  At the highest level there is a perfect match with the CSAF, indicating that both screening instruments are valid measures of corporate sustainability performance. At the lowest level the situation is quite different. Here we can see that overlap is relatively poor. Besides, the weight distributions of the SAM and KLD instruments and the CSAF are also largely dissimilar.

  When we break the validity discussion down to the Dimension level, we can see that the overlap between the SAM and KLD instruments is also relatively poor, although the SAM instrument is slightly more attuned to the CSAF than the KLD instrument. This most notably applies to the Governance Dimension. The SAM instrument scores relatively poor on environmental issues. This is due to the fact that we analyzed the generic (i.e. non-industry specific) screening instrument. SAM typically addresses environmental issues in industry specific supplements.

  8.2 Compatibility

  The second question in this study concerns the compatibility of the SAM and KLD screening instruments. Or more accurately phrased: the extent to which these rating schemes are compatible.
  If we would consider a compatibility assessment of screening instruments as a binary phenomenon, then we should conclude that the SAM and KLD instruments are not compatible.
  However, just like validity, assessing the compatibility of screening instruments is not a binary phenomenon.
  The extent to which the rating schemes of the two SRAs are compatible relates to the aggregation level. At the highest aggregation (or Dimension) level the screening instruments are perfectly compatible. Both instruments cover governance, social, environmental and economic issues.
  At the lowest (or Sustainability Item) level the situation is quite different. At this level, compatibility of both instruments is poor. Overlap is poor and the weight distributions hardly correlate. Obviously both instruments reflect different interpretations of corporate sustainability performance.
  Compatibility is the highest for the Social Dimension. For the Governance, Environment and Economic Dimension compatibility is very poor.
  The overall conclusion should therefore be that compatibility of the SAM and KLD screening instruments is (very) poor and that for this reason these instruments cannot be used interchangeably

• 9. Discussion and Limitations

  In the context of this paper a screening instrument is called a valid measure of corporate sustainability performance if it identifies performance that is important to society. Therefore, the question in this case is: what exactly is performance that is important to society?
  In this study the CSAF was used to assess the validity of the screening instruments of two prominent players in the corporate sustainability performance arena. This framework builds on a set of 24 popular sustainability (reporting) guidelines. In the construction process of these guidelines a multitude of stakeholder groups across the globe were engaged. Therefore, with a slight leaning to exaggeration, we might say that the CSAF is a valid measure of corporate sustainability performance, since it contains and embodies issues that companies are deemed responsible or accountable for and hence identifies performance that is important to society. Although the CSAF is based on partial analysis sustainability (reporting) guidelines – the number of guidelines presently available greatly exceeds the number of guidelines that lie at the basis of the CSAF – the framework presents a well enough outline of what corporate sustainability performance might look like. To further improve the quality of the CSAF many more sustainability (reporting) guidelines need to be added to the selection presented in Appendix 1.
  Also, the CSAF has proven to be a suitable format for assessing the contents of screening instruments. The SAM and KLD instruments only cover a handful of content areas that do not precisely match the list of Sustainability Items included in the CSAF.

  Another limitation of the CSAF is that it provides a static picture of a process that is inherently dynamic.⁷⁷xSethi 1975. Expectations of stakeholders concerning the contribution to a better world by businesses are not static. This implies that the contents regarding sustainability (reporting) guidelines that voice stakeholder expectations are also subject to change. Besides, new sustainability (reporting) guidelines are continuously developed, while others cease to exist. All this means the contemporal vision on corporate sustainability performance as captured by the CASF needs constant testing and readjustment.

  In the outset of this paper, a reference was made to the confusion that exists in business community, but also elsewhere regarding to the exact interpretation of what corporate sustainability performance is or might look like. This confusion is – at least partly – due to a multitude of corporate sustainability performance constructs, definitions and guidelines that exist today. However, this confusion is not a new phenomenon. In 1975(!), Sethi already stated that “corporate social responsibility has been used in so many different ways that it has lost all meaning. Devoid of an internal structure and content, it has become to mean many different things to all people”.⁷⁸xIbid.
  Although we have compared the screening instruments of only two SRAs, we can say that the limited compatibility of the SAM and KLD screening instruments adds to the existing confusion in the sustainability arena. Not only because the rating schemes cannot be used interchangeably, but also because each of these rating schemes represents a unique perception of corporate sustainability performance. And if it is interpreted differently by different SRAs, it should also be no surprise that companies are rated differently by different SRAs. This not only adds to the corporate sustainability performance confusion, but also increases skepticism of the business (and financial) community towards SRAs, since a multitude of inherently different rating schemes undermines the credibility of corporate sustainability performance assessment agencies.
  The fact that corporate sustainability performance is interpreted differently by different SRAs is not necessarily a problem. If SRAs are transparent about their rating methodologies, clients, scientists as well as other stakeholders would be able to analyze, benchmark and compare these methodologies. However, this is not likely to happen, at least not in the short term, because these rating methodologies very often are trademarks of SRAs, enabling SRAs to distinguish themselves in the corporate sustainability performance rating scene.

  SAM and KLD are also rather reluctant to provide more detailed information about their rating methodologies, especially the rating procedures. How the screening results are processed and what weights are assigned to the various sustainability indicators is not publicly disclosed.
  Ruf et al. (2001) state that the KLD screening instrument reflects the concerns historically held by social investors and social fund managers. This means that the instrument is based on the views of just a few stakeholder groups and may – with the fable of the six blind men and the elephant in mind, discussed in paragraph 6.2 – not comprehensively capture corporate sustainability performance .
  SAM, on the other hand, claims that its screening instrument is based on widely accepted standards, implying that the SAM instrument already takes expectations of a wide variety of stakeholders into account. However, the rather low correlations between the SAM screening instrument and the CSAF do not seem to support this claim.

  The poor validity of the SAM and KLD screening instruments also has consequences for the products and indexes that have been developed on the basis of these instruments. SAM launched (and still supports) the Dow Jones Sustainability Index (DJSI), while KLD is responsible for the KLD 400 Social Index (DSI).⁷⁹xThis index is also known as the Domini Social 400 Index. Both are famous and popular indexes in their own right. Being added to these indexes has great reputational value for companies. However, if the screening instrument underlying these indexes are – as we have seen – relatively poor measures of corporate sustainability performance, then the credibility and reputation of these indexes may also be at stake.
  Interestingly however, SustainAbility’s “Rate the Raters” study proves otherwise.⁸⁰xSustainAbility, Rate the Rater Phase Two, Taking Inventory of the Ratings Universe, October 2010b. Available at: <www.sustainability.com/library/rate-the-raters-phase-two>. This study shows that the DJSI and DSI are considered credible sustainability performance ratings. This in itself is remarkable, because SAM as well as KLD are not very keen on providing in-depth information about their rating schemes and procedures underlying these indexes. This makes it very difficult for outside audiences to effectively assess and compare the rating instruments and procedures used by these SRAs, and – in consequence – to effectively judge the transparency and credibility of the indexes as measures of corporate sustainability performance.

• Appendix 1 Sustainability (Reporting) Guidelines Analyzed for Identfying Sustainability Issues

  1	Principles for Global Corporate Responsibility: Bench Marks for Measuring Business Performance
  2	Business in the Community: Winning with Integrity
  3	The Caux Round Table Principles for Business
  4	Ceres Principles
  5	Ceres Reporting Requirements for Small enterprises and Non-Profit Organizations
  6	CSR Europe: Communicating Corporate Social Responsibility
  7	Department for Environment, Food and Rural Affairs: Environmental Reporting, General Guidelines.
  8	Ethical Trading Initiative, The ETI Base Code
  9	Fair Labor Association, Workplace Code of Conduct
  10	The UN Global Compact
  11	Global Reporting Initiative, Sustainability Reporting Guidelines 2002
  12	The Global Sullivan Principles of Social Responsibility
  13	International Chamber of Commerce, The Business Charter for Sustainable Development
  14	Social Venture Network, Standards of corporate Social Responsibility
  15	OECD Principles of Corporate Governance
  16	The OECD Guidelines for Multinational Enterprises
  17	ISO 14031 Environmental Management-Environmental Performance Evaluation- Guidelines
  18	Public Environmental Reporting Initiative
  19	Worker Rights Consortium, Model Code of Conduct
  20	Keidanren, Global Environmental Charter
  21	WBCSD, Measuring Eco-Efficiency
  22	NIVRA, Environmental Reporting Checklist81xThe NIVRA Checklist Milieuverslaggeving was originally written in Dutch. The translations of the name of the publication as well as of the analyzed sections were made by the author. (Checklist Milieuverslaggeving)
  23	Social Accountability International, SA8000
  24	The Stakeholder Alliance, The Sunshine Standards

• Appendix 2: Sustainability Items, Sustainability Issues, Aspects and Dimensions⁸²xIn parenthesis the weights of the Dimensions, Aspects, Sustainability Issues and Sustainability Items are mentioned.

  0. Governance Dimension (20.47)
  0.1 Overarching Principles (1.99)
  0.1.1 Ethical behavior/Integrity (1.39)
  0.1.1.1 Ethical behavior/Integrity (1.39)
  0.1.2 Mission/Governing philosophy/codes of conduct (0.60)
  0.1.2.1 Mission/Governing philosophy/codes of conduct (0.60)

  0.2 Overarching processes (11.05)
  0.2.1Objectives/Targets/Goals/Priorities (0.68)
  0.2.1.1 Governance Objectives/Targets/Goals/
  Priorities (0.11)
  0.2.1.2 Social objectives/targets/ goals/priorities (0.10)
  0.2.1.3 Environmental objectives/targets/goals/
  priorities (0.36)
  0.2.1.4 Economic objectives/targets/goals/
  priorities (0.11)
  0.2.2 Compliance
  0.2.2.1 Compliance with (international) standards, protocols and conventions (2.51)
  0.2.2.2 Compliance with laws, rules and regulations (1.60)
  0.2.2.3 Fines/Prosecutions/offences/ violations (0.25)
  0.2.3 Managements systems (1.27)
  0.2.3.1 Management systems (0.21)
  0.2.3.2 Social Management systems (0.21)
  0.2.3.3 Environmental management systems (0.75)
  0.2.3.4 Economic management systems (0.08)
  0.2.3.5 Sustainable Management systems (0.01)
  0.2.4 Responsible individuals/teams (0.11)
  0.2.4.1 Responsible individuals/teams (0.11)
  0.2.5 Audits/Validations/ Assessments/Monitoring (0.77)
  0.2.5.1 Audits/Validations/Assessments/Monitoring (0.77)
  0.2.6 Strategies/Policies/Programs/practices and procedures (1.74)
  0.2.6.1 Strategies/Governance policies/programs/procedures and activities (0.66)
  0.2.6.2 Social strategies/policies/programs/procedures and activities (0.13)
  0.2.6.3 Environmental strategies/policies/programs/procedures and activities (0.82)
  0.2.6.4 Economic strategies/policies/programs/ procedures and activities (0.14)

  0.3 Stakeholders (2.73)
  0.3.1 Stakeholder rights (0.07)
  0.3.1.1 Stakeholder rights (0.07)
  0.3.2 Stakeholder participation (0.25)
  0.3.2.1 Stakeholder participation (0.25)
  0.3.3 Stakeholder communication (1.64)
  0.3.3.1 Stakeholder communication (1.64)
  0.3.4 Stakeholder interests (0.39)
  0.3.4.1 Stakeholder interests (0.39)
  0.3.5 Stakeholder identification and selection (0.38)
  0.3.5.1 Stakeholder identification and selection (0.38)

  0.4 Shareholders (1.74)
  0.4.1 Shareholder rights (general indication) (0.08)
  0.4.1.1 Shareholder rights (0.08)
  0.4.2 (Rights to) secure methods of ownership registration (0.01)
  0.4.2.1 (Rights to) secure methods of ownership registration (0.01)
  0.4.3 (Right to) Convey or transfer shares (0.03)
  0.4.3.1 (Right to) Convey or transfer shares (0.03)
  0.4.4 Right to information/Shareholder and investor communication (0.81)
  0.4.4.1 Right to information/Shareholder and investor communication (0.81)
  0.4.5 Right to participate/ Shareholder participation (0.14)
  0.4.5.1 Right to participate/ Shareholder participation (0.14)
  0.4.6 Right to vote/ shareholder voting (0.28)
  0.4.6.1 Right to vote/ shareholder voting (0.28)
  0.4.7 Right to profit sharing/shareholder compensation (0.11)
  0.4.7.1 Right to profit sharing/shareholder compensation (0.11)
  0.4.8 Ethical conformance towards shareholders (0.20)
  0.4.8.1 Equitable treatment of shareholders (0.17)
  0.4.8.2 Fair treatment of shareholders (0.03)
  0.4.9 Shareholder value (0.07)
  0.4.9.1 Shareholder value (0.07)
  0.4.10 Sustainable performance of subsidiaries/parent (0.01)
  0.4.10.1 Sustainable performance of subsidiaries (0.01)

  0.5 The Board (2.95)
  0.5.1 Composition of the Board (0.53)
  0.5.1.1 Composition of the Board (0.53)
  0.5.2 Selection, election and removal of Board members (0.13)
  0.5.2.1 Selection, election and removal of Board members (0.13)
  0.5.3 Communication to/with the Board/senior management (0.36)
  0.5.3.1 Communication to/with the Board/senior management (0.36)
  0.5.4 Expertise of Board members (0.06)
  0.5.4.1 Expertise of Board members (0.06)
  0.5.5 Board Committees (0.20)
  0.5.5.1 Board Committees (0.20)
  0.5.6 Key functions/responsibilities of the Board (0.99)
  0.5.6.1 Key functions/responsibilities of the Board (0.99)
  0.5.7 Compensation of Board members (0.57)
  0.5.7.1 Compensation of Board members (0.57)
  0.5.8 Board/senior management commitment (0.13)
  0.5.8.1 Board/senior management commitment (0.13)

  1. Social Dimension (54.45)
  1.1 Employees (26.86)
  1.1.1 Absenteeism and turn-over (0.45)
  1.1.1.1 Absenteeism and turn-over (0.45)
  1.1.2 Diversity (1.28)
  1.1.2.1 Diversity (1.28)
  1.1.3 Employee representation (0.64)
  1.1.3.1 Employee representation (0.64)
  1.1.4 Employee requirements (0.03)
  1.1.4.1 Employee requirements (0.03)
  1.1.5 Employer-Employee relationship (0.47)
  1.1.5.1 Employer-Employee relationship (0.47)
  1.1.6 Equal opportunity and equal remuneration (0.79)
  1.1.6.1 Equal Opportunity (0.63) Equal remuneration (0.17)
  1.1.7 Human rights (11.14)
  1.1.7.1 Human rights (0.98)
  1.1.7.2 Child labor (1.25)
  1.1.7.3 Forced Labor (0.57)
  1.1.7.4 Freedom of association (0.38)
  1.1.7.5 Collective bargaining (0.20)
  1.1.7.6 Voice complaints (0.36)
  1.1.7.7 Non-discrimination (6.55)
  1.1.7.8 Discipline (0.28)
  1.1.7.9 Harassment and abuse (0.50)
  1.1.7.10 Women’s and girls’ rights (0.07)
  1.1.8 Health and Safety (3.96)
  1.1.8.1 Health and Safety (3.96)
  1.1.9 Interaction with employees (1.35)
  1.1.9.1 Open communication with/to employees (1.09)
  1.1.9.2 Employee participation (0.26)
  1.1.10 Retention and hiring (0.36)
  1.1.10.1 Retention and hiring (0.36)
  1.1.11 Security (0.18)
  1.1.11.1 Security (0.18)
  1.1.12 Training and (opportunities for) Development (2.75)
  1.1.12.1 Training and (opportunities for) development (2.75)
  1.1.13 Wages and Benefits (1.87)
  1.1.13.1 Wages and benefits (1.87)
  1.1.14 Work-life balance (1.21)
  1.1.14.1 Work-life (0.22)
  1.1.14.2 Working hours (0.52)
  1.1.14.3 Telecommuting (0.03)
  1.1.14.4 Retirement planning (0.04)
  1.1.14.5 Job sharing (0.03)
  1.1.14.6 Child/elder care (0.13)
  1.1.14.7 Other forms of care (0.07)
  1.1.14.8 Maternity/paternity leave (0.08)
  1.1.14.9 Other forms of leave (0.10)
  1.1.15 Ethical attitude towards employees (0.38)
  1.1.15.1 Respect towards employees (0.14)
  1.1.15.2 Ethical issues/performance (0.24)

  1.2 Customers (7.36)
  1.2.1 Advertising-Marketing (0.75)
  1.2.1.1 Advertising-marketing (0.75)
  1.2.2 Commitment to customers (0.47)
  1.2.2.1 Commitment to customers (0.47)
  1.2.3 Interaction with customers (1.76)
  1.2.3.1 Open communication to/with customers (0.78)
  1.2.3.2 Customer participation (0.11)
  1.2.3.3 Customer complaints (0.81)
  1.2.3.4 Customer education (0.06)
  1.2.4 Customer health and safety (0.45)
  1.2.4.1 Customer health and safety (0.45)
  1.2.5 Customer privacy (0.14)
  1.2.5.1 Customer privacy (0.14)
  1.2.6 Customer rights (0.04)
  1.2.6.1 Customer rights (0.04)
  1.2.7 Products and services (3.57)
  1.2.7.1 Misuse of company products (0.03)
  1.2.7.2 Product characteristics (0.29)
  1.2.7.3 Product information/labeling (1.53)
  1.2.7.4 Product price (0.07)
  1.2.7.5 Product quality/customer requirements/specifications (0.24)
  1.2.7.6 Product recalls/returns and defective products (0.11)
  1.2.7.7 Safety of products and services (0.81)
  1.2.7.8 Social impacts of product and services (0.31)
  1.2.7.9 Research and development/New products and services (0.10)
  1.2.7.10 Distribution of products (0.01)
  1.2.7.11 Product or services produced, delivered or sold to customers (0.04)
  1.2.7.12 Storage of products (0.03)
  1.2.8 Special interest groups (0.18)
  1.2.8.1 Special interest groups (0.18)

  1.3 Business partners (6.93)
  1.3.1 (Monitoring) Business partners conformance to ethical conduct (0.66)
  1.3.1.1 Monitoring of business partners (0.20)
  1.3.1.2 Ethical conformance of business partners (0.46)
  1.3.2 Commitment/ethical conformance towards business partners (1.59)
  1.3.2.1 Payment conduct toward business partners (0.25)
  1.3.2.2 Ethical conformance towards business partners (1.34)
  1.3.3 Compliance with law, regulations and (international) standards (0.61)
  1.3.3.1 Compliance with law, regulations and (international) standards (0.61)
  1.3.4 Environmental conformance of business partners (0.52)
  1.3.4.1 Environmental performance of business partners (0.24)
  1.3.4.2 Use of hazardous materials (0.03)
  1.3.4.3 Use of cleaning agents (0.03)
  1.3.4.4 Use of recyclable and reusable materials (0.04)
  1.3.4.5 Waste generation (0.04) Transportation (0.14)
  1.3.5 Open communication with/to/of business partners (0.36)
  1.3.5 1 Open communication with/to business partners (0.25)
  1.3.5.2 Open communication of business partners (0.06)
  1.3.5.3 Education of business partners (0.06)
  1.3.6 Cooperation of/with business partners (0.40)
  1.3.6.1 Cooperation of/with business partners (0.40)
  1.3.7 Purchasing decisions/selection of business partners (0.54)
  1.3.7.1 Selection of business partners (0.33) Purchasing decisions (0.21)
  1.3.8 Social behavior towards business partners (2.24)
  1.3.8.1 Social performance (0.14)
  1.3.8.2 Human rights (1.42)
  1.3.8.3 Wages and benefits (0.26)
  1.3.8.4 Working hours (0.15)
  1.3.8.5 Health and safety (0.20)
  1.3.8.6 Education and training (0.07)

  1.4 Community (12.33)
  1.4.1 Awards (0.10)
  1.4.1.1 Awards (0.10)
  1.4.2 Bribery and corruption (1.46)
  1.4.2.1. Bribery and corruption (1.46)
  1.4.3 Indigenous people (1.20)
  1.4.3.1 Indigenous people (1.20)
  1.4.4 Interaction with the community (2.15)
  1.4.4.1 Open communication with/to the community (1.30)
  1.4.4.2 Cooperation, participation and partnerships (0.85)
  1.4.5 Lobbying (0.33)
  1.4.5.1 Lobbying (0.33)
  1.4.6 Political activities (0.32)
  1.4.6.1 Political activities (0.32)
  1.4.7 Public policy involvement (0.26)
  1.4.7.1 Public policy involvement (0.26)
  1.4.8 Social conformance (5.99)
  1.4.8.1 Enhancement of community well-being (0.67)
  1.4.8.2 Social impacts (0.47)
  1.4.8.3 Cultural conformance (0.43)
  1.4.8.4 Human rights (0.67)
  1.4.8.5 Education/training (0.43)
  1.4.8.6 Science and technology (0.31)
  1.4.8.7 Employment of local personnel (0.18)
  1.4.8.8 Female and minority-owned businesses (0.14)
  1.4.8.9 Public health and safety (1.35)
  1.4.8.10 Cash donations (0.24)
  1.4.8.11 In-kind donations (1.10)
  1.4.9 Taxes paid and benefits received (0.53)
  1.4.9.1 Taxes and compensations paid (0.38)
  1.4.9.2 Benefits/subsidies received (0.14)

  1.5 Competitors (0.89)
  1.5.1 Competitive conduct (0.60)
  1.5.1.1 Competitive conduct (0.60)
  1.5.2 Conformance to competitors (0.26)
  1.5.2.1 Conformance to competitors (0.26)
  1.5.3 Cooperation with authorities (0.03)
  1.5.3.1 Cooperation with authorities (0.03)

  1.6 Providers of capital (0.08)
  1.6.1 Interest payments (0.08)
  1.6.1.1 Interest payments (0.08)

  Environment Dimension (21.31)
  2.1 Emissions (3.60)
  2.1.1. Emissions to air (2.16)
  2.1.1.1 Greenhouse gas emissions (0.67)
  2.1.1.2 Acidifying gases (0.35)
  2.1.1.3 Ozone depleting gases (0.28)
  2.1.1.4 (Other) emissions to air (0.86)
  2.1.2 Emissions to land/erosion (0.36)
  2.1.2.1 Emissions to land/erosion (0.36)
  2.1.3 Emissions to water/water pollution (0.56)
  2.1.3.1 Emissions to water/water pollution (0.56)
  2.1.4 Other emissions (0.52)
  2.1.4.1 Emissions (0.32)
  2.1.4.2 Other emissions (0.20)

  2.2 Life support (1.73)
  2.2.1 Animal welfare (0.06)
  2.2.1.1 Animal welfare (0.06)
  2.2.2 Biodiversity/ecosystems (1.57)
  2.2.2.1 Biodiversity (0.85)
  2.2.2.2 Ecosystems (0.72)
  2.2.3 Precautionary principle (0.10)
  2.2.3.1 Precautionary principle (0.10)

  2.3 Products and services (2.87)
  2.3.1 GMOs (0.15)
  2.3.1.1 Genetically Modified Organisms (0.15)
  2.3.2 Products and services (2.72)
  2.3.2.1 Resource efficiency of products and services (0.15)
  2.3.2.2 Environmental impact of products and services (1.49)
  2.3.2.3 Life-cycle of products and services (0.42)
  2.3.2.4 Recycling of products and services (0.14)
  2.3.2.5 Reuse of products and services (0.11)
  2.3.2.6 Disposal/end-of-life of products and services (0.21)
  2.3.2.7 Biodegradability of products (0.03)
  2.3.2.8 Innovative products and services (0.17)

  2.4 Research and technology (0.78)
  2.4.1 Research and technology (0.78)
  2.4.1.1 Research (0.28)
  2.4.1.2 Technology/know-how (0.50)

  2.5 Resources (4.22)
  2.5.1 Energy (1.44)
  2.5.1.1 Energy use and conservation (0.96)
  2.5.1.2 Energy generation (0.03)
  2.5.1.3 Renewable energy (0.26)
  2.5.1.4 Energy sold to others (0.18)
  2.5.2 Land (0.50)
  2.5.2.1 Use of land (0.24)
  2.5.2.2 Renewable land/soils (0.03)
  2.5.2.3 Use and conservation of (natural) resources (0.24)
  2.5.3 Materials (1.80)
  2.5.3.1 Materials (re)use and conservation (1.18)
  2.5.3.2 Use of hazardous substances and materials (0.31)
  2.5.3.3 Recycling (of materials) (0.20)
  2.5.3.4 Use and production of recycled and recyclable materials (0.11)
  2.5.4 Water (0.49)
  2.5.4.1 (Re)use of water (0.45)
  2.5.4.2 Recycling of water (0.04)

  2.6 Transport and equipment (0.89)
  2.6.1 Business-related travel and transport (0.72)
  2.6.1.1 Business-related travel/Transport (0.72)
  2.6.2 Equipment (0.17)
  2.6.2.1 Equipment (0.17)

  2.7 Waste (2.31)
  2.7.1 Generation of waste (0.99)
  2.7.1.1 Generation of non-hazardous waste (0.88)
  2.7.1.2 Generation of hazardous waste (0.11)
  2.7.2 Import and export of waste (0.03)
  2.7.2.1 Export of waste (0.01)
  2.7.2.2 Import of waste (0.01)
  2.7.3 Recycling and reuse of waste (0.47)
  2.7.3.1 Recycling and reuse of non-hazardous waste (0.35)
  2.7.3.2 Recycling and reuse of hazardous waste (0.13)
  2.7.4 Storage, disposal and transport of waste (0.82)
  2.7.4.1 Storage/disposal of non-hazardous waste (0.31)
  2.7.4.2 Storage/disposal of hazardous waste (0.33)
  2.7.4.3Transport of waste (0.18)

  2.8 Environmental conformance (4.91)
  2.8.1 (Reduction of) environmental impacts (2.47)
  2.8.1.1 (Reduction of) environmental impacts (2.47)
  2.8.2 Corrective actions (0.43)
  2.8.2.1 Corrective actions (0.43)
  2.8.3 Emergency preparedness/emergency events (0.28)
  2.8.3.1 Emergency preparedness/emergency events (0.28)
  2.8.4 Enhancement of the environment (1.25)
  2.8.4.1 Enhancement of the environment (1.25)
  2.8.5 Extraction and exploitation of resources (0.47)
  2.8.5.1 Extraction and exploitation of resources (0.47)

  Economic Dimension (3.76)
  3.1 Economic market value (1.67)
  3.1.1 Economic Value Added (0.24)
  3.1.1.1 Economic Value Added (EVA) (0.01)
  3.1.1.2 EVA-drivers (0.22)
  3.1.2 Bookkeeping measures (1.37)
  3.1.2.1 Assets (0.14)
  3.1.2.2 Liabilities (0.04)
  3.1.2.3 Revenues (0.01)
  3.1.2.4 Costs/expenditures (0.77)
  3.1.2.5 Financial and operating results (0.39)
  3.1.2.6 Cash flow (0.01)
  3.1.3 Return measures (0.07)
  3.1.3.1 Return measures (0.07)

  3.2 Economic performance drivers (0.31)
  3.2.1 Market forces (0.10)
  3.2.1.1 Market share (0.10)
  3.2.2 Innovation (0.11)
  3.2.2.1 Innovation (0.11)

  3.3 Externalities (1.88)
  3.3.1 Positive externalities (1.07)
  3.3.1.1 Economic stability (0.13)
  3.3.1.2 Employment (0.32)
  3.3.1.3 Economic growth (0.14)
  3.3.1.4 Income improvement (0.17)
  3.3.1.5 Investments (0.32)
  3.3.2 Negative externalities (0.46)
  3.3.2.1 Unemployment (0.45)
  3.3.2.2 Income deterioration (0.01)
  3.3.3 Economic impact on the community (0.35)
  3.3.3.1 Economic impact on the community (0.35)

Noten

• 1 In this paper the term corporate sustainability performance throughout is used as a generic term expressing the business-society relationship. Corporate sustainability performance relates to business’ performance in areas like corporate governance, social relations, contributions to a healthy environment and the creation of economic value.

• 2 J. Elkington, Cannibals with Forks. The Triple Bottom Line of the 21st Century Business, New Society Publishers, Gabriola Island, BC 1999.

• 3 A.B. Carroll, “The Pyramid of Corporate Social Responsibility: Toward the Moral Management of Organizational Stakeholders”, 34 Business Horizons 4, 1991, pp. 39-48.

• 4 N. Dentchev, To What extent is Business and Society Literature Idealistic?, Working Paper 2004/245, Ghent University.

• 5 J. Ranganathan, “Signs of Sustainability, Measuring Corporate Environmental and Social Performance”, in M. Bennet, P. James & L. Klinkers (eds.), Sustainable Measures, Evaluation and Reporting of Environmental and Social Performance, Greenleaf Publishing, Sheffield 1999, pp. 475-495.

• 6 To get an impression of the guidelines presently available, see: <www.oesorichtlijnen/navigator>.

• 7 At the time this research was conducted, KLD Research & Analytics was still an independent SRA. As of November 2009 KLD Research & Analytics became part of the RiskMetricsGroup, which is now part of MSCI, Inc. In this paper we will continue to use the abbreviation KLD.

• 8 This index is presently known as the MSCI KLD 400 Social Index

• 9 Mistra, “Values for Money”, 2004. Available at: <www.sustainability.com/publications/latest/values-for-money.pdf>; SustainAbility, Rate the Rater Phase One, Look Back and Current State. May 2010 (2010a). Available at: <www.sustainability.com/library/rate-the-raters-phase-one>; SustainAbility, Rate the Rater Phase Three, Uncovering Best Practices, February 2011 (2011a). Available at: <www.sustainability.com/library/rate-the-raters-phase-three>; SustainAbility, Rate the Rater Phase Four, The Necessary Future of Ratings, July 2011 (2011b). Available at: <www.sustainability.com/library/rate-the-raters-phase-four>.

• 10 P.D. Kinder & A.L. Domini, “Social Screening: Paradigms Old and New”, 4 Journal of Investing 6, 1997, pp. 12-19.

• 11 E. Dommerholt, Corporate Sustainability Performance: Constructs, Measures and Investors’ Responses, Doctoral dissertation, VU, September 2009.

• 12 E.g. E.H. Bowman & M. Haire, “A Strategic Posture Toward Corporate Social Responsibility”, XVII California Management Review 2, 1975, pp. 49-58; H.R. Fogler & F. Nutt , “A Note on Social Responsibility and Stock Valuation”, 18 Academy of Management Journal 1, 1975, pp. 155-160; B.H. Spicer, “Investors, Corporate Social Performance and Information Disclosure: An Empirical Study”, 53 The Accounting Review 1, 1978, pp. 94-111.

• 13 E.g. J. Rockness, P. Schlachter & H.O. Rockness, “Hazardous Waste Disposal, Corporate Disclosure, and Financial Performance in the Chemical Industry”, 1 Advances in Public Interest Accounting 1986, pp. 167-191.

• 14 G. Jarrell & S. Pelzman, “The Impact of Product Recalls on the Wealth of Sellers”, 93 Journal of Political Economy 3, 1985, pp. 512-536; S.W. Pruitt & D.R. Peterson, “Security Price Reactions Around Product Recall Announcements”, IX The Journal of Financial Research 2, 1986, pp. 113-122; P. Bromiley & A. Marcus, “The Deterrent to Dubious Corporate Behaviour: Profitability and Safety Recalls”, 10 Strategic Management Journal 1986, pp. 233-250; W.N. Davidson & D.N.Worrell, “Research Notes and Communications: The Effect of Product Recall Announcements on Shareholder Wealth”, 3 Strategic Management Journal 1992, pp. 467-473.

• 15 W.N. Davidson III & D.L. Worrell, “The Impact of Announcements of Corporate Illegalities on Shareholder Returns”, 31 Academy of Management Journal 1, 1988, pp. 195-200; J. Frooman, “Socially Irresponsible and Illegal Behavior and Shareholder Wealth, A Meta Analysis of Event Studies”, 36 Business & Society 3, 1997, pp. 221-249.

• 16 D.M. Patten, “The Market Reaction to Social Responsibility Disclosures: The Case of the Sullivan Principles Signings”, 15 Accounting, Organizations and Society 6, 1990, pp. 575-587; McMillan, “Corporate Social Investments: Do they Pay?”, 15 Journal of Business Ethics 1996, pp. 309-314.

• 17 R.D. Klassen & C.P. McLaughlin, “The Impact of Environmental Management on Firm Performance”, 42 Management Science 8, 1996, pp. 1199-1214; K.B. Hendricks & V.R. Singhal, “Quality Awards and the Market Value of the Firm: An Empirical Investigation”, 42 Management Science 3, 1996, pp. 415-436.

• 18 S.J. Feldman, P.A. Soyka & P.G. Ameer, “Does Improving a Firm's Environmental Management System and Environmental Performance Result in a Higher Stock Price?”, 6 Journal of Investing 4, 1997, pp. 87-97.

• 19 S. Webley & E. More, Does Business Ethics Pay?, Institute of Business Ethics, London 2003.

• 20 L. Becchetti, R. Ciciretti & I. Hasan, Corporate Social Responsibility and Shareholder Value: An Event-study, Federal Reserve Bank of Atlanta, Working Paper 2007-6, April 2007. Available at: <www.frbatlanta.org/filelegacydocs/wp0607.pdf>; Dommerholt 2009.

• 21 A.B. Carroll, “A Commentary and an Overview of Key Questions on Corporate Social Performance Measurement”, 39 Business and Society 4, 2002, pp. 466-478; T. Rowley & S. Berman, “A Brand New Brand of Corporate Social Performance”, 39 Business & Society 4, 2000, pp. 397-418.

• 22 J. Griffin & J. Mahon, “The Corporate Social Performance and Corporate Financial Performance Debate, Twenty-Five Years of Incomparable Research”, 36 Business & Society 1, 1997, pp. 5-31.

• 23 S. Zyglidopoulos, “The Impact of Accidents on Firms’ Reputation for Social Performance”, 40 Business & Society 4, 2001, pp. 416-441.

• 24 G.J. Alexander & R.A. Buchholz, “Corporate Social Responsibility and Stock Market Performance”, 21 Academy of Management Journal 3, 1978, pp. 479-486.

• 25 R. Wokutch & B. Spencer, “Corporate Saints and Sinners: The Effect of Philanthropic and Illegal Activity on Organizational Performance”, 29 California Management Review 2, 1987, pp. 63-77; S.A. Waddock & S.B. Graves, “The Corporate Social Performance-Financial Performance Link”, 18 Strategic Management Journal 4, 1997, pp. 303-319; Zyglidopoulos 2001; P. Dev Sinha & T. Salas, The Relationship Between Corporate Social Responsibility and Profitability of Hospitality Firms: Do Firms that Do Good Also Do Well?, School of Hotel Administration, Cornell University, Ithaca NY 2002.

• 26 D.J. Wood & R.E. Jones, “Stakeholder Mismatching: A Theoretical Problem in Empirical Research on Corporate Social Performance”, 3 The International Journal of Organizational Analysis 3, 1995, pp. 229-267.

• 27 W.F. Abbott & R.J. Monsen, “On the Measurement of Corporate Social Responsibility: Self Reported Disclosures as a Method of Measuring Corporate Social Involvement”, 22 Academy of Management Journal 3, 1979, pp. 501-515.

• 28 P.L. Cochran & R.A. Wood, “Corporate Social Responsibility and Financial Performance’, 27 Academy of Management Journal 1, 1984, pp. 42-56.

• 29 Bowman & Haire 1975.

• 30 R.W. Ingram, “An Investigation of the Information Content of (Certain) Social Responsibility Disclosures”, 16 Journal of Accounting Research 2, 1978, pp. 270-285.

• 31 D.W. Walden & B.N. Schwartz , “Environmental Disclosures and Public Policy Pressures”, 16 Journal of Accounting and Public Policy 1997, pp. 125-154.

• 32 D. Neu, H. Warsame & K. Pedwell, “Managing Public Impressions: Environmental Disclosures in Annual Reports”, 23 Accounting, Organizations and Society 3, 1998, pp. 265-282.

• 33 E.g., D.M. Patten, “Variability in Social Disclosure: A Legitimacy Based Analysis”, 6 Advances in Public Interest Accounting 1995, pp. 273-285.

• 34 M. Sharfman, “The Construct Validity of the Kinder, Lydenberg & Domini Social Performance Ratings Data”, 5 Journal of Business Ethics 1996, pp. 287-296.

• 35 Kinder & Domini 1997.

• 36 Ibid.

• 37 Ibid.

• 38 H. Schäfer, “International Corporate Social Responsibility Rating Systems”, 20 Journal of Corporate Citizenship 2005, pp. 107-120.

• 39 Schäfer 2005.

• 40 Sustainability, 2011b

• 41 Schäfer 2005.

• 42 Ibid.

• 43 Schäfer 2005; T. van den Brink, Guide Screening and Rating Sustainability, Triple P Performance Centre, Vrije Universiteit Amsterdam 2002.

• 44 Schäfer 2005.

• 45 Sustainability, 2011a

• 46 Van den Brink 2002.

• 47 WBCSD, World Business Council for Sustainable Development. Sustainable Development Reporting: Striking the Balance, Geneva, World Business Council for Sustainable Development 2003; Van den Brink 2002

• 48 Van den Brink 2002.

• 49 S. de Hoo & H. King, “Company Sustainability Reporting and Sustainability Rating”, in van den Brink 2002.

• 50 Another, more practical reason for selecting these SRAs is they were the only SRAs that positively responded to my request to provide additional information about the (composition of) sustainability indexes they launched and still support: the Dow Jones Sustainability World Index family (SAM) and the Domini Social 400 Index (KLD).

• 51 See note 3.

• 52 See note 4.

• 53 Sharfman 1996; S.B. Graves & S.A. Waddock, “A Look at the Financial-Social Performance Nexus when Quality of Management is Held Constant”, 12 International Journal of Value-Based Management 1999, pp. 87-99.

• 54 S.A. Waddock & S.B. Graves, “The Corporate Social Performance-Financial Performance Link”, 18 Strategic Management Journal 4, 1997, pp. 303-319.

• 55 B. Ruf, K. Muralidhar, R. Brown, J. Janney & K. Paul, “An Empirical Investigation of the Relationship Between Change in Corporate Social Performance and Financial Performance: A Stakeholder Theory Perspective”, 32 Journal of Business Ethics 2001, pp. 143-156.

• 56 Ruf et al. 2001, p. 149

• 57 J. Harrison & R. Freeman, “Stakeholders, Social Responsibility and Performance: Empirical Evidence and Theoretical Perspectives”, 42 Academy of Management Journal 5, 1999, pp. 479-485.

• 58 Dommerholt 2009.

• 59 P. Brownell , Research Methods in Management Accounting, Coopers & Lybrand Accounting Research Methodology, Monograph no. 2, Coopers & Lybrand Australia 1995.

• 60 A. Chatterji & D. Levine, “Breaking Down the Wall of Codes: Evaluating Non-Financial Performance Measurement”, 48 California Management Review 2, 2006, pp. 29-51.

• 61 Sharfman 1996.

• 62 A.K. Chatterji & D.I.I. Levine, Imitate or differentiate? Evaluating the Validity of Corporate Social Responsibility Ratings, Working Paper Series, University of California, Berkely 2008. Available at: <http://repositories.cdlib.org/crb/wps/37>.

• 63 De Hoo & King 2002; WBCSD 2003.

• 64 These questions are excluded from the analyzed sections because they focus on the verification of a performance score by the responsible industry analyst.

• 65 J.M. Milne & R.W. Adler, “Exploring the Reliability of Social and Environmental Disclosures in Content Analysis”, 12 Accounting, Auditing & Accountability Journal 2, 1999, pp. 237-256.

• 66 S. Parsa & R. Kouhy, Disclosure of Information By UK Companies – A Case of Legitimacy Theory, London, Middlesex University Business School, 2001. Available at: <www.mubs.mdx.ac.uk/research/Discussion_Papers/Accounting_and_Finance>.

• 67 V.J. Duriau, R.K. Reger & M.D. Pfarrer, “A Content Analysis of the Content Analysis Literature in Organization Studies”, 10 Organizational Research Methods 1, 2007, pp. 5-34.

• 68 Milne & Adler 1999.

• 69 Duriau et al. 2007, p. 6.

• 70 Dommerholt 2009.

• 71 M.B.E. Clarkso, “A Stakeholder Framework for Analyzing and Evaluating Corporate Social Performance”, 20 Academy of Management Review 1, 1995, pp. 92-117; P. Bansal, “The Corporate Challenges of Sustainable Development”, 16 Academy of Management Executive 2, 2002, pp. 122-131; R.K. Mitchell, B.R. Agle & D.J. Wood, “Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts”, 22 Academy of Management Review 4, 1997, pp. 853-886; C. Oliver, “Strategic Responses to Institutional Processes”, 16 Academy of Management Review 1, 1991, pp. 145-179.

• 72 A.B. Carroll, “A Three-Dimensional Conceptual Model of Corporate Performance”, 4 Academy of Management 4, 1979, pp. 497-505.

• 73 The fable is adopted from: <www.jainworld.com/literature/story25.htm>.

• 74 S.P. Sethi, “Dimensions of Corporate Social Performance: An Analytical Framework”, California Management Review 1975, pp. 58-64.

• 75 B.M. Mitnick, “Commitment, Revelation, and the Testament of Belief: The Metrics of Measurement of Corporate Social Performance”, 39 Business & Society 4, 2000, pp. 419-465.

• 76 L. Steg, C.H. Vlek, S. Lindenberg, T. Groot, H. Moll, T. Schoot-Uiterkamp & A. van Witteloostuijn, Towards a Comprehensive Model of Sustainable Corporate Performance. Three dimensional modelling and practical measurement (Second Interim Report), Groningen, Departments of Economics, Environmental Sciences, Management Science, Psychology and Sociology, University of Groningen, Vrije Universiteit Amsterdam, University College Windesheim 2003.

• 77 Sethi 1975.

• 78 Ibid.

• 79 This index is also known as the Domini Social 400 Index.

• 80 SustainAbility, Rate the Rater Phase Two, Taking Inventory of the Ratings Universe, October 2010b. Available at: <www.sustainability.com/library/rate-the-raters-phase-two>.

• 81 The NIVRA Checklist Milieuverslaggeving was originally written in Dutch. The translations of the name of the publication as well as of the analyzed sections were made by the author.

• 82 In parenthesis the weights of the Dimensions, Aspects, Sustainability Issues and Sustainability Items are mentioned.