Skip to main content.
Home / Market & Economic Insights / ESG Scoring in an Imperfect World: Why We Rely on Our Own Proprietary System

ESG Scoring in an Imperfect World: Why We Rely on Our Own Proprietary System

At Sustainable Growth Advisers (SGA), we believe Environmental, Social, and Governance (ESG) factors have a meaningful impact on a company’s ability to generate long-term sustainable growth. One way we integrate these factors into our bottom-up fundamental research process is by utilizing ESG scores to enhance the rigor of our investment research and modeling, inform our portfolio decision making, and focus our management engagement.

ESG ratings are offered by many third party research service providers, but scoring varies widely across firms due to numerous challenges. These include 1) lack of consistent disclosure, 2) the qualitative nature of the subject matter and subjectivity in scoring, and 3) a lack of widely accepted reporting standards. Additionally, most ESG rating systems are based on industry-relative assessments rather than absolute scoring frameworks, further detracting from their value to our company-focused investment process.

In order to address these shortcomings, SGA utilizes a proprietary system to calculate E, S, and G scores based upon our assessment of material risks and opportunities for companies on our Qualified Company List that are eligible for purchase in our portfolios. As with other aspects of our investment process, we seek input from various respected external sources in the identification and analysis of ESG factors, including MSCI and ISS. However, we rely on our own informed judgment in determining the appropriate scope, measurement, and weightings of the various factors that contribute to ESG scores on a company by company basis.

Shortcomings of Third Party ESG Scoring

Understanding and comparing relative ESG risks and opportunities to companies’ future cash flows and empirical value is an important endeavor. However, formulating a comprehensive, objective scoring system is a conundrum that third party ESG ratings services have yet to solve satisfactorily. There are several reasons for this. First and foremost, disclosure is inconsistent across companies and incomplete even in the best of cases. For example, according to Goldman Sachs, 43% of all general ESG metrics have a disclosure rate of less than 20% per company.1

Secondly, the majority of ESG disclosure is qualitative in nature, particularly for environmental and social factors. In fact, according to the same Goldman Sachs research report, 40 of the top 50 environmental and social disclosures are policy-based. Lack of disclosure and the qualitative nature of the subject matter create numerous challenges to scoring systems. For instance, different ratings agencies have different methodologies to address data gaps, ranging from penalizing companies for each incidence of lack of disclosure to simply assuming industry average performance for a missing metric. In addition, some ratings agencies assess policies on a binary basis, rewarding the mere presence of disclosure rather than passing judgement on the content, leading to positive biases for companies operating in regions with higher disclosure requirements and larger companies with the resources to support more rigorous disclosure.

Lastly, many ESG scoring systems are based on assessments made on an industry-relative basis, determining the material factors of focus and the peer groups to which companies are compared. Such industry-relative orientations often result in inadequate consideration of company-specific nuances in terms of the appropriate scope, weight and measurement of ESG factors, while also generating misleading peer comparisons. In addition, an industry-relative orientation is incongruent with our company-focused, bottom-up investment process.

The challenges for ESG scoring systems related to disclosure levels and the qualitative nature of many ESG factors make it difficult for ESG ratings services to design scoring frameworks, and lead to wide variances in scores. In fact, a recent MIT study comparing the ESG scores of five leading agencies found a correlation of just 0.61.2 By comparison, scores for the two leading credit rating agencies have a 0.99 correlation thanks to the robust, standardized, and objective financial disclosure at their disposal. The MIT study attributed the variance in scores to three factors: scope (i.e., the factors’ designated material), weight (i.e., the level of importance ascribed), and measurement (i.e., the metrics and methodologies chosen). Of the three factors, measurement explained 50% of the variance, scope nearly 40%, and weight the remainder.

SGA’s ESG Scoring System

Our proprietary Portfolio Opportunity Scoring system follows our “Identify”, “Assess”, “Model,” and “Engage” framework. It factors in not only the raw E, S, and G risks and opportunities to a business, but also management’s ability to mitigate the identified risks and/or capture the opportunities.

Our ESG scores serve several important purposes in our investment process. These include 1) representing a reference point to focus our due diligence and engagement with management, 2) contributing to the determination of risk category designations and the discount rates applied in our discounted cash flow valuation system, and 3) providing input for our proprietary Portfolio Opportunity Scoring system which serves as a uniform tool to check our portfolio decisions. While integral in our research process, however, ESG scores do not dictate specific buy or sell decisions in themselves.

Assessing ESG: A Few Examples of Our Differing Opinion

Linde PLC Linde is an example of the advantage of prioritizing primary research over third party input when assessing ESG factors. As an industrial gas company, Linde’s operations result in significant consumption of electricity and production of CO2. We engaged management early on to understand this risk, starting discussions with Linde’s chief sustainability officer almost two years ago. Our work continued with engagement at the CEO and CFO level over the past year.

As a result, we gained a greater appreciation for the benefits of Linde’s use of sustainability as a framework to improve productivity, as well as the ways in which the company’s gases can be applied to reduce emissions and energy consumption for its customers. We assigned Linde a high score for environmental considerations over one year ago based on our due diligence. By contrast, MSCI upgraded Linde’s ESG score from “BBB” to “A” just last June based on a carbon mitigation strategy and efficiency programs we had already identified.

Kansas City Southern ― Our ESG scoring for railroad operator Kansas City Southern (KSU) is an example where we have a different opinion than MSCI in the assessment of certain environmental and social factors. In terms of environmental considerations, MSCI assigns a low score to KSU due in part to poor relative performance versus a peer group that includes not only freight transporters but also Uber, a company whose ride sharing business model is less comparable to KSU but whose relatively higher scores for environmental factors penalize KSU in terms of peer-based comparisons.

Conversely, MSCI scores KSU favorably on social factors in part because the company’s union hasn’t struck, a standard consideration applied broadly without nuance. It is worth noting, however, that federal law prohibits rail employees from striking due to the impact on the broader economy (instead they have mediation and arbitration rights). Thus, in this case, KSU was assigned a more favorable rating on a questionable basis. By comparison, our favorable scoring of KSU for social factors is based on the consideration of more relevant factors such as safety and community impact.

Autodesk ― Another example of where our ESG score differs significantly from MSCI is Autodesk, where we believe opportunities related to environmental factors are more material to Autodesk than does MSCI, and we thus assigned a higher ESG score. Globally, the construction industry uses three billion metric tons of natural resources to manufacture building materials every year, up to 30% of which is estimated to be wasted. As more companies shift to clean energy and circular materials, and design with the sustainability of the environment in mind, there is a considerable market opportunity for Autodesk’s innovative products such as 3D computer assisted design to help customers deliver better products, at lower cost, and with less waste.

Match Group ― A final example where our ESG scores differ from MSCI is online dating service Match Group. MSCI identifies human capital development as a key ESG issue for the company due to employee turnover and “disclosed compensation practices [that] are poor relative to peers.” This is inconsistent with our findings. For example, Match scores well above average on Glassdoor’s overall employee rating. In addition, input from former Match employees indicates that Match has solid employee morale and a positive work environment, and contradicts MSCI’s finding of a lack of non-pay benefits. MSCI’s view that initiatives to address data privacy and security appear to lag better performing peers is inconsistent with our findings as well. Match’s broader focus on user security, such as its recent partnership with the Noonlight personal safety service, leads its competition in the online dating space.

Finally, in the Customer Relations category, MSCI rates as “severe” the fact that Match faced a class-action lawsuit over alleged age discrimination in the pricing of Tinder’s subscription offerings (at one point, Tinder briefly charged users over 30 years old more for subscriptions, but no longer does so). This seems to us an overly punitive assessment of a transient and not especially harmful issue, which ultimately shouldn’t factor meaningfully into an ESG score.

In Summary

The utility of scores across ESG research and ranking firms is limited by a lack of consistent disclosure, the qualitative nature of subject matter, a lack of reporting standards, the subjectivity imbedded in formulas underlying the scores, and industry-relative assessments. Recognizing these limitations and understanding the lens through which conclusions are drawn, we selectively incorporate underlying data from service providers with proprietary fundamental analysis to create what we believe is a more meaningful and actionable ESG framework to capture material opportunities and risks of a company. We believe this results in more accurate identification of risk and reward opportunities for our clients.


1 Evan Tylenda, The PM's Guide to the ESG Revolution 2, ESG Building Blocks (Goldman Sachs Equity Research, July 28, 2020).
2 Berg, Florian and Kölbel, Julian and Rigobon, Roberto, Aggregate Confusion: The Divergence of ESG Ratings (May 17, 2020).

The opinions expressed herein reflect the opinions of Sustainable Growth Advisers, LP and are subject to change without notice. Past performance is no guarantee for future results. The securities referenced in the article are not a solicitation or recommendation to buy, sell or hold securities. The securities referenced do not represent the complete list of holdings in SGA portfolios. Upon request, free of charge, SGA can provide a list of all portfolio holdings held in SGA portfolios.