Difference-in-differences: Analysis of SME impacts

Study purpose and focus

In 2024 SQW was commissioned by the Energy Systems Catapult to carry out an econometric analysis of their impacts on business performance of supported SMEs, with a particular focus on firm growth in employment, turnover, assets and attracted investment.  

With the growing importance of the Net Zero agenda, the Energy Systems Catapult was set up to accelerate the transformation of the UK’s energy system and ensure UK businesses capture opportunities of clean growth. The Catapult works with SMEs to boost innovation in the energy sector and help bring new products and services to the market. It does this by enabling businesses to test, trial and scale their new products and services, supporting SMEs to design and evaluate their business models, by providing guidance on new energy markets, advising on strengths and weaknesses of approaches and providing tools for commercial analysis.

Methodology

To obtain robust estimates of impacts on business growth of beneficiaries we followed a difference-in-differences (DiD) methodology: we compared the beneficiaries to several complementary sets of comparison companies over time. The comparison groups included:

  • Unsupported companies from the wider business population selected using Propensity Score Matching to match the profile of beneficiaries on key characteristics, including sector, age, location and signals for high growth potential, innovation activity and operating in more modern, high-tech sectors (e.g. company being ‘tracked’ on Beauhurst; company having ‘buzzwords’ attached to it, such as CleanTech, Software-as-a-Service, Internet of Things; company having the Data City’s Real Time Industrial Classification (RTIC) score attached to it).
  • Unsuccessful applicants and recipients of light-touch support.

Findings

Overall, we find that Energy Systems Catapult appears to meet the demand from a specific type of companies at a particular stage of evolution – pushing the development of high-potential green technologies that likely require substantial further investment to commercialise.

The beneficiaries are more likely to attract external finance, and attract substantially more than what they would have been able to access without the support. On average, since the support the beneficiaries have accessed 70%-100% more external finance than they would likely have secured without it. Consequently, they are able to increase their assets above the counterfactual path predicted for them.

Although supported companies continued to grow, at this stage and from a statistical point of view, the beneficiaries did not demonstrate faster employment or turnover growth than the comparator companies. However, the interpretation of this finding is uncertain and the overall result is inconclusive. Further research is needed to help us interpret them, including gaining insights into how the support is used by the companies, what they see themselves as success and as appropriate metrics to measure it and what other factors beyond the Catapult’s support affect the outcomes.

The report has now been published by the Catapult and can be accessed here: Difference-in-differences Analysis of SME impacts.

If you would like to find out more about SQW's study, please contact Sergei Plekhanov via SPlekhanov@sqw.co.uk.