As an evaluator, consultant and economic development professional I see a lot of good and a lot of bad in the development world. In evaluator-talk, this means I see interventions that are working well and achieving sustainable development results, as well as those that aren’t. As an optimist, I like to think that we identify and learn from lessons from what has worked well, and (just as important) what has not, and that these lessons can inform efforts at continuous improvements. Why re-invent the wheel if others have been there and done that?
Globally, small business development and entrepreneurship is recognised as a key driving force responsible for accelerating economic growth, job creation, and poverty reduction. It’s all about jobs and our politicians boldly commit to job targets. But do they know what is involved and what is needed to achieve these targets?
Obtaining rigorous evidence on what is working well, and what is not, to effectively promote entrepreneurship and business development has become increasingly critical to governments and funders as well as to development practitioners. The field of evaluation holds much promise in providing rigorous evidence on the impacts of initiatives aimed at promoting entrepreneurship and business development.
So I decided to conduct a meta-review of four systematic reviews (which in turn examined over 100 evaluations in developing country contexts) focusing on the effectiveness of small business support interventions to find out what has been learnt. For those of you interested in the methodological details of this meta-review check out my journal article in the International Journal of Entrepreneurship in Developing Countries: View article here
First, let’s quickly unpack two key terms:
- A systematic review is “A review of the research literature using systematic and explicit accountable methods” (Gough, Oliver, & Thomas, 2012a).
- A meta-review is a review ‘of’ or ‘about’ other reviews (sometimes also referred to as umbrella reviews or reviews of reviews) (Gough et al., 2012a; Thomson, Russell, Becker, Klassen, & Hartling, 2011).
Systematic reviews differ from traditional literature reviews in that they employ a systematic methodology for finding relevant literature, for including or excluding certain literature, and for synthesising and reporting the results (Gough et al, 2012a). Systematic reviews bring together all existing research studies focused on a specific question or intervention as a shortcut to the literature.
There is a growing body of international and local evidence on what is working (as well as what is not working well) to support entrepreneurship, and enterprise (For example see: Attanasio et al, 2012; Augsburg et al, 2012; Banerjee, Duflo, Glennerster, and Kinnan et al, 2009; Cho and Honorati, 2013; Grimm and Paffhausen, 2014; Karlan and Zinman. 2010 and 2011; Crépon, Devoto, Duflo, & Parienté, 2011; Mckenzie and Woodruff, 2012; Zandniapour, Sebstad, &, Snodgrass 2004).
My meta review focused on these four systematic reviews:
I reviewed the findings from evaluations focusing on the following five types of business support interventions and themes:
- Access to finance.
- Entrepreneurship training.
- Business development services including incentives and wage subsidies.
- Business environment improvements and formalisation of informal businesses
- Findings specific to target groups (youth and women).
Access to finance.
- Access to finance interventions have not been found to have much impact on employment for existing small and micro businesses but larger employment impacts have been achieved with both the creation of new (mostly micro) enterprises and the expansion of already larger, well-established and profitable firms.
- There is a long results chain from provision of finance, to how businesses use that finance and the time needed to potentially impact on employment.
- Finance interventions targeting women have been found to be less successful at creating employment. One interpretation is that women face additional constraints to be overcome when running businesses.
- The details of the loan contracts matter. Short repayment periods, which over the loan period translate into lower outstanding loans and shorter maturities, prevent poor entrepreneurs from investing since they fear not being able to repay on time.
- The beneficial impacts of training have been found to include higher investment, process or product innovations, and/ or growth in sales and revenue. Very few studies report impacts on profit and/or employment levels. Even non-existent or negative employment effects can be good news about trainings’ effectiveness as entrepreneurship training was found to help non-profitable firms either to become profitable or to close down. Likewise, training can prevent non-profitable business ideas from being started.
- Training may enable less analytically able and poorer individuals to start businesses, and may prop up the survivorship of less profitable businesses. Training therefore appears to have some success in generating short-run impacts on business start-up. However, this does not necessarily increase employment, since it may come from people switching from wage work.
- The length of training provided is an important issue influencing its impact, with a U-shaped relationship being found, suggesting that either intensive, short training or substantially extended training is best- however the optimal solution depends on the goals of such interventions. It appears that entrepreneurship training needs to be substantial in order to be effective, where substantial means that the training runs over an entire year with at least one training session per week
- The most successful training is tailor made to address specific knowledge gaps.
- There were mixed results as to whether providing a package of training together with financial support is more effective than just providing training.
- Most of the studies take a single snapshot of the impact of the training at a relatively short interval after training has ended. Two studies which have traced trajectories of impacts suggest that effects can indeed vary a lot over time. In de Mel et al., (2012) the impacts on business start-up fade with time, as control firms catch up. Bloom et al., (2012) find that introducing management practices in larger firms shows immediate effects on quality, then slowly leads to changes in inventory levels, output and productivity, and it is only after several years of using these practices that impacts start to show in terms of employment generation (through new plants being opened) (Grimm & Paffhausen , 2014).
- The evidence is mixed on whether trainings’ return is higher for those with initially lower The review suggests, however, that entrepreneurship training is more helpful for start-ups than for business expansion.
Business development services including incentives and wage subsidies.
- With respect to wage subsidies, a demand-driven design where employers receive wage subsidies, as opposed to employees receiving vouchers, appears to work best at creating sustainable employment. In some cases, a supply driven approach where workers receive the subsidy in the form of a voucher was found to have a negative impact on the employment of youth beneficiaries and only a few beneficiaries were retained in their job once the subsidy came to an end.
- There are a range of possible reasons for this including that there may be a better fit between employers and employees when employers choose the workers they hire (as opposed being forced to hire a worker).
- Firm conclusions regarding the impacts of incentives are difficult due to participant self-selection of firms into such programs (i.e. that have a higher chance of succeeding that the general population). Nevertheless, tax breaks and fiscal incentives which are conditional on process and/or product innovations seem to be particularly effective (however the sample of studies is quite small and this finding must be interpreted with caution).
- Business services were found to impact positively on enterprise sales, revenues, net profits or client income appear in 10 out of 15 evaluations that included these variables. However, the extent to which sales or revenues increased varied substantially across these studies and across enterprises within these studies. The findings also seem to suggest that relatively larger firms gain more in terms of increase in sales, profits or income compared to smaller, more micro firms. On the other hand, smaller firms generally attribute more of the increase in their sales/revenues and profits to their participation in the program compared to larger firms.
Business environment improvements and formalisation of informal businesses:
- Programs that “force” firms to formalize are unlikely to produce any significant employment effects as for many formerly informal firms formality does not translate into extra profits but into additional cost. In general, it seems easier to formalise firms while they are set up than formalising firms that already exist.
- Programs that offer cheaper and easier formalization procedures are more likely to have success but only for a relatively small group of entrepreneurs and firms that already show a higher initial performance.
- For many informal firms, it is not the costs of registration but the expected benefits of formality that is pivotal for their decision to So, the best incentive governments can provide for formalization is to offer useful public services in return and in addition to simplifying administrative procedures
Findings specific to target groups (youth and women):
- Youth targeted programs have shown good successes. For nearly all outcomes, particularly labour market activities and business performance, youth is highly associated with program success.
- Women face additional constraints which need to be addressed if support programmes are going to be successful. For women, the impacts from microcredit interventions seem higher than training programs. Women are generally more severely credit constrained, and this in turn can hamper their potential gains from skills training. Financing support performs better for women throughout all outcomes. Women are not associated with any large and significant impacts other than the outcome of attitudes, indicating that entrepreneurship programs seem useful for female empowerment but may not be sufficient to address various barriers faced by women.
Summing this all up: many conditions have to be met before interventions in favour of individual enterprises do not only improve business performance but also lead to additional jobs. The results chain between enhanced knowledge and skills, and business investment, profitability, and growth is a long and complex one and long time periods are needed to track changes in outcomes over time to better understand the dynamics impacting on these results chains. Not all potential and actual entrepreneurs can make good use of support. Different types of interventions will be required to increase employment for different groups. Interventions also need to pay attention to the interaction between different binding constraints. For instance, just improving business skills without facilitating access to capital (and vice versa), might often not be enough to have an effect on investment and employment
While these lessons may have some value, they do not tell us too much about the why and how, about the change processes, mechanisms and specific contexts within which certain interventions work or don’t work and for whom.
The insufficient utilisation of theory of change approaches (in the underlying evaluations) limited the analysis and findings in terms of a better understanding what works for who, where, when and why. The use of the realist evaluation theory of change approach (see Pawson & Tilley 1997, Pawson, 2004; Pawson 2006; Pawson 2013) in future evaluations could contribute towards a more detailed understanding of why certain interventions do or do not contribute towards achieving certain results or outcomes. Realist evaluation “Makes explicit the underlying theories or assumptions about how an intervention is supposed to work and evaluates theories against the available evidence.” (Pawson et al., 2005). A realist review focuses on understanding and unpacking the mechanisms by which an intervention works (or fails to work), thereby providing an explanation, as opposed to a judgment, about how it works. “Mechanisms” are understood to be a combination of resources offered by an intervention and stakeholders’ reasoning in response. Mechanisms can also be understood as “underlying entities, processes, or structures which operate in particular contexts to generate outcomes of interest.” (Astbury & Leeuw, 2010) or the process of how subjects interpret and act upon the intervention. Mechanisms are usually hidden (not visible), are sensitive to variations in context, and are responsible for generating outcomes (intended or unintended, positive or negative).
Do you agree with these lessons given your own experiences?
What lessons have you learnt about what works with entrepreneurship and small business support?
Please share your views and comments below:
Future blogs will focus on learnings from evaluating key economic development interventions to accelerate job creation, including cluster initiatives and value chain upgrading, business incubators, and red tape reduction.
Astbury, B., & Leeuw, F. L. (2010). Unpacking black boxes: mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363-381.Bloom, N., Eifert, B., Mahajan, A., McKenzie, D., and Roberts J. (2011), “Does management matter? Evidence from India”, working paper 16658, National Bureau of Economic Research,Cambridge, Massachusetts.
Cho, Y., and Honorati, M. (2013), “Entrepreneurship programs in developing countries: a meta-regression analysis.”, working paper 6402, World Bank Policy Research, Washington, DC, April 2013.
De Mel, S., McKenzie, D., and Woodruff, C. (2012), “One-time transfers of cash or capital have long-lasting effects on microenterprises in Sri Lanka”, Science, 335 No. 6071, 962-966.
Gough, D., Oliver, S., and Thomas, J. (2012a), Introduction to Systematic Reviews, Sage Publications: London.
Gough, D. Thomas, J., and Oliver, S. (2012b), “Clarifying differences between review designs and methods.”, Systematic Reviews, 1 No. 28, 1-9.
Greenhalgh, T. (1997), “How to read a paper: "Papers that summarize other papers (systematic review and meta analyses).”, British Medical Journal, 315 (7109), 672-675.
Grimm, M., & Paffhausen, A. L. (2013). Interventions for employment creation in micro, small and medium-sized enterprises in low-and middle-income countries.
Lavis, J. N. (2009). How can we support the use of systematic reviews in policymaking?. PLoS Med, 6(11).
Mckenzie, D., and Woodruff, C. (2012), “What are we learning from business training and entrepreneurship evaluations around the developing world?”, working paper 6202, World Bank. Policy Research, Washington D.C.
Pawson. R., and Tilley, N. (1997), Realist Evaluation, Sage Publications: London.
Pawson. R. (2004), Realist Synthesis: An Introduction, University of Leeds, Leeds..
Pawson, R., Greenhalgh, T., Harvey, G., and Walshe, K. (2005), “Realist review: A new method of systematic review designed for complex policy interventions”, Journal of Health Services Research and Policy, 10 No. 1, 21-34.
Pawson, R. (2006). Evidence-Based Policy: A Realist Perspective, Sage Publications, London.
Pawson, R. (2013). The Science of Evaluation: A Realist Manifesto, Sage Publications, London.
Thomson, D., Russell, K,. Becker, L., Klassen, T., and Hartling, L. (2011), “The evolution of a new publication type: Steps and challenges of producing overviews of reviews.”, Research Synthesis Methods, 1 No. 3-4, 198-211.
Zandniapour, L., Sebstad, J., & Snodgrass, D. (2004). Review of evaluations of selected enterprise development projects. Microenterprise report, 3, Washington D.C.