Skip to main content

As an evaluator, consultant and economic development professional I see a lot of good and a lot of bad in the development world. In evaluator-talk, this means I see interventions that are working well and achieving sustainable development results, as well as those that aren’t. As an optimist, I like to think that we identify and learn from lessons from what has worked well, and (just as important) what has not, and that these lessons can inform efforts at continuous improvements. Why re-invent the wheel if others have been there and done that?

Globally,  small  business  development  and  entrepreneurship  is  recognised  as  a  key driving  force  responsible  for  accelerating  economic  growth,  job  creation,  and  poverty reduction. It’s all about jobs and our politicians boldly commit to job targets. But do they know what is involved and what is needed to achieve these targets?

rural

Obtaining rigorous evidence on what is working well, and what is not, to effectively promote entrepreneurship and business development has become increasingly critical to governments and funders as well as to development practitioners. The field of evaluation holds much promise in providing rigorous evidence on the impacts of initiatives aimed at promoting entrepreneurship and business development.

So I decided to conduct a meta-review of four systematic reviews (which in turn examined over 100 evaluations in developing country contexts) focusing on the effectiveness of small business support interventions to find out what has been learnt. For those of you interested in the methodological details of this meta-review check out my journal article in the International Journal of Entrepreneurship in Developing Countries: View article here

First, let’s quickly unpack two key terms:

  • A systematic review is “A review of the research literature using systematic and explicit accountable methods” (Gough, Oliver, & Thomas, 2012a).
  • A meta-review is a review ‘of’ or ‘about’ other reviews (sometimes also referred to as umbrella reviews or reviews of reviews) (Gough et al., 2012a; Thomson, Russell, Becker, Klassen, & Hartling, 2011).

Systematic  reviews  differ  from  traditional  literature  reviews  in  that  they  employ  a systematic methodology for finding relevant literature, for including or excluding certain literature,  and  for  synthesising  and  reporting  the  results  (Gough et al, 2012a).  Systematic  reviews  bring  together  all  existing  research studies  focused  on  a  specific  question  or  intervention  as  a  shortcut  to  the  literature. kzn_art

There is a growing body of  international and local evidence on what is working   (as  well  as  what  is  not  working  well)  to  support  entrepreneurship,  and enterprise (For example see: Attanasio et al, 2012; Augsburg et al, 2012; Banerjee, Duflo, Glennerster, and Kinnan et al, 2009; Cho and Honorati, 2013; Grimm and Paffhausen, 2014; Karlan and Zinman. 2010 and 2011; Crépon, Devoto, Duflo, & Parienté, 2011; Mckenzie and Woodruff, 2012; Zandniapour, Sebstad, &, Snodgrass 2004).

My meta review focused on these four systematic reviews:

Cho. And Honorati. 2013. Entrepreneurship programs in developing countries: a meta-regression analysis

Grimm and Paffhausen. 2014. Interventions for employment creation in micro, small and medium-sized enterprises in low-and middle-income countries.

Mckenzie and Woodruff. 2012. What are we learning from business training an entrepreneurship evaluations around the developing world?

Zandniapour, et al., 2004. Review of evaluations selected enterprise development projects.

I reviewed the findings from evaluations focusing on the following five types of business support interventions and themes:

  1. Access to finance.
  2. Entrepreneurship training.
  3. Business development services including incentives and wage subsidies.
  4. Business environment improvements and formalisation of informal businesses
  5. Findings specific to target groups (youth and women).

Lessons Learnt

Access to finance.

  1. Access to finance interventions have not been found to have much impact on employment for existing small and micro businesses but larger employment impacts have been achieved with both the creation of new (mostly micro) enterprises and the expansion of already larger, well-established and profitable firms.
  2. There is a long results chain from provision of finance, to how businesses use that finance and the time needed to potentially impact on employment.
  3. Finance interventions targeting women have been found to be less successful at creating employment. One interpretation is that women face additional constraints to be overcome when running businesses.
  4. The details of the loan contracts matter. Short repayment periods, which over the loan period translate into lower outstanding loans and shorter maturities, prevent poor entrepreneurs from investing since they fear not being able to repay on time.

Developing Diverse Economies

Entrepreneurship training.

  1. The beneficial impacts of training have been found to include higher investment, process or product innovations, and/ or growth in sales and revenue. Very few studies report impacts on profit and/or employment levels. Even   non-existent   or   negative   employment   effects   can   be   good   news   about   trainings’ effectiveness  as  entrepreneurship  training  was  found  to  help  non-profitable  firms  either  to become profitable or to close down. Likewise, training can prevent non-profitable business ideas from being started.
  2. Training may enable less analytically able and poorer  individuals  to  start  businesses,  and  may  prop  up  the  survivorship  of  less  profitable businesses. Training therefore appears to have some success in generating short-run impacts on business start-up. However, this does not necessarily increase employment, since it may come from people switching from  wage work.
  3. The length of training provided is an important issue influencing its impact, with a U-shaped relationship being found, suggesting that either intensive, short training or substantially extended training is best- however the optimal solution depends on the goals of such interventions. It appears   that entrepreneurship training needs to be substantial in order to be effective, where substantial means that the training runs over an entire year with at least one training session per week
  4. The most successful training is tailor made to address specific knowledge gaps.
  5. There were mixed results as to whether providing a package of training together with financial support is more effective than just providing training.
  6. Most of the studies take a single snapshot of the impact of the training at a relatively short interval after training has ended. Two studies which have traced trajectories of impacts suggest that effects can indeed vary a lot over time. In de Mel et al., (2012) the impacts on business start-up fade with time, as control firms catch up. Bloom et al., (2012) find that introducing management practices in larger firms shows immediate effects on quality, then slowly leads to changes in inventory levels, output and productivity, and it is only after several years of using these practices that impacts start to show in terms of employment generation (through new plants being opened) (Grimm & Paffhausen , 2014).
  7. The evidence  is  mixed  on whether  trainings’  return  is  higher  for  those  with  initially  lower    The  review  suggests, however,   that   entrepreneurship   training   is   more   helpful   for   start-ups   than   for   business expansion.

Business development services including incentives and wage subsidies.

  1. With respect to wage subsidies, a demand-driven design where employers receive wage subsidies, as opposed to employees receiving vouchers, appears to work best at creating sustainable employment. In some cases, a supply driven approach where workers receive the subsidy in the form of a voucher was found to have a negative impact on the employment of youth beneficiaries and only a few beneficiaries were retained in their job once the subsidy came to an end.
  2. There are a range of possible reasons for this including that there may be a better fit between employers and employees when employers choose the workers they hire (as opposed being forced to hire a worker).
  3. Firm conclusions regarding the impacts of incentives are difficult due to participant self-selection of firms into such programs (i.e. that have a higher chance of succeeding that the general population). Nevertheless, tax breaks and fiscal incentives which are conditional on process and/or product innovations seem to be particularly effective (however the sample of studies is quite small and this finding must be interpreted with caution).
  4. Business services were found to impact positively on enterprise sales, revenues, net profits or client income appear in 10 out of 15 evaluations that included these variables. However, the extent to which sales or revenues increased varied substantially across these studies and across enterprises within these studies. The findings also seem to suggest that relatively larger firms gain more in terms of increase in sales, profits or income compared to smaller, more micro firms. On the other hand, smaller firms generally attribute more of the increase in their sales/revenues and profits to their participation in the program compared to larger firms.

Business environment improvements and formalisation of informal businesses:

  1. Programs that  “force”  firms  to  formalize  are  unlikely  to  produce  any  significant  employment effects  as for many  formerly  informal firms formality  does  not  translate  into extra profits but into additional cost. In general, it seems easier to  formalise  firms while  they  are  set  up than formalising firms that already exist.
  2. Programs that offer cheaper and easier formalization procedures are more likely to have success but only for a relatively small group of entrepreneurs and firms that already show a higher initial performance.
  3. For many informal  firms,  it  is  not  the  costs  of  registration  but  the  expected  benefits  of formality  that  is  pivotal  for  their  decision  to    So, the  best  incentive governments can provide for formalization is to offer useful public services in return and in addition to simplifying administrative procedures

Findings specific to target groups (youth and women):

  1. Youth targeted programs have shown good successes. For nearly all outcomes, particularly labour market activities and business performance, youth is highly associated with program success.
  2. Women face additional constraints which need to be addressed if support programmes are going to be successful. For women, the impacts  from microcredit interventions seem higher than training programs. Women are generally more  severely credit constrained, and this in turn can hamper their potential gains from skills training.  Financing support performs better for women throughout  all outcomes. Women are not associated with any large and significant impacts other than the outcome of attitudes, indicating that entrepreneurship programs seem useful for female empowerment but may not be sufficient to address various barriers faced by women.

Summing this all up: many conditions have to be met before interventions in favour of individual enterprises do not only improve business performance but also lead to additional jobs. The results chain between enhanced knowledge and skills, and business investment, profitability, and growth is a long and complex one and long time periods are needed to track changes in outcomes over time to better understand the dynamics impacting on these results chains. Not all potential and actual entrepreneurs can make good use of support. Different types of interventions will be required to increase  employment for different groups. Interventions also need to pay attention to the interaction  between different binding constraints. For instance, just improving business skills without facilitating  access to capital (and vice versa), might often not be enough to have an effect on investment and employment

While these lessons may have some value, they do not tell us too much about the why and how, about the change processes, mechanisms and specific contexts within which certain interventions work or don’t work and for whom.

The insufficient utilisation of theory of change approaches (in the underlying evaluations)  limited the analysis and findings in terms of a better understanding what works for who, where, when and why.  The use of the realist evaluation theory of change approach (see Pawson & Tilley 1997, Pawson, 2004; Pawson 2006; Pawson 2013) in future evaluations could contribute towards a more detailed understanding of why certain interventions do or do not contribute towards achieving certain results or outcomes.  Realist evaluation “Makes explicit the underlying theories or assumptions about how an intervention is supposed to work and evaluates theories against the available evidence.” (Pawson et al., 2005).  A realist review focuses on understanding and unpacking the mechanisms by which an intervention works (or fails to work), thereby providing an explanation, as opposed to a judgment, about how it works. “Mechanisms” are understood to be a combination of resources offered by an intervention and stakeholders’ reasoning in response.  Mechanisms can also be understood as “underlying entities, processes, or structures which operate in particular contexts to generate outcomes of interest.” (Astbury & Leeuw, 2010) or the process of how subjects interpret and act upon the intervention. Mechanisms are usually hidden (not visible), are sensitive to variations in context, and are responsible for generating outcomes (intended or unintended, positive or negative).

Do you agree with these lessons given your own experiences?

What lessons have you learnt about what works with entrepreneurship and small business support?

Please share your views and comments below:

Future blogs will focus on learnings from evaluating key economic development interventions to accelerate job creation, including cluster initiatives and value chain upgrading, business incubators, and red tape reduction.

 

References:

Astbury, B., & Leeuw, F. L. (2010). Unpacking black boxes: mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363-381.Bloom, N., Eifert, B., Mahajan, A., McKenzie, D., and Roberts J. (2011), “Does management matter? Evidence from India”, working paper 16658, National Bureau of Economic Research,Cambridge, Massachusetts.

Cho, Y., and Honorati, M. (2013), “Entrepreneurship programs in developing countries: a meta-regression analysis.”, working paper 6402, World Bank Policy Research, Washington, DC, April 2013.

De Mel, S., McKenzie, D., and Woodruff, C. (2012), “One-time transfers of cash or capital have long-lasting effects on microenterprises in Sri Lanka”, Science, 335 No. 6071, 962-966.

Gough, D., Oliver, S., and Thomas, J. (2012a), Introduction to Systematic Reviews, Sage Publications: London.

Gough, D. Thomas, J., and Oliver, S. (2012b), “Clarifying differences between review designs and methods.”, Systematic Reviews, 1 No. 28, 1-9.

Greenhalgh, T. (1997), “How to read a paper: "Papers that summarize other papers (systematic review and meta analyses).”, British Medical Journal, 315 (7109), 672-675.

Grimm, M., & Paffhausen, A. L. (2013). Interventions for employment creation in micro, small and medium-sized enterprises in low-and middle-income countries.

Lavis, J. N. (2009). How can we support the use of systematic reviews in policymaking?. PLoS Med, 6(11).

Mckenzie, D., and Woodruff, C. (2012), “What are we learning from business training and entrepreneurship evaluations around the developing world?”, working paper 6202, World Bank. Policy Research, Washington D.C.

Pawson. R., and Tilley, N. (1997), Realist Evaluation, Sage Publications: London.

Pawson. R. (2004), Realist Synthesis: An Introduction, University of Leeds, Leeds..

Pawson, R., Greenhalgh, T., Harvey, G., and Walshe, K. (2005), “Realist review: A new method of systematic review designed for complex policy interventions”, Journal of Health Services Research and Policy, 10 No. 1, 21-34.

Pawson, R. (2006). Evidence-Based Policy: A Realist Perspective, Sage Publications, London.

Pawson, R. (2013). The Science of Evaluation: A Realist Manifesto, Sage Publications, London.

Thomson, D., Russell, K,. Becker, L., Klassen, T., and Hartling, L. (2011), “The evolution of a new publication type: Steps and challenges of producing overviews of reviews.”, Research Synthesis Methods, 1 No. 3-4, 198-211.

Zandniapour, L., Sebstad, J., & Snodgrass, D. (2004). Review of evaluations of selected enterprise development projects. Microenterprise report, 3, Washington D.C.

Leave a Reply