I loved the sentiment of Margherita Bacigalupo’s tweet about the Christmas-like excitement that the IEEC conference generates. Unlike Christmas though, there were no dodgy gifts, no awkward family gatherings and definitely no falling asleep in a chair after dinner. Thank you Leeds Becket for hosting, and the team behind IEEC for three brilliant days – here’s three things that I’m still thinking about after #IEEC2018…

1. Impact measurement; a trip down memory lane….

In entrepreneurship and enterprise education, impact measurement and evaluation is the debate that keeps on giving. So it was with great interest delegates attended Gabi Kaffka and Norris Krueger’s workshop on their new project about the ‘Evaluation of Entrepreneurship Education Programmes in Higher Education Institutions and Centres.’ EEEPHEIC (pronounced ‘epic’), will ‘categorise existing entrepreneurship education programmes as well as their results (impacts)’. You can get involved by completing surveys about your own entrepreneurship education programmes/approaches. At the workshop our table had a discussion about the diversity of entrepreneurship programmes and the difficulty this poses for evaluation. I shared the Realist Evaluation approach, which tries to move the evaluative question along from ‘what works?’ and towards ‘what works, for whom, in what circumstances and why?’ It prioritises recognising the multi-layered complexities of social programmes as a crucial element of its approach. The conversations reminded me of Allan Gibb’s witty and insightful keynote at IEEC 2013. He bounced  off this BIS study which looked at the impact of entrepreneurship education in further and higher education and he highlighted how the evaluation exercise is fraught with difficulties. He likened entrepreneurship education programmes to a supermarket shampoo aisle, where products might be positioned on the same shelf but promise different results: this one will fix split ends, this one will make hair shiny, this one will protect colour.

His point was that each bottle has different ingredients, and promises different and incomparable results, as such, one size fits all evaluation is problematic. This issue is one which Realist evaluators recognise, and as a result, they advise that a crucial early step in the evaluation exercise is to surface ‘programme theory’, that is, to make explicit what this particular programme is meant to be doing for participants, and then to test, refute or refine the theory. Or, as Allan asked back in 2013, what are the specific ingredients, processes and hoped for outcomes of this particular product? At the Epic workshop, our table identified that a ‘process’ to undertake such a task (rather than a measurement tool) could be useful. There was also recognition of the importance of context, as Realist Evaluation guru Ray Pawson (2013) says: “All evaluators now understand that what works in Wigan on a wet Wednesday will not necessarily work in Thurso on a thunderous Thursday.” This contextual complexity was something Phil Clegg and I attempted to (vegetable) model when we described how using Realist Evaluation had helped us think about how to innovate and evaluate an Enterprise Placement Year programme at The University of Huddersfield.

Our goal was to underscore Pawson’s point, that the contextual conditions which influence outcome patterns are infinite. We highlighted four layers – individual, inter-personal, institutional and infra-structural (Pawson, 2006), which interact and contribute to the complexity of the evaluation enterprise. And yes, we aimed to model this with onions, barbecue skewers and sliced courgette. Well, it was a Friday morning slot…

2. Social context

My brain gets snagged on social context; it’s the lens through which I recognise I often look at things. At the Wednesday night welcome reception there was a reminder about how tangible its effects can be when we heard from the founders of the Bloomin Buds Theatre Company, two social entrepreneurs who are ‘challenging the stigma of the class divide through community based theatre’ with support from the University of Leeds’ Spark programme. Their story, of feeling like ’fish out of water’ at university, and then deciding to bring theatre opportunities to students from lower and working class backgrounds stayed with me through the conference. It provided a kind of ‘What would the Bloomin Buds say?’ soundtrack to my internal conversations. The fantastic debate style workshop which asked us to argue if we should ‘Give up on Enterprise’ and focus resources on getting more students to Top 100 companies made me think ‘Which students end up at those companies?’ The thought provoking keynote about China’s rapid expansion of enterprise provision, including the 622,000 teams which took part in enterprise competitions made me ask ‘Who wins? Whose rules? Who gets to judge? Whose playing field?’ Todd Davey’s insights into the future of universities got me wondering what it would be like to ask forty of the students googling ‘university is…killing me’ to find out what they thought, what language they would use, what priorities they would have. Interest in these sorts of questions prompted my application to the social track this year, introducing the Horizon 2020 project I’m working on called Nemesis, which aims to facilitate the co-creation of social innovation projects by multi-stakeholder groups including teachers, students, social innovation practitioners, parents and community members. Nemesis is about developing alternative, social approaches to enterprising and entrepreneurial learning, indeed, in the wake of global financial meltdown, rising inequality, the gig economy and precarious working, the narrative of ‘business as usual’ feels increasingly problematic to maintain. Social innovation is seen as an alternative paradigm, but, just as with mainstream enterprise discourse, the focus can sometimes be on a heroic entrepreneur, or their clever product or service. This obscures an important element which distinguishes social innovation and enterprise – that they are defined by the way they are owned, governed and managed in democratic and participatory ways. One element of Nemesis is looking at how participatory methods help people to co-create and work together, and the workshop I ran took people through a method called OPERA (IEEC delegates at work, in the picture below), which helps all members of a group participate in a decision making or visioning process.

Every OPERA session starts with a question and schools working on the project have been using it to answer their own questions. It’s a method which, it strikes me now, would be a great way to surface possible answers to a question prompted by Martin Lackeus’ session ‘What is the egg in the hollandaise sauce?’

3. A sea turtle mistaking a plastic bag for a jelly fish

I started a PhD at Sheffield Hallam University last year and have enjoyed, as part of the research degree process, learning about the philosophies of business and management and trying to remember the difference between ontology, epistemology, axiology, and the many different theoretical perspectives which underpin research approaches. It’s the kind of mind boggling learning that feels designed to bring on an existential crisis, in a good way. It’s left me with a more critical appreciation of how philosophies operate subliminally, how we enact teaching and research in sometimes automatic ways, reproducing handed down practices and approaches without questioning their basis. The perennial conversation about measuring and/or evaluating the impact of entrepreneurship education programmes is perhaps, a good example of this. Is a research approach underpinned by a scientific philosophy – to measure and quantify phenomena, potentially through experiments where we aim to observe pre-test and post-test differences on ‘subjects’ and position ourselves as neutral and objective observers. Or is it underpinned by a constructivist philosophy, where we are engaged in a search for meaning, and use qualitative research design to prioritise individual perspectives whilst recognising that researchers are also engaged in a process of interpretation. It might seem a little abstract, but considering the deeper pre-suppositions which underlie practice, research, teaching and learning can shed light on inconsistencies and contradictions, or the illuminate how they might be better aligned. Keynote Martin Lackeus’ thesis on Value Creation as an educational philosophy, is an example of trying to make such an alignment, where theory, practice, experiential learning through value creation and assessment through LoopMe have been carefully thought through.

It is interesting to reflect on the dissonance which may be felt between practice which aims to be student-centred, co-created, relational, social and ethical…. And then when it comes to researching such programmes, our conversations seem to return to an inescapable true north, the orienting point of scientific measurement. Elements of scientific research philosophy have become ingrained in education, for example the Education Endowment Fund utilising only Randomised Control Trails to test educational interventions for lower socio-economic pupils, the framing of experimental methods as the ‘gold standard’ in entrepreneurship education (Rideout and Gray, 2013), and the goal to reduce deeply complex and multi layered social interventions and programmes to an effect size. Perhaps this is not surprising, as Jing Zhang said in her keynote, policy makers often want visible and instant results and the demand is that these results be measurable; but I was reminded of Heidi Neck’s comment last year ‘stepping on the scales won’t tell you how to lose weight.’ Perhaps a step forward would be talking about how all approaches to researching the impact of entrepreneurship education are just that – possible approaches, perspectives, philosophies, rather than taken for granted facts of life. Have a listen to Rob Newman’s brilliant Radio 4 show Total Eclipse of Descartes. He talks about the nature of educational philosophies, starting with a dark and funny metaphor: “The most destructive philosophies I believe are the ones we don’t recognise as philosophies. They damage us most when we mistake them for being how things are, simply natural; like a sea turtle mistaking a plastic bag for a jelly fish: it’s only once they’re inside us the trouble starts.” IEEC is just the place to continue such conversations, and I look forward to next year at Oxford Brookes for IEEC 2019, put the date in your diary.