The Difference Between Innovation-Led Evidence And Evidence-Led Innovation

by Conrad Wolfram, Computer-Based Math

Ed note: This post has been republished from previous posting.

Recently, I was part of a high-level discussion about maths and computer science education, how we could improve their reach and effectiveness.

Rather quickly the question of evidence came up, and its role in driving innovation. It’s taken me a few days to realize that there were actually two very different “importance of evidence” conversations–one with which I completely concur, and one with which I vehemently disagree.

In the end, what I believe this exposes is a failure of many in charge of education to understand how major innovation usually happens–whether innovation in science, technology, business or education–and how “evidence” can drive effective innovation rather than stifle it. In an age of massive real-world change, the correct and rapid reflection of this in education is crucial to future curricula, their effective deployment, and achieving optimization for the right educational outcomes.

I’m going to call the two evidence utilizations ‘innovation-led evidence’ and ‘evidence-led innovation.’ The difference is whether you build your ‘product’ (e.g., phone, drug, curriculum) first, then test it (using those tests for iterative refinement or rejection) or whether formal evidence that exists from previous products becomes the arbiter of any new products you build.

The former–‘innovation-led evidence’–is highly productive in achieving outcomes, though of course care must be taken that those outcomes represent your objectives effectively. The latter–‘evidence-led innovation’ almost by definition excludes fundamental innovation because it means only building stuff that past evidence said would work.

When you build something significantly new it isn’t just a matter of formally assembling evidence from the past in a predictable way. A leap is needed, or several. Different insights. A new viewpoint. Often in practice these will occur from a mixture of observation, experience and what still appears to be very human-style intelligence. But wherever it comes from, it isn’t straightforwardly ‘evidence-led.’

I strongly agree with the late physicist (and friend of my brother’s) Richard Feynman who explained nicely in one of his famous 1960s Caltech lectures how the scientific process works. I could summarize: Guess, Make a theory, Test it and compare with theory. (Film of this lecture exists–see the first minute!)

In the case of technology, ‘theory’ is the product, in pharmaceuticals it’s the drug and in education (for the most part) it’s the curriculum.

‘Evidence-led innovation’ stifles major innovation–it locks out the guess–yet I firmly believe that that’s what most of “evidence-led education” is talking about with painfully little “innovation-led evidence” applied.

I’ve faced this repeatedly with Computer-Based Maths. I’m asked “Do you have evidence it works”? I sometimes answer, “Where’s your evidence that today’s traditional maths education works? Have you done randomized control trials?”

As quickly as we can build curricula, fund their development and set up projects in different countries, we are starting to gather evidence. Something that slows this down is the need to have student assessments that accurately reflect required outcomes: it’s not just a matter of comparing exam results before and after, open-ended computer-based maths assessments are needed too.

One problem with the ‘evidence-led innovation’ crowd is that they often have no idea how hard it is to build something completely new. They think you can do micro-innovations, then test, then micro-innovate then test.

Actually, so far CBM is the hardest innovation I’ve been involved in. It’s been amazing to me just how different every aspect of the maths curriculum becomes when you do not need to assume hand-calculating. Equally amazing is how deep everyone needs to dig into their own understanding to uncover those differences, particularly since those involved have learnt maths traditionally.

You might ask whether now is the time for a new maths curriculum? Can we really take the risk? As guesses go, the idea that maths education should be the same subject as maths in the real world (ie. using mechanised computation) and not the current hand-calculating proxy is an extremely sure-footed one. The risk of not leaping with the real-world poses a very significant danger.

Let’s have the courage to develop and test CBM around the world, indeed more thoroughly than any maths curriculum has been tested before.

About Conrad Wolfram
‘Conrad Wolfram, physicist, mathematician and technologist, is strategic director and European co-founder/CEO of the Wolfram group of companies. Described as the place where “Computation meets Knowledge” and “The Math Company”, the Wolfram Group specializes in pushing boundaries at the intersection of computation, data science, maths, and knowledge, including making Mathematica software, the Wolfram|Alpha knowledge engine (powering knowledge answers for Apple’s Siri), the Computable Document Format (CDF) standard and Wolfram Language. All Wolfram technology is now being hybridised to the cloud, including providing private computation cloud installations. Wolfram is now established as a key organisation in modern data analytics, uniquely spanning broad-based technology and consulting solutions to enable sectors from medical to finance to government to maximise decision-making from data, way beyond traditional solutions. Conrad is also a leading advocate for a fundamental shift of maths education to be computer-based and founded (CBM) to drive implementation of the change. The movement is now a worldwide force in re-engineering the STEM curriculum with early projects in Estonia, Sweden and across Africa. Conrad was educated at Eton College, and University of Cambridge in the UK, from which he holds degrees in natural sciences and maths.’

*previously published at


Submit a Comment

Your email address will not be published. Required fields are marked *

The following GDPR rules must be read and accepted:
This form collects your name, email and content so that we can keep track of the comments placed on the website. For more info check our privacy policy where you will get more info on where, how and why we store your data.

This site uses Akismet to reduce spam. Learn how your comment data is processed.