News

Assessment for basic research needs fine tuning, policy experts say

Assessing the impact of basic science in terms of economic returns can be a futile and self-defeating exercise, according to economist Wolfgang Polt of the Joanneum Research Institute (Vienna).

Speaking at Science Impact, a joint conference hosted by the European Science Foundation (ESF) and the Austrian Science Fund (FWF) in May, Professor Polt issued a stark warning: "Normally we would expect an economist working in the field to say that more research, or even more basic research is needed, but I would say, should we really do a cost benefit analysis when we don’t know the costs, and we don’t know the benefits?"

"I think, what I can offer to the debate is a warning and a recommendation to fence off overly-simplistic approaches to quantifying ranking of research areas, technologies and projects," added Polt, addressing an audience of leading economists, science sociologists and historians in Vienna, May 12th.

Ben Martin of the Science and Technology Policy Research centre (Sussex), in turn, detailed the problems associated with econometric studies, which he felt were too often "used to derive heroic or sometimes simplistic assumptions about the nature of innovation".

He cited several problems with this approach ranging from the issue of causality (the inability to attribute with certainty benefits to specific causes); the issue of timescale (the risk of “over-emphasising research that leads to short term benefits” when measurements are framed too narrowly: after all, the impacts of some types of research are only seen after decades); and the issue of internationalisation of research in a globalised world.

"Similar research going on countries A, B, and C for example might lead to a technological development in A, subsequently leading to innovation in A, also drawing on technological developments from country C, all the while benefiting all three countries in terms of socio-economic impacts," he said.

Other, qualitative forms of assessment such as industrial surveys, and case studies which trace historical inputs to innovations have their place, Martin indicated, but are also limited. Surveys sent to industry to ask about how basic research affects them tend to focus only on large firms, who show bias towards internal activities of their own companies. Historical studies, on the other hand, are only so useful when it comes to moving beyond specific examples into generalised axioms.

Added to this, the tendency to over-emphasise 'codified' i.e. newly discovered knowledge, at the expense of 'tacit' knowledge (as embodied in the skilled workforce, who will implement innovations, for example), makes matters more difficult still.

Wolfgang Polt conceded, however, that "If such [case] studies are undertaken ex-post, they can be very good illustrative examples of different channels that impacts can take"” but added, "this is the remit of historians of technology and the sociologists of science, and not the sole remit of the economists."

A recent study commissioned by the American Competitiveness Institute on the development of the ubiquitous iPod, he noted, illustrated well the circuitous pathways research can take from basic to applied form, as well as the lengthy time-scales involved.

The United States Department of Defence, Department of Energy, National Institute for Health, the National Science Foundation, the Defence Advanced Research Projects Agency, the Army Research Office had all contributed technology over the 60s, 70s, 80s, and 90s that, when combined, led to the development of it iPod.

"It helps us to understand how different pieces of knowledge add up and we see the complementarity of different types of knowledge – basic, applied, tacit, codified. In the end we cannot really separate what the most decisive impact is. If you would now compare the profits from iPod sales to the DoD funds into research, of course this would be an enormous multiplier."

Asked about the future of impact assessment modelling, and whether we are any close to a catch-all formula for rates of return, Martin said: "We may never be at a stage where we have an all-encompassing model…[because] that would involve a great deal of foresight and indeed predicting the future." 

He added that this should not deter governments from funding research because of the spectacular innovations society would subsequently miss out on, outlining his own methodology for assessing impact based on looking at the overall stock of useful knowledge; the construction of knowledge 'networks' (i.e. between researchers) and stimulating social interaction; increasing the capacity for technological problem-solving; creating new firms and thereby stimulating industry; and the provision of social knowledge that in turn leads to on-the-ground benefits.

---

Media contact:

Mr. Issam AhmedE-Mail