News, Announcements & Press Releases

Evaluation of the Hungarian Scientific Research Fund (OTKA) published

The European Science Foundation (ESF) was commissioned by the Hungarian Scientific Research Fund (OTKA) to conduct an evaluation of OTKA in the five-year period 2009 – 2013. The scope of the evaluation included an assessment of OTKA’s governance and management structures, its funding portfolio (alignment with strategic goals, internationalisation, etc.) and the efficiency and transparency of the procedures for funding schemes. The overall goal was to identify strengths and make recommendations for further improvement to governance and management structures.

The report can be downloaded under the "files" link at the bottom of the page

The evaluation design featured a mixed-methods approach including a scoping visit, desk research and an outcomes survey of OTKA’s grant applicants (both successful and unsuccessful), which achieved a response rate of 68%. In parallel, an expert international peer review of OTKA’s funding portfolio, governance, strategy and procedures was conducted by an Evaluation Committee that was chaired by Professor Christoph Kratky, from Karl-Franzens-Universität Graz in Austria and former President of the Austrian Science Fund (FWF). It comprised five experts: Professor Jens Christian Djurhuus (University of Aarhus, Denmark), Professor Frank Hegarty (University College Dublin, Ireland), Professor Milena Horvat (Institut Jozef Stefan, Slovenia), Professor Anu Realo (University of Tartu, Estonia) and Dr Barend van der Meulen (Rathenau Instituut, Netherlands).

The conclusions from the Evaluation Committee review and the survey are overwhelmingly positive in terms of OTKA’s governance, administrative procedures and potential for an expanded role in the Hungarian research system. OTKA is a trusted, effective and valuable resource. In the words of the Evaluation Committee:

“OTKA conforms to the research council conditions of: competitive on the basis of research excellence, political independence, equal treatment of all scientific disciplines, transparency and fairness and equal chances for all applicants regardless of gender, race or position which makes it the jewel in the crown of the Hungarian research system”.

The report addresses the effectiveness of OTKA’s support of fundamental research as part of a complex national research support system and must be read in the context of a research environment that is rapidly being aligned to European norms.

The findings are presented in the report and summarised in its Executive Summary. The report concludes with a set of recommendations addressing: (i) OTKA mission, governance and management structures, (ii) funding portfolio, (ii) internationalisation and (iv) procedures.

The report was presented to OTKA in November 2014. In the words of Professor László Kollár, OTKA President: 

“The Hungarian Scientific Research Fund (OTKA) is pleased to acknowledge the excellent evaluation of its operation by the European Science Foundation (ESF). We are confident that the resulting outstanding report Organisational Evaluation of the Hungarian Scientific Research Fund will serve as an important point of reference to The National Office for Research, Development and Innovation that is taking over OTKA’s functions.” (The Hungarian Parliament passed a law on November 25, 2014, terminating the OTKA Office with legal successor and overruling the previous OTKA law as of December 31, 2014. Given this legal regulation, the OTKA programs, the administration of the running projects, and the new calls for proposals will be handled by the National Office for Research, Development, and Innovation).

For further information, contact Ms. Siobhan Philips, Head of the ESF evaluation services (evaluation[at]esf.org)

ESF provides a range of evidence-based evaluation services from framework design, through the management, support, and implementation of ex-ante, formative and ex-post evaluation. We will coordinate and supply the necessary subject matter and evaluation expertise across all science domains and conduct independent evaluations drawing on international expertise at the level of national research and innovation systems, organisations and programmes.
www.esf.org/evaluation