Measuring Innovation in Education 2023

9. A survey on educational R&D: a proposed international standard for questionnaire and methodology

Stéphan Vincent-Lancrin
OECD

Abstract

Countries routinely collect information about their investment and operation in research and development, a key indicator of countries’ ability to innovate. Adapting the cross-sectorial approach promoted by the Frascati Manual, this chapter presents a survey instrument (questionnaire) and method to collect information about countries’ educational research and development. The questionnaire, presented in Annex, can be used (and adapted) by jurisdictions interested in knowing how much is spend on educational research and development, by whom, and on what broad topics.

This work has benefited from the inputs of many former OECD colleagues, notably Francisco Martinez as an intern, Richard Scott and Nicolas Jonas, as well as from country delegates.

Introduction

Most governments collect some statistics about their investment in educational research and development (R&D) as part of their general statistics on R&D, which allow to compare their spending for educational research and for other socio-economic objectives. Education policy makers interested in driving their educational research agenda would need more granular and accurate information to better understand who performs and funds educational research within their system, and whether educational R&D focuses on their policy priorities.

Given limited budgets for educational R&D, governments need to invest them in a smart way and steer the system based on a better understanding of the educational R&D that is already performed or funded. For example, is enough knowledge generated to improve the quality of tertiary education and early childhood education compared to primary and secondary education? Is there enough educational research on teaching and learning to support practices in the classroom? Depending on what foundations and the higher education sector already fund or work on, what would it be more strategic or useful for governments to support? In the current state of affair, most education ministries and research agencies have very little information to take informed decisions. Therefore collecting specific information that can guide their decisions and inform them on how much their system in the generation and application of new knowledge is important.

This chapter proposes an exemplar questionnaire and methodology to collect information about the performance and funding of educational R&D within a country or education system – or internationally. Countries or regions could adapt it to their context or interest, or implement it as it is. Annex 9.A presents the model questionnaire. It follows the guidelines and definitions of the Frascati Manual (OECD, 2002[1]; OECD, 2015[2]), but mainly focuses on the public funding and performance of educational R&D and proposes questions that are customised and specific to the education sector.

The rest of the chapter is organised as follows. After presenting the structure of the questionnaire and explaining some choices made, a suggested method to define the universe of the respondents and collect the data is proposed. Finally, a couple of examples of implementation are briefly presented.

Outline of the questionnaire

The questionnaire focuses on the funding or investment in educational research by collecting information about the performance and the funding of educational research and development, and proposes definitions of what should be counted as performance and as funding to avoid double counting.

After an introduction about the questionnaire, which can be customised by the administering organisation, and contact information about the respondents, section 2 tries to categorise the responding organisation and identify whether if is a funder and/or performer of educational R&D.

Compared to usual R&D questionnaires, the business sector has been left out. This is mainly for practical reasons: the possible units to survey would then be too numerous if those were included, but in principle the questionnaire could also work for for-profit companies performing or funding educational research. In education, most research comes from the higher education sector and the government. In some countries (for example the United States), the private non-profit sector (foundations) plays a large role too, either as a performer or funder of educational research. This is why this is an explicitly included in the scope of the survey. Because of the educational nature of the questionnaire and the fact that most of the answers will be of interest to departments of education, question 4 requires details about the nature of the responding organisation. All the questionnaire focuses on a specific fiscal year (to be specified depending on the date of administration).

The questionnaire is divided in two big sections: performance of educational R&D (section 3) and funding of educational R&D (section 4), with mirroring questions. Question 5 is both a summary and a “skip” question that collects the total amount that the responding organisation spends on performance and funding of educational R&D. The questionnaire is thus relevant for organisations that only perform educational research, only fund educational research or do both.

Section 3 requires details about the performance of educational R&D, using respondents to provide a (rough) percentage of the allocation of their performance between different research topics (question 6), levels of education (question 7), sources of funding (question 8).

The different areas of educational research propose high level categories that build on “special interest groups” as defined or identified by different research associations (e.g. American Educational Research Association (AERA), European Educational Research Association (EERA), European Association for Research on Learning and Instruction (EARLI), etc.). While educational research associations routinely have around 20-30 interest groups, our categories summarised the main themes in six broad umbrella topics (e.g. education “statistics”, “teacher education”, “learning, instruction and curriculum”, plus the possibility to add one’s own category). It is indeed unlikely that respondents can provide a very granular answer to this question so framing it according to what could be their main departments seemed more appropriate.

Question 6 asks about the level of education the educational research is about – noting that there might be differences across countries. (For an international survey, countries could use the International Standard Classification of Education (ISCED) classification and translate it into their national categories). Vocational education and training was omitted on the ground that it covers secondary education tertiary education and “adult learning”. Relevant categories could be added in a domestic context. When research is valid for all or several levels of education, a response category was provided.

The sources of funding defined in Question 8 take into account the federal nature of some countries (central/local government), the fact that for-profit companies (industry) sometimes fund research and development, the role of foundations and other non-profit organisations, the role of the organisations’ own funds, and finally, the role of international (or inter-governmental) organisations such as the European Commission, the World Bank, other banks of regional development, and others.

Questions 9 and 10 try to identify the amount of development in education, which is typically much lower for the target organisations than actual research. Question 9 requests a raw percentage on the divide between research (basic and applied) and development, while question 10 enumerates the typical development outputs an organisation involved in educational R&D could have.

Question 11 provides a shortened adaptation of a “barrier” question that is typical in R&D surveys, limiting the answer options to those most relevant to the education sector. Acknowledging that those answer proposals were mainly designed for the private sector, two open answers are suggested if need be.

Question 12 asks about the headcount and full time equivalent personnel devoted to educational R&D in the responding organisation. In order to make it easier to answer, but also to provide information about whether educational researchers work in “specialised” organisations or not, the question asks for the count for all fields of science, for educational R&D and for personnel that is not devoted to R&D (e.g. administrative or communication staff).

Section 5 asks for the same questions in the frame of the funding rather than performance of educational R&D, using the same sub-categories: sub-areas of R&D (question 13), target level of education (question 14), sector of recipient (rather than source) organisation (question 15), research or development (question 17) and type of development output (question 18).

Question 16 acknowledges that a funding institution of educational R&D could get its funding from different types of organisations. For example, a national research agency may get the funding it allocates to educational R&D from government appropriations or local appropriations, but also from other sources of funding. It is also common for foundations to pool their resources so that a funding organisation may allocate funds that were received from another non-profit organisation.

Section 6 allows respondents to enter comments to qualify and explain their answers.

Suggested survey methodology

Collecting information about educational research and development is challenging because only a small part of a country’s research and development ecosystem invest in or perform educational research and development. Surveying the entire ecosystem for this purpose would be a difficult and expensive exercise, often unrealistic for most countries or jurisdictions and leading to high rates of non-response. Therefore, the suggested methodology is comprised of three phases: the first is to conduct an initial mapping of educational R&D within the jurisdiction (or each country participating if the survey is conducted internationally). The second phase is to use this mapping to collect data on educational R&D from key actors. The third phase is to analyse the questionnaire responses and prepare a report.

Phase 1: Mapping of educational R&D

The main aim of the mapping exercise is to establish the universe for the survey on educational R&D. In the case of an international implementation, this phase also allows to verify that the approaches are consistent across countries and establishing which institutions should be involved in the survey in each country).

The target universe is institutions in the public, higher education and non-profit sectors financing or undertaking educational R&D. Relevant institutions in the mapping will typically include: government departments (not solely those in charge of education), government agencies, universities, public research centres/organisations, foundations, research councils and statistical agencies. The survey is intended to include both performers and funders of educational R&D (with separate questions to minimise any risk of double counting). The private sector and schools are not within the scope of the proposed survey instrument. In the higher education sector, the focus is not solely on education science departments –research on education topics carried out in a range of disciplines such as economics, sociology or medicine is within scope and should be included in the mapping process in principle.

The questionnaire takes into account the diverse activities within educational R&D. Curriculum development and educational evaluation or statistics should therefore be included in the mapping exercise if the results of these exercises are made public. Parts of the public educational inspectorate are also likely to meet the definition.

However, depending on the context, only the main actors with a significant activity in educational R&D should be included

The mapping exercise will typically include the following key actions for countries/jurisdictions:

• Identify the universe of domestic educational research based on the definitions above (the focus of the project should be made explicit to avoid misinterpretation). The priority of the mapping is to identify large performers of educational R&D – an exhaustive census of relevant actors is not expected. Annex 9.B gives some example of the information that countries or jurisdictions could aim to collect about the educational research units for their internal discussions.

• To conduct the mapping, jurisdictions should draw on expert knowledge to help identify where educational R&D takes place. In particular, national associations dealing with educational research or major educational research institutes should be consulted. Countries may benefit from establishing a review group of experts.

• Countries may also wish to draw on bibliometrics to identify relevant institutions.

• Countries should then review the mapping process and output, ideally with external experts not involved in the mapping group in order to introduce a layer of internal checks.

• The mapping phase will allow countries to undertake cognitive testing of the draft questionnaire with national experts. If implemented locally, the questionnaires should be adjusted accordingly. If implemented internationally, feedback should be shared with the international coordinator at this early stage so that a wording that work for all participating countries/jurisdictions can be agreed upon.

Phase 2: Survey of educational R&D practitioners

The model of administration is similar to the one for an innovation survey (see chapter 2 of this report, (Van Lieshout, Arundel and Vincent-Lancrin, 2023[3])).

Depending on the size of the relevant R&D practitioners, they may opt for two strategies: a census of all large R&D performers and funders that they have identified; a stratified random sampling based on characteristics of importance within the concerned jurisdiction.

One of the key challenges in such a survey will be to ensure a reasonable response rate and to minimise selection bias. Establishing contact persons for each institution and conducting questionnaires in‑person may assist this process. Another issue may come from how the possible use of the survey information will be perceived by the respondents, and how the confidentiality of the information will be managed. Countries could resort to their official statistical agency, if any, commission a third party that would provide them with the survey data in an aggregated form only to protect anonymity for the participating organisations. They may also consider guaranteeing that the responses will not be used for other purposes than statistical ones. Institutions may also be wary of responding to non-compulsory government surveys; partnering with educational research institutions may be one way to resolve this problem.

The data collection will typically include the following key actions for countries/jurisdictions:

• Once finalised, countries will need to organise translation of the questionnaire into their national language (in the case of an international survey).

• Identify contact persons in key institutions to whom to send the questionnaire (the expert group during the mapping exercise may help with this). For universities, targeting the relevant departments directly will be necessary, though in some countries the institutions would have to be formally notified.

• Consider visiting institutions in-person in order to encourage participation. In many instances, responding to the questionnaire will require collaboration between researchers and administrative or budget officers (e.g. in defining how much expenditure is allocated to R&D).

• Countries also have the option of using fieldwork teams to assist the data collection, or draw on the expertise of the community identified during the mapping. Working collaboratively with relevant educational associations or institutes is advised, as it may change the way the survey is perceived and encourage participation.

• Reach at least a 60% response rate and analyse the non-response to ensure that it can be considered as random.

• Undertake a check of the collected data, highlighting any issues in consistency and comparability.

Phase 3: Report, analysis and discussion

Once the data from the questionnaires are collected, countries should ensure some level of quality assurance of the data collection. The data analysis performed on the aggregated responses will allow for the preparation of a draft report providing comparative information on:

• the amounts spent on educational R&D, presented in absolute terms and in relative terms to account for the size of the country/jurisdiction and its level of public expenditures on education;

• the relative weight of different types of actors in the performance and funding of educational R&D (government, higher education, private non-profit);

• the levels of education and topics of education the most commonly researched and funded in absolute terms and by different types of actors to see whether different actors are inclined to perform or fund different types of research, whether there is a good balance to improve different dimensions of the education process;

• the share of experimental development in the total R&D and what are its typical outputs;

• the level of overlap between funding and performance (are the sectors funding also the sectors that are performing, and do they do so through specialised institutions, within ministries or by other means);

• the composition of human resources of educational R&D.

Ideally, the repetition of the survey at regular intervals will allow to monitor the evolution of this funding and perhaps will allow for zeroing in on specific topics from one edition to the next.

Two examples of successful implementation

A pilot implementation of the questionnaire was proposed to OECD countries in 2013-15. Three countries/education systems participated. One succeeded in doing the mapping and implemented the data collection, but failed to engage one of the larger educational research performers, a non-response that made the other collected results impossible to redress or interpret with enough accuracy. Two other countries successfully implemented the methodology and the data collection: the Netherlands (de Jonge, 2016[4]) and Norway (Gunnes and Rørstad, 2015[5]).

Norway has carried out surveys of educational R&D since 2007, so the mapping to identify the relevant units to survey for the data collection was straightforward. Adjustments consisted mainly in mapping Norwegian response categories to the ones provided by the OECD questionnaire (for example go from the “Institutes”, which includes a few private research institutes, to the “Government” category) and in adding some questions. Gunnes and Rørstad (2015[5]) present the implementation of the OECD survey. Since their 2013 survey, Norway has replicated the OECD questions in its subsequent surveys of educational R&D (Rørstad et al., 2021[6]; Rørstad et al., 2019[7]; Gunnes, Hovdhaugen and Olsen, 2017[8]) .

The Netherlands also successfully implemented the proposed approach in 2015, in their case with no prior survey infrastructure. De Jonge (2016[4]) summarises the main findings as follows:

“The most important finding is that investment in educational R&D is limited in terms of both budget appropriations and number of individuals working in this field. Approximately 150 institutions in the Netherlands are involved in educational R&D. They employ almost 1000 FTEs [full time equivalents] in research, with a further 175 FTEs providing research support. According to our calculations, the Netherlands invests a total of approximately EUR 120 million per annum in educational R&D. If we compare these figures to total employment in education or to total expenditure on education, we see that spending on educational R&D is limited. Expressed as a percentage, it comes to 0.28%. That is relatively little compared with the almost 2% spent on R&D nationally (across all sectors). Our conclusion is that education is a research-extensive sector in which investment in education itself has very little grounding in research and development work evaluating the effectiveness and efficiency of that investment.”

Both examples show that the methodology and standards could easily be used internationally once a political will emerges within OECD countries.

References

[4] de Jonge, J. (2016), Onderwijsonderzoek in Nederland - Inventarisatie voor de OECD, Rathenau Instituut.

[8] Gunnes, H., E. Hovdhaugen and B. Olsen (2017), Utdanningsforskning i Norge 2015: Ressurser og resultater, http://hdl.handle.net/11250/2442190.

[5] Gunnes, H. and K. Rørstad (2015), Educational R&D in Norway 2013: Resources and Results, http://hdl.handle.net/11250/2412424.

[2] OECD (2015), Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris, https://doi.org/10.1787/9789264239012-en.

[1] OECD (2002), Frascati Manual 2002: Proposed Standard Practice for Surveys on Research and Experimental Development, The Measurement of Scientific and Technological Activities, OECD Publishing, Paris, https://doi.org/10.1787/9789264199040-en.

[7] Rørstad, K., B. Sarpebakken, K. Wendt, H. Gunnes, B. Magne Olsen and M. Østby (2019), Ressursinnsatsen til FoU innenfor tematiske områder i 2017, http://hdl.handle.net/11250/2619010.

[6] Rørstad, K., B. Sarpebakken, K. Wendt, M. Nedberg Østby, H. Gunnes, E. Denisova and F. Sæbø Steine (2021), Ressursinnsatsen til FoU innenfor tema- og teknologiområder i 2019, https://hdl.handle.net/11250/2771665.

[3] Van Lieshout, K., A. Arundel and S. Vincent-Lancrin (2023), "Measuring Innovation through Surveys: Main Considerations and Applications to Education", in Vincent-Lancrin, S. (ed.), Measuring Innovation in Education 2023: Tools and Methods for Data-Driven Action and Improvement, OECD Publishing.

Annex 9.A. OECD questionnaire on educational research and development

Innovation and improvement in education partly relies on educational research and experimental development. The [name of surveying organisation] seek to improve the quality of data on educational research and development by undertaking a new data collection following internationally comparable categories. The answers collected through this survey are intended to be used for drafting reports and papers produced by the [name of surveying organisation] and not be used for any other purposes. The questionnaire has been developed by the OECD Centre for Educational Research and Innovation (CERI), mainly for an international implementation allowing for comparative information. [If relevant: It has been adapted by <name of the surveying organisation>.]

Definition

Research and experimental development (R&D) is defined as “creative and systematic work undertaken in order to increase the stock of knowledge – including knowledge of humankind, culture and society – and to devise new applications of available knowledge”.

Research by students at the PhD level carried out at universities should be counted, whenever possible, as a part of R&D. However, all education and training of personnel in universities and special institutions of higher and post-secondary education should be excluded.

Guidelines and scope

The questionnaire follows the methodological guidelines of the Frascati Manual. Most terms used in the questionnaire are clarified in the Frascati Manual.

This survey collects data on educational research and experimental development performed by organisations from the government, higher education and non-profit sectors in OECD countries.

The person responsible for completing this survey should be an individual or officer who leads or supervises R&D undertaken in the organisation. Complete responses are essential to support a comprehensive collection of key elements of educational research. Please answer all questions. Where exact data are not available, please provide an estimate and indicate it in section 5.

The questionnaire is structured as follows:

Section 1: Contact information

Section 2: Organisation information

Section 3: Performance of Educational R&D (for organisations performing R&D)

Section 4: Funding of Educational R&D (for organisations funding R&D)

Section 5: Additional information

For further assistance in interpreting the questions contained in the survey or for any related questions, please contact:

 

Ms/Mr xxxxxxxxxxxxxxxx

Ms/Mr xxxxxxxxxxxxxxxxx

Section 1: Contact Information

Primary Contact (person who is responsible for the institution’s survey answers)

Full name __________________________________

Position __________________________________

E-mail address __________________________________

Telephone number __________________________________

Fax number __________________________________

Alternate Contact (person to contact if the primary contact is unavailable)

Full name __________________________________

Position __________________________________

E-mail address __________________________________

Telephone number __________________________________

Fax number __________________________________

Section 2: Organisation Information

1. Name of organisation / unit ______________________

2. Name of parent organisation ______________________

3. Country ______________________

4. In which sector of the economy would you classify your organisation? (please tick one answer)

 

☐ Higher Education ☐ Government ☐ Private non-profit

 

This sector includes all universities, colleges, and other institutions of post-secondary education. It also includes all research institutes, experimental stations and clinics operating under the control of, administered by or associated with higher education institutions.

This sector includes all bodies, departments and establishments of government at all levels – central, state or provincial, district or municipal. It includes non private institutions controlled and mainly financed by government, but not administered by the higher education sector.

This sector includes non-market, private non-profit institutions serving the general public, other than those mainly financed by government or providing higher education services or administered by higher education institutions

Please give additional details of your main activity (and only tick one box corresponding to the sector selected in your previous answer).

 

Higher Education

Government

Private non-profit

 

☐	University	☐	Central	☐	Foundation
☐	College	☐	Regional (state)	☐	Non-governmental organisation
☐	Research institution	☐	Local (district, city, etc.)	☐	Other (please specify)
☐	Clinic	☐	Other (please specify)
☐	Other (please specify)
_____________________________________________________________________________________

5. During fiscal year <20xx>, how much of your organisation’s expenditures were allocated to perform and fund educational R&D? (please write down the amounts in your national currency)

Educational R&D consists of a broad range of activities, including but not limited to research and experimental development on: educational policies; management, organisations and leadership; curriculum studies; learning; instructional methods; teaching and teacher education; assessment and evaluation; education statistics; and educational technology. Please consider also all other activities not listed here which you regard as educational R&D. All education and training of personnel in universities and special institutions of higher and post-secondary education should be excluded. However, research by students at the PhD level carried out at universities should be counted, whenever possible, as a part of R&D.

All your R&D expenditures should fall in either performance or funding of R&D. Please ensure you do not count any expenditure twice. Educational R&D can be performed with your own funds or other funds, and includes sub‑contracted research. If your organisation has been sub-contracted to carry out educational R&D that belongs to another organisation, it should NOT be included.

Copyright is not the only criterion of belonging. If your organisation or its members have performed some research that will be published in a scientific journal or by a publisher that will own the copyright of your work, please report this research as a performer. If your organisation or its members perform research for another agency, for example a statistical agency, which will publish the work under its corporate name, please do NOT report it as performed by your organisation, even if you are acknowledged as author: the commissioning agency will report it in its performance of research (and consider your contribution as sub-contracted and thus part of its performance budget). If you produce by-products of commissioned work for scientific journals, commercial publishers, etc., they should be reported as performed educational R&D.

Funding only concerns research grants and awards, and not commissioned research as part of your own research. Research grants may be given through all sorts of mechanisms and can be given for specific areas of research, specific types of research, or with full freedom in topics and methods to researchers.

 

	Amount (National Currency)
Perform educational R&D R&D is considered performed by your organisation if its outputs (e.g., publications, patents, product, services) belong to your organisation or the researchers of your organisation. Include current costs (both direct and indirect) and capital expenditures (report in full when purchased in fiscal year <20xx>, exclude depreciation provisions).
Fund educational R&D R&D is considered funded by your organisation if its outputs (e.g., publications, patents, product, services) belong to the recipient of the funds. Do not count the same resources twice: amounts in this cell should not contribute to the previous cell.

If your organisation did not allocate any expenditures to perform educational R&D, please skip to question 13 (Section 4).

Section 3. Performance of Educational R&D

The questions in this section refer to the educational R&D activities performed by your organisation in fiscal year <20xx>.

6. Specify the percentage of the total educational R&D performed in fiscal year <20xx> by sub-area of educational R&D.

If the R&D activities performed by your institution belong to several of the categories below, please allocate them proportionately to the amount spent on the different sub-areas in your research. Should this be too difficult to estimate, please allocate the percentage evenly between the sub-areas concerned.

Example: a research programme may produce indicators on the management of schools. If the research mainly emphasises the statistical dimension, you may allocate its expenditures as 80% in "education statistics" and 20% in "Management, Organisation, and Leadership". If you do not find a satisfactory allocation, please allocate 50% to each sub-area.

 

	Percentage
Educational Policy, Politics, and Social Context Research on educational policy, including political, economic, legal and fiscal issues. Research on relations between educational processes and the social, political, and economic contexts in which they occur. Research on the evaluation of specific educational policies. Educational policy design.
Management, Organisation, and Leadership Research on administration, management and organisation in the education sector, and sources and types of educational leadership, including administrators and other actors inside and outside of schools.
Learning, Instruction and Curriculum Research on learning processes and instruction practices in formal and informal settings, for all age groups. Development of instructional methods based on learning and instruction research. Research on curriculum. Research on education for professions other than teaching (e.g., architecture, engineering, health professions, law).
Teacher Education Research on teaching, on teacher development and education from preservice through professional induction to the in-service stages of teachers’ careers.
Assessment and Evaluation Psychometrics and quantitative methods as applied to educational research as well as qualitative and evaluation methods as applied to educational research. Development of assessment tools.
Education Statistics and Research Methods Research aimed at development and consolidation of educational indicators at different levels. Research aimed at improving quantitative and qualitative research methods in education.
Other (please specify) ______________________________________________
Total (sum of all rows)	100

7. Specify the percentage of the total educational R&D performed in fiscal year <20xx> about specific levels of education.

If the R&D activities performed by your institution belong to several of the categories below, please allocate them proportionately to the amount spent on the different education levels in your research. Should this be too difficult to estimate, please allocate the percentage evenly between the sub-areas concerned.

 

	Percentage
Early childhood education
Primary education
Secondary education
Tertiary education
Adult education and lifelong learning
Not applicable R&D that cannot be classified by sector of education
Total (sum of all rows)	100

8. Specify the percentage of the total educational R&D performed in fiscal year <20xx> by source of funding.

Funds received for R&D performed during earlier periods or for R&D not yet started should be excluded from the sources of funds reported for the specific period. If the funding passes through another organisation, you should report the initial source of funding.

 

	Percentage
Central government Include awards and funds for educational R&D (including direct and reimbursed indirect costs) granted by all agencies of the central / federal government.
Local government Include awards and funds for educational R&D (including direct and reimbursed indirect costs) from state, county, municipal, or other local government and their agencies.
Industry Include all awards and funds for educational R&D (including direct and reimbursed indirect costs) from profit-making organizations. Do not include awards from non-profit foundations financed by industry; these should be included under Private non-profit.
Private non‑profit Include all awards and funds for educational R&D (including direct and reimbursed costs) from private non‑profit institutions, regardless of their source of funding.
Own organisation funds Include all funds (including related indirect costs) that your institution spent for educational R&D activities from the following unrestricted sources: tuition and fees; endowment income; gifts; and other funds generated by the institution.
International funds Include all funds from the EU, international organisations, foreign governments, private non-profit, businesses, including from the same group.
Other sources (please specify) _______________________________________________
Total (sum of all rows)	100

9. Specify the percentage of the total educational R&D performed by type of R&D.

 

	Percentage
Research (Basic and Applied) Experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundation of phenomena and observable facts, without any particular application or use in view AND original investigation undertaken in order to acquire new knowledge but directed primarily towards a specific practical aim or objective.
Experimental development Systematic work, drawing on existing knowledge gained from research and/or practical experience, which is directed to producing new materials, products or devices, to installing new processes, systems and services, or to improving substantially those already produced or installed.
Total (sum of all rows)	100

If your organisation did not perform any development activities during fiscal year <20xx>, please skip to question 11 in this section.

10. During fiscal year <20xx>, did your organisation engage in the development of the following products/services, in whole or in part: (tick as many as applicable)

 

	Curriculum
	Textbooks and other learning/teaching resources
	Digital learning resources using advanced technology (e.g. intelligent tutoring systems, etc.)
	Assessment tools
	Educational software
	Learning tools (electronic devices, learning objects, maps, etc.)
	Data systems for educational institutions
	Teaching strategies or pedagogical models
	Other: (please specify) __________________________________________________
	Other: (please specify) ___________________________________________________

11. Please rank the two most significant barriers to your institution’s educational R&D activities? (place a “1” for the most significant barrier and a “2” for the second most important barrier.)

 

	Access to funding
	Access to skilled human resources
	Public perception / acceptance of R&D activities
	Other: (please specify) _________________________________________________________
	Other: (please specify) _________________________________________________________

12. Specify the headcount and full time equivalent (FTE) personnel devoted to total R&D activities and educational R&D activities in fiscal year <20xx>, by occupation.

Headcount refers to the total number of persons who are mainly or partially employed in R&D. This includes both “full-time” and “part-time” employed staff.

Full-time equivalent (FTE) refers to the total effort devoted to R&D in terms of personnel. As R&D may be carried out by persons who work solely on R&D projects or by persons who devote only part‑time to R&D, it is necessary to estimate the full-time equivalent of the persons working only part-time in R&D. Examples of calculation: if out of five researchers engaged in R&D work, one works solely on R&D projects and the remaining four devote only one quarter of their working time to R&D, then: FTE = 1 + ¼ + ¼ + ¼ + ¼ = 2 researchers; if out of 3 researchers, one works for 6 months, one for 3 months, and one for one year, then: FTE = ½ + ¼ + 1 = 1.75.

Researchers are professionals engaged in the conception or creation of new knowledge, products, processes, methods and systems and also in the management of the projects concerned. Managers and administrators engaged in the planning and management of the scientific and technical aspects of a researcher’s work also fall into this category. Their rank is usually equal or superior to that of persons directly employed as researchers and they are often former or part-time researchers. Postgraduate students at the PhD level engaged in R&D should be considered as researchers.

Technicians and equivalent staff are persons whose main tasks require technical knowledge and experience in one or more fields of engineering, physical and life sciences or social sciences and humanities. They participate in R&D by performing scientific and technical tasks involving the application of concepts and operational methods, normally under the supervision of researchers. Equivalent staff perform the corresponding R&D tasks under the supervision of researchers in the social sciences and humanities.

Administrative support staff includes skilled and unskilled craftsmen, secretarial and clerical staff participating in R&D projects or directly associated with such projects.

 

	Headcount	FTE
Total R&D (all fields of science)
Researchers
Technicians and equivalent staff
Administrative support staff
Educational R&D
Researchers
Technicians and equivalent staff
Administrative support staff
Personnel not devoted to R&D

Section 4. Funding of Educational R&D

If your organisation did not allocate any expenditures to fund educational R&D, please skip to section 5.

R&D is considered funded by your organisation if its outputs (e.g., publications, patents, product, services) belong to the recipient of the funds. Sub-contracting research should not be considered as funding. Funds passing through your organisation to another organisation should not be reported as funding either, for example because you co‑ordinate a project funded by another agency and allocate the funding to different partners.

13. Specify the percentage of the total educational R&D funded by your organisation during fiscal year <20xx> by sub-area of educational R&D.

If the R&D activities funded by your institution belong to several of the categories below, please allocate them proportionately to the amount spent on the different sub-areas in your research. Should this be too difficult to estimate, please allocate the percentage evenly between the sub-areas concerned.

 

	Percentage
Educational Policy, Politics, and Social Context Research on educational policy, including political, economic, legal and fiscal issues. Research on relations between educational processes and the social, political, and economic contexts in which they occur. Research on the evaluation of specific educational policies. Educational policy design.
Management, Organisation, and Leadership Research on administration, management and organisation in the education sector, and sources and types of educational leadership, including administrators and other actors inside and outside of schools.
Learning, Instruction and Curriculum Research on learning processes and instruction practices in formal and informal settings, for all age groups. Development of instructional methods based on learning and instruction research. Research on curriculum. Research on education for professions other than teaching (e.g., architecture, engineering, health professions, law).
Teaching and Teacher Education Research on teaching and on teacher development and education from preservice through professional induction to the in-service stages of teachers’ careers.
Assessment of Learning and Evaluation Psychometrics and quantitative methods as applied to educational research as well as qualitative and evaluation methods as applied to educational research. Development of assessment tools.
Education Statistics and Research Methods Research aimed at development and consolidation of educational indicators at different levels. Research aimed at improving quantitative and qualitative research methods in education.
Other (please specify) ______________________________________________
Total (sum of all rows)	100

14. Specify the percentage of the total educational R&D funded by your organisation during fiscal year <20xx> by level of education.

If the R&D activities funded by your institution belong to several of the categories below, please allocate them proportionately to the amount spent on the different education levels in the research you funded. Should this be too difficult to estimate, please allocate the percentage evenly between the sub-areas concerned.

 

	Percentage
Early childhood education
Primary education
Secondary education
Tertiary education
Adult education and lifelong learning
Not applicable Educational R&D that cannot be classified by sector of education
Total (sum of all rows)	100

15. Specify the percentage of the total educational R&D funded by your organisation during fiscal year <20xx> by the sector of recipient institution.

 

	Percentage
Government This sector includes all bodies, departments and establishments of government at all levels –central, state or provincial, district or municipal. It includes non-private institutions controlled and mainly financed by government, but not administered by the higher education sector.
Higher education This sector includes all universities, colleagues and other institutions of post-secondary education. It also includes all research institutes and experimental stations operating under the control of, administered by or associated with higher education institutions.
Industry This sector includes all firms, organizations and institutions whose primary activity is the market production of goods and services for sale to the general public at an economically significant price.
Private non-profit This sector includes non-market, private non-profit institutions serving the general public, other than those mainly financed by government or providing higher education services or administered by higher education institutions
Total (sum of all rows)	100

16. Specify the percentage of the total educational R&D funded in fiscal year <20xx> by source of funding.

Funds received for R&D funded during earlier periods or for R&D not yet started should be excluded from the sources of funds reported for the specific period.

 

	Percentage
Central government Include awards and funds for educational R&D (including direct and reimbursed indirect costs) granted by all agencies of the central / federal government.
Local government Include awards and funds for educational R&D (including direct and reimbursed indirect costs) from state, county, municipal, or other local government and their agencies.
Industry Include all awards and funds for educational R&D (including direct and reimbursed indirect costs) from profit-making organizations. Do not include awards from non-profit foundations financed by industry; these should be included under Private non-profit.
Private non‑profit Include all awards and funds for educational R&D (including direct and reimbursed costs) from private non‑profit institutions, regardless of their source of funding.
Own organisation funds Include all funds (including related indirect costs) that your institution spent for educational R&D activities from the following unrestricted sources: tuition and fees; endowment income; gifts; and other funds generated by the institution.
International funds Include all funds from the EU, international organisations, foreign governments, private non-profit, businesses, including from the same group.
Other sources (please specify) ____________________________________________
Total (sum of all rows)	100

17. Specify the percentage of the total educational R&D funded by your organisation during fiscal year <20xx> by type of R&D.

R&D is considered funded by your organisation if its outputs belong to the recipient of the funds. Please do not include any research sub-contracted to another organization.

 

	Percentage
Research (Basic and Applied) Experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundation of phenomena and observable facts, without any particular application or use in view AND original investigation undertaken in order to acquire new knowledge but directed primarily towards a specific practical aim or objective.
Development Systematic work, drawing on existing knowledge gained from research and/or practical experience, which is directed to producing new materials, products or devices, to installing new processes, systems and services, or to improving substantially those already produced or installed.
Total (sum of all rows)	100

18. During fiscal year <20xx>, did your organisation fund the development of the following products/services, in whole or in part: (tick as many as applicable).

 

	Curriculum
	Textbooks and other learning/teaching resources
	Digital learning resources using advanced technology (e.g. intelligent tutoring systems, etc.)
	Assessment tools
	Educational software
	Learning tools (electronic devices, learning objects, maps, etc.)
	Data systems for educational institutions
	Teaching strategies or pedagogical models
	Other: (please specify)
	___________________________________________________________
	Other: (please specify)
	___________________________________________________________

Section 5. Additional information

Please provide comments below on any information you have supplied. Please specify the questions where you provided estimates rather than exact data.

 

Thank you for completing this questionnaire.

Annex 9.B. Example of information table for the unit mapping exercise

 

Institution information			Contact information
Name of organisation	Sector	Educational R&D activity	First Name	Last Name	Position	E-mail	Phone	Remarks / Obs.