Oslo Manual 2018

Chapter 2. Concepts for measuring innovation

Abstract

This chapter provides the context and key foundations for innovation measurement underpinning this manual. It describes major perspectives and theories of innovation, user needs for innovation data, a framework for innovation measurement, and different approaches to measuring innovation. Although this manual focuses on the measurement of innovation in the Business enterprise sector, this chapter provides a general definition of innovation that applies to all sectors and discusses the measurement of innovation in both the Business enterprise sector and in other sectors.

2.1. Introduction

2.1.  This chapter provides the context for innovation measurement, and outlines its rationale and possibilities. It describes the concepts that underlie major perspectives and theories of innovation, user needs for innovation data, the elements of a framework, and different approaches for innovation measurement. A general definition of innovation that is suitable for all sectors is developed and presented in the final section of the chapter

2.2.  Innovation is more than a new idea or an invention. An innovation requires implementation, either by being put into active use or by being made available for use by other parties, firms, individuals or organisations. The economic and social impacts of inventions and ideas depend on the diffusion and uptake of related innovations. Furthermore, innovation is a dynamic and pervasive activity that occurs in all sectors of an economy; it is not the sole prerogative of the Business enterprise sector. Other types of organisations, as well as individuals, frequently make changes to products or processes and produce, collect, and distribute new knowledge of relevance to innovation.

2.3.  These dynamic and complex activities and relationships represent significant, but not insurmountable, challenges for measurement. Precise definitions of innovation and innovation activities are required to measure innovation and its subsequent economic outcomes. This manual draws upon the academic and management literature, and recent experience with innovation measurement in multiple countries, to update relevant definitions and measurement guidelines.

2.4.  Data about innovation are relevant to managers and stakeholders of private and public organisations, academics and policy users. Policy analysts and governments around the world seek to promote innovation because it is a key driver of productivity, economic growth and well-being. In addition, policies require an empirically grounded understanding of how innovation works in order to support economic and social changes that can address domestic and global challenges. These challenges include changing demographics, the need for food and housing security, climate change and other environmental issues, and many other obstacles to well-being.

2.5.  Innovation occurs in all of the four broad sectors of an economy, as defined by the United Nations’ (UN) System of National Accounts (SNA): Business enterprises (referred to within the SNA as the corporate sector), General government, Households, and Non-profit institutions serving households (NPISHs) (EC et al., 2009). Although the concepts discussed in this chapter are broadly applicable to all four sectors, the focus of this edition of the Oslo Manual (as with previous editions) is the Business enterprise sector and its linkages within and outside this sector. However, this chapter also provides relevant information for readers interested in measuring innovation in the other three SNA sectors.

2.6.  The structure of this chapter is as follows. Section 2.2 discusses key innovation concepts that set innovation apart from other related phenomena. This is followed by section 2.3, which discusses user needs for innovation data; and section 2.4, identifying the subject and phenomena that characterise the possible scope of innovation measurement. The formulation of a general measurement framework for innovation is completed by section 2.5, which deals with general strategies for measuring innovation and sets out the basis for the measurement choices that this manual applies to the Business enterprise sector. Section 2.6 provides a general definition of innovation and short descriptions of the context for innovation in the government, NPISH and Household sectors. No guidelines for measuring innovation outside of the Business enterprise sector are provided, in the expectation that other guidance, consistent with this manual, will be developed in the future for other SNA sectors.

2.2. The concept of innovation

2.2.1. Conceptual foundations

2.7.  The conceptual foundations for innovation measurement are primarily derived from the management and economics disciplines (Smith, 2006). Management perspectives on innovation cover how innovation can change a firm’s position in the market and how to generate ideas for innovation. Economic perspectives examine why organisations innovate, the forces that drive innovation, the factors that hinder it, and the macroeconomic effects of innovation on an industry, market or economy. Schumpeter’s (1934) theories on how firms search for new opportunities and competitive advantage over current or potential competitors are a major influence in this regard. Schumpeter introduced the concept of “creative destruction” to describe the disruption of existing economic activity by innovations that create new ways of producing goods or services or entirely new industries. The economic growth literature has used this paradigm to investigate the drivers of long-term economic growth.

2.8.  Diffusion theory (Rogers, 1962) examines the processes by which innovations are communicated and adopted over time among the participants in a social system. Evolutionary theories (Nelson and Winter, 1982) view innovation as a path-dependent process (Dosi, 1982) whereby innovations are developed through interactions between various actors and then tested on the market. These interactions and market tests determine, to a large extent, which products are developed and which ones are successful, thereby influencing the future path of economic development. The work by Simon (1982, 1969) into decision-making and problem-solving has influenced the literature on innovation and the emergence of design thinking methods that harness creativity to solve complex problems (Verganti, 2009) for innovations in both private and public sector organisations.

2.9.  Theories of innovation such as Kline and Rosenberg’s (1986) chain-link model and innovation systems theory (Freeman, 1987; Lundvall, 1992; Nelson [ed.], 1993; OECD, 1997) stress that innovation is not a linear, sequential process, but involves many interactions and feedbacks in knowledge creation and use. In addition, innovation is based on a learning process that draws on multiple inputs and requires ongoing problem-solving.

2.10.  The systems perspective of innovation calls for multidisciplinary and interdisciplinary approaches to examine the interdependencies among actors, the uncertainty of outcomes, as well as the path-dependent and evolutionary features of systems that are complex and non-linear in their responses to policy intervention. Innovation systems include organisations from the Business enterprise sector and the three other SNA sectors. Innovation systems can be delineated by industry, technology, or geography and are often interrelated, with local systems linked to national and global systems. Measurement usually collects data at the firm level, with the resulting data then aggregated to provide results at the national or industry level. Innovation measurement that covers multiple countries is of high potential value, but requires considerable co-ordination efforts.

2.11.  Systems perspectives are used for developing innovation policies to co-ordinate system transformations that serve broad societal objectives (OECD, 2016). An example of a system transformation is a regime shift to decarbonise transportation systems (Kemp, Schot and Hoogma, 1998). This would require co-ordination among producers and consumers to ensure that each complementary component of a complex network is in place, particularly when some of the key actors may not exist (such as a dense network of electric vehicle charging stations). Systemic changes can be the outcome and the channel by which new technologies are adopted, for example the application of artificial intelligence across a broad range of uses.

2.12.  An evaluation of innovation theories points to four dimensions of innovation that can guide measurement: knowledge, novelty, implementation, and value creation. Each is discussed below.

2.2.2. Knowledge

2.13.  Innovations derive from knowledge-based activities that involve the practical application of existing or newly developed information and knowledge. Information consists of organised data and can be reproduced and transferred across organisations at low cost. Knowledge refers to an understanding of information and the ability to use information for different purposes. Knowledge is obtained through cognitive effort and consequently new knowledge is difficult to transfer because it requires learning on the part of the recipient. Both information and knowledge can be sourced or created within or outside a relevant organisation.

2.14.  Research and experimental development (R&D), described in detail in the OECD’s Frascati Manual (OECD, 2015a), is one of a range of activities that can generate innovations, or through which useful knowledge for innovation can be acquired (see Chapter 4). Other methods of gaining potentially useful knowledge include market research, engineering activities to assess the efficiency of processes, or analysing data from the users of digital goods or services. Innovation-relevant information can be gathered without a specific application in mind, for instance to help develop and evaluate options for future actions.

2.15.  Knowledge has specific attributes that are relevant to and influence its measurement (Arrow, 1962). Knowledge is non-rival because its use by one organisation or person does not diminish the amount potentially available for use by others. The scope for spillovers that create new knowledge provides a policy motivation for ensuring that knowledge is widely available. However, the resources required to assimilate and effectively use knowledge can be rival (for instance if there is a limited supply of skilled and proficient people or other scarce complementary resources), as well as the ability to realise value from knowledge. Depending on the context, knowledge can be more or less valuable to a given actor if other parties hold it or are able to use it.

2.16.  A number of practices that are supported by economic and social institutions can make knowledge an excludable good, including the use of secrecy or other intellectual property (IP) protection methods. These practices affect the incentives and ability to source and transform new knowledge into innovations. Technological, market and regulatory changes can also influence incentives. For example, the growing ability to digitise, organise and access information at a nil or marginal cost has increased the stock of knowledge that can be made potentially available, and created advantages from being able to exclude other users (Cameron and Bazelon, 2013).

2.2.3. Novelty with respect to potential uses

2.17.  Knowledge can be used to develop new ideas, models, methods or prototypes that can form the basis of innovations. These can be sourced externally or developed within an organisation. The novelty of an innovation is related to its potential uses, as determined by the characteristics of a product or process compared to alternatives, and by the previous experiences of its provider and intended users.

2.18.  Some characteristics can be objectively measured, such as energy efficiency, speed, material strength, fault rates, and other physical attributes, while subjective characteristics such as user satisfaction, usability, flexibility, responsiveness to changing conditions and emotional affinity can be more challenging to measure. Novelty can be difficult to ascertain for subjective characteristics, although the boundary between what can and cannot be measured has shrunk as organisations develop methods to gauge experiential and emotional responses. Furthermore, novelty can be intrinsically subjective because users can assign different priorities to specific attributes, for example one group of users could give higher priority to the ease of use of a mobile phone, while a second group could prioritise its technical performance.

2.2.4. Implementation and actual use

2.19.  In order for a new idea, model, method or prototype to be considered an innovation, it needs to be implemented. Implementation requires organisations to make systematic efforts to ensure that the innovation is accessible to potential users, either for the organisation’s own processes and procedures, or to external users for its products. The requirement for implementation is a defining characteristic of innovation that distinguishes it from inventions, prototypes, new ideas, etc.

2.20.  At a minimum, innovations must contain characteristics that were not previously made available by the relevant organisation to its users. These features may or may not be new to the economy, society, or a particular market. An innovation can be based on products and processes that were already in use in other contexts, for instance in other geographical or product markets. In this case the innovation represents an example of diffusion. Innovation diffusion can generate substantial economic and social value and is consequently of policy importance. This manual defines innovation to include diffusion processes (see Chapter 3), while providing guidelines for identifying different levels of novelty, including new-to-world innovations.

2.21.  Lastly, implementation is not the final step for an innovative organisation. Follow-on activities to review innovations after their implementation can result in minor improvements or radically new innovations, e.g. through a fundamental redesign or major improvements. Some of these follow-on efforts could potentially result in innovations in their own right. Post-implementation reviews can also lead to the abandonment of innovations.

2.2.5. Value creation

2.22.  Viewed as an economic activity, innovation requires resources that could be used for other purposes. The existence of opportunity costs implies the likely intention to pursue some form of value creation (or value preservation) by the actors responsible for an innovation activity. Value is therefore an implicit goal of innovation, but cannot be guaranteed on an ex ante basis because innovation outcomes are uncertain and heterogeneous.

2.23.  Value-related measures are thus important for understanding the impacts of innovation, although there is no single measure of economic or social value in established statistical frameworks such as the SNA. Statistical measures of gross value added capture the production surplus over and above the cost of intermediate inputs (excluding employee compensation or the cost of meeting financing obligations). Financial measures such as net worth capture the value of all assets owned by an institutional unit or sector, minus the value of all outstanding liabilities. These measures can be extended to account for outputs and assets that escape formal accounting conventions and for which market prices cannot provide reliable indicators of economic value.

2.24.  Although it is not possible to make broad generalisations about the drivers of organisational behaviour, decisions to innovate can be presumed, a priori, to have an implicit motive to directly or indirectly benefit the innovative organisation, community or individual. In the Business enterprise sector, benefits often involve profitability. In normally functioning markets, customers have the freedom to decide whether to acquire a new product on the basis of its price and characteristics. Therefore, the markets for products and finance fulfil a selection function for innovations by guiding the processes of resource allocation in the Business enterprise sector. This is replaced by different mechanisms in the other SNA sectors.

2.25.  The realisation of the value of an innovation is uncertain and can only be fully assessed sometime after its implementation. The value of an innovation can also evolve over time and provide different types of benefits to different stakeholders. Complementary measures and analytical strategies can be used to trace innovation outcomes after a suitable length of time. The importance of outcome measures depends on the intended uses of innovation data. They are particularly necessary for the study of government policy initiatives to promote innovation that delivers socially desirable outcomes such as inclusion, sustainability, jobs, or economic growth.

2.3. User needs and relevance of statistical evidence on innovation

2.26.  User needs drive the construction of a system for measuring and reporting innovation and the subsequent production of innovation data, statistics, indicators, and in-depth analyses of innovation activities. There is widespread interest in understanding what drives firms, communities and individuals to innovate and the factors that influence their innovation activities. The relevance of innovation data for understanding innovation processes and drivers can vary across countries, industries and institutional settings. The usefulness of innovation data also depends on the ability to connect them with other types of data.

2.27.  There are three main current or potential users of innovation data: academics, managers, and policy makers or policy analysts. The data needs of all three types of users are similar, with an interest in: (i) obtaining comparable data across industries, regions and time; (ii) keeping up with changes in the nature of innovation, such as open innovation or the use of design thinking principles; (iii) enabling analyses of innovation impacts on innovative organisations, other parties, and regional or national economies; (iv) providing data on the factors that enable or hinder innovation; and (v) linking innovation data to other relevant data, such as administrative registers or data on individual users of innovations.

2.3.1. Research academics

2.28.  Academics use innovation data to improve society’s understanding of innovation and its socio-economic effects, and to test the predictions and implications of a broad range of models on the role of innovation in economic development, organisational change, firm dynamics and social transformation. Academics have a strong interest in research that can provide predictive and causal interpretations of innovation outcomes, which requires longitudinal data on innovation linked to data for variables such as value added, employment, productivity and user/stakeholder satisfaction. Robust causal inference studies are an important input to policy development, as they overcome the limitations of cross-sectional studies that can only identify correlated phenomena.

2.29.  Experience gained from using innovation data for research can point to desirable changes in the measurement framework for collecting innovation data and the types of data that are required to improve analysis (Gault, 2018). Academic researchers conducted many of the initial studies to measure innovation and consequently had a strong influence on the first edition of the Oslo Manual (Arundel and Smith, 2013). Academics also use the Oslo Manual guidelines to develop specialised or “one-off” surveys that test new questions for evaluating theories or hypotheses about innovation and innovation policies. Some of these approaches or questions have been adapted for general data collection.

2.3.2. Business managers

2.30.  Managers can also benefit from statistical evidence on innovation. Although micro-level innovation data collected on a confidential basis cannot be publicly released, managers can use aggregated results for their industry to benchmark their organisation’s innovation activities and outcomes. It is also worth noting that the act of collecting data on innovation in an organisation can indirectly influence managerial decisions by raising awareness of potential innovation activities and resources. This could induce search, learning and other actions leading to innovation among targeted survey respondents (Gault, 2013). The interests and incentives of innovation managers, as main providers of data on innovation, should be placed at the centre of data collection efforts in order to ensure high-quality data.

2.3.3. Innovation and other public policy makers

2.31.  The core target user of innovation data is the policy community, consisting of policy analysts and policy makers. An important function of innovation data is to provide an informed basis for public policy decisions through benchmarking indicators and research using innovation data. Public policy interest in innovation is extensively reflected in the literature (OECD, 2015b, 2010a) and is relevant to all industries and SNA sectors (OECD, 2015c). Consequently, coherent policies across multiple government portfolios are required to marshal the transformational power of innovation in order to achieve key policy objectives.

2.32.  The scope for establishing international benchmarking comparisons is of relevance to this manual’s methodological guidelines, which are intended for use in different economies and to support mutual economic co-operation and development in a multilateral setting. However, not all indicators that are useful for benchmarking or analysis within a single country are suitable for benchmarking across countries, due to linguistic, cultural and contextual differences.

2.33.  In order to determine if a set of data and indicators is well-suited to inform public policy, the goals of public policy need to be identified to ensure that the measurement framework matches policy needs. While policy interests influence the types of data that are needed, policy can also influence the extent and quality of collected data through support for funding new data collection or data linkage to existing sources.

2.34.  The user base for innovation statistics is evolving over time as statistical data on innovation prove to be more or less relevant for informing decisions, or as new data become available. Innovation data are relevant to a wide array of policy areas, including general macroeconomic management, public services and industry, taxation, and environmental policies. Innovation data can be particularly informative for the study of structural policies because of the high degree of persistence of many innovation-related behaviours. This means that some types of innovation data do not need to be collected on a frequent basis, although the value of timely data will increase in the presence of rapid structural change or at times of economic or financial crises.

2.35.  A potential area for future development from a user perspective is the scope for improving the relevance of innovation data to other statistical frameworks. For example, innovation statistics are of relevance to productivity statistics and the measurement of output gaps, trade and foreign investment, deflators, and other economic statistics. Greater recognition of the value of innovation statistics would help integrate innovation measurement in the broader framework of national statistics, where the precedent of satellite accounts on R&D (mainstreamed into the core accounts since the SNA 2008) may one day be followed by innovation satellite accounts.

2.4. Elements of an innovation measurement framework

2.36.  An innovation measurement framework covers a defined scope, such as an SNA sector of interest, a jurisdiction or geographic a rea where data will be collected, a set of relevant phenomena of interest for understanding innovation, and measurement strategies. The latter are discussed separately in section 2.5.

2.37.  The phenomena of interest must be measurable, which requires instruments that can reliably capture intended concepts (Griliches, 1986). For example, survey respondents must be able to understand a question as intended and provide valid responses (meeting one among various validity criteria). The definitions of innovation in Chapter 3 meet basic validity requirements as a result of extensive cognitive testing with potential respondents. This distinguishes them from other definitions in the literature that have not been rigorously assessed for measurability.

2.38.  In addition, valid statistical data must be representative of the target population. This contrasts with other data collection methods based on case studies or other non-representative samples, although these methods can provide very useful information for specific purposes. Further discussion of data quality requirements is provided in Chapters 9 and 11 for business innovation measurement.

2.4.1. Scope of innovation measurement: SNA sectors and jurisdictions

2.39.  As much as possible, the scope of measurement should be consistent with general statistical frameworks. The SNA (EC et al., 2009) provides a globally adopted, generic framework for measuring the economic activities of production, consumption, and accumulation and the associated concepts of income and wealth. The SNA framework is useful for the collection of innovation statistics because it permits the integration of innovation data with other statistical sources that are consistent with the SNA. Furthermore, guidance for measuring innovation in all SNA sectors should follow SNA terminology to ensure consistency.

2.40.  The fundamental unit for analysis in the SNA is the institutional unit, which has legal responsibility for its actions and consequently can own assets, incur liabilities and engage in the full range of economic transactions. In practice, institutional units can be controlled by other units, as in the case of a domestic subsidiary of an international corporation. This can place limits on the autonomy of decision-making.

The jurisdiction for data collection

2.41.  This manual adopts the SNA’s jurisdictional perspective as a reference framework for compiling innovation statistics. The main jurisdiction for data collection on innovation is a country or economy, but innovation data can also be provided at the level of sub-divisions such as regions, states, provinces, municipalities, etc. The “rest of the world” consists of all non-resident organisations that enter into innovation-related relationships or transactions with resident (domestic) units located in a specific country. For some purposes, it can be convenient to describe the rest of the world as if it were a sector.

2.42.  The globalisation of economic activities represents a challenge for measuring jurisdiction-based activities because actors outside the reference country can make decisions on innovation. For example, a head office located in a different jurisdiction could be responsible for such decisions, or a domestic innovation could depend on innovation activities conducted by organisations in other countries. Some of the contributions of non-resident actors can be captured by collecting data on the linkages between non-resident organisations and domestic institutional units. As in other statistical areas, collaboration across different jurisdictions can be necessary to obtain a complete picture of innovation activities that span national boundaries.

SNA sectors and this manual’s focus on business enterprises

2.43.  Institutional units are classified in the SNA into four sectors on the basis of their principal functions, behaviours and objectives:

• The SNA Corporations sector consists of corporations that are principally engaged in the production of market goods and services. This manual adopts the convention of referring to this sector as the Business enterprise sector, in line with the terminology adopted in the OECD’s Frascati Manual (OECD, 2015a).

• General government consists of institutional units that, in addition to meeting their political and regulatory responsibilities, redistribute income and wealth and produce services and goods for individual or collective consumption, mainly on a non-market basis. The General government sector also includes non-profit institutions controlled by the government.

• NPISHs are legal entities that are principally engaged in the production of non-market services for households or the community at large and whose main resource is from voluntary contributions. If controlled by government, they are part of the General government sector. If controlled by firms, they are assigned to the Business enterprise sector.

• Households are institutional units consisting of one or more individuals. In the SNA, individuals must belong to only one household. The principal functions of households are to supply labour, to undertake final consumption and, as entrepreneurs, to produce market goods and services.

2.44.  An institutional unit can be assigned to only one SNA sector. The total economy consists of all institutional units resident in the economic territory of a country. As previously mentioned, the main focus of this manual is the Business enterprise sector, although innovation data can also be collected for institutional units and individuals employed in other SNA sectors, as discussed in section 2.6 below.

2.45.  The Business enterprise sector includes a type of government-controlled unit known as public business enterprises.

2.46.  The “public sector” is a broader concept than the General government sector, with the former including all institutions controlled by government, including public business enterprises. The latter should not be confused with publicly listed (and traded) corporations.

2.47.  The borderline between business enterprises and households presents a number of challenges when dealing with the entrepreneurial activities of households, which consist of unincorporated enterprises that remain within the Household sector, except under specific conditions. These can be particularly relevant for the study of innovation and can be also hard to separate from the Business enterprise sector.

2.48.  Self-employed persons work for themselves, often through establishing an unincorporated enterprise that is not legally separate from its owner. The self-employed include the sole or joint owners of unincorporated enterprises in which they work, contributing family members, and members of producer co-operatives. Examples of unincorporated enterprises include small farms or communal construction.

2.49.  Under some conditions, self-employed and unincorporated enterprises (with or without employees) can be part of the “informal sector” or the “informal economy”. The informal sector can play a very significant economic role, not only in low- and middle-income countries, but also in high-income countries.

2.50.  According to the SNA, the following factors can influence inclusion in the informal sector:

• Registration practices, which differ across countries and activity characteristics. Generally, registered unincorporated enterprises are part of the Business enterprise sector.

• Legal incorporation: units for which a full set of accounts, including a balance sheet, are available or can be drawn up are part of the Business enterprise sector.

• Size in terms of employment or turnover, with very small units more likely to be included in the informal sector.

• Activities such as services for own consumption, which may occasionally be offered to third parties.

• Activities not according to or authorised by law.

• Terms of employment at the boundary of service provision, as in the “gig economy” (individuals working as independent contractors or freelancers instead of as full- or part-time employees).

2.51.  For a wide number of statistical purposes, individuals, rather than the households to which they belong, can be the more appropriate measurement targets.

2.52.  Institutional units with similar principal economic activities are grouped into industries according to the UN International Standard Industrial Classification of All Economic Activities Revision 4 (ISIC Rev.4) (see UN, 2008), or compatible regional classifications (e.g. NACE within Europe, NAICS in North America, and ANZSIC in Australia and New Zealand).

2.53.  Policy interest in measuring innovation often calls for evidence on institutional units engaged in specific economic activities that do not match with SNA institutional sectors. In particular, the Frascati Manual (OECD, 2015a) assigns a special “headline sector” status to units active in providing higher education services, regardless of which SNA sector they belong to. Similarly, many countries also pay special attention and grant special status to many research institutes specialising in the provision of R&D services. Both are called out specifically within this manual’s Chapter 6 in the context of capturing knowledge-based linkages with business enterprises.

2.54.  This manual’s coverage of economic activities in the Business enterprise sector expanded from manufacturing industries in the first edition, to manufacturing and selected service industries in the second edition. The current edition provides guidance for all industries in the Business enterprise sector (see Chapter 9).

2.4.2. Innovation phenomena for measurement

The object of innovations

2.55.  Innovations and innovation activities are the central object of analysis in an innovation measurement framework. Chapter 3 describes the characteristics of product and process innovations from the perspective of business enterprises. Products and processes are generic concepts that are also applicable to the other three SNA sectors.

2.56.  The SNA defines a product as a good or service that results from production activities. Products can be exchanged and used as inputs into the production of other goods and services, for final consumption, or for investment.

2.57.  Goods are objects for which current or potential demand exists and for which ownership rights can be established. Ownership permits goods (and rights to such goods) to be transferred from one owner to another through market transactions.

2.58.  Services are the result of a production activity that changes the conditions of users or facilitates the exchange of products, including financial assets. They cannot be traded separately from their production. By the time their production is completed, they must have been provided to their users. As indicated in the SNA, changes in the condition of users include:

• Changes in the condition of the user’s goods: the producer works directly on goods owned by the user by transporting, cleaning, repairing or otherwise transforming them. Users include other firms, for example a firm can provide materials to another firm to be transformed into a product that the original firm then sells.

• Changes in the physical condition of a person: the producer transports a person or provides accommodation, medical or surgical treatments, changes the appearance of their hair, etc.

• Changes in the psychological condition of a person: the producer provides education, information, advice, entertainment, experience or similar services, potentially but not necessarily in a “face-to-face” manner. These services may be digitally delivered.

2.59.  The boundary between a good and a service can be difficult to identify and is subject to constant change. The provision of goods can shift to service-based models and vice versa. Furthermore, some products can combine features of both goods and services, for example, knowledge-capturing products that concern the provision, storage, safekeeping, communication and dissemination of information that users can copy, share and access repeatedly have features of both goods and services (see Chapter 3). Digital technologies have contributed to an increase in the variety of information and knowledge-based products available, as well as the ways in which production (understood in a general sense) and consumption takes place in all SNA sectors.

2.60.  Production processes (or production activities) are defined in the SNA as all activities, under the control of an institutional unit, that use inputs of labour, capital, goods and services to produce outputs of goods and services. These activities are the focus of innovation analysis.

2.61.  The SNA classifies production activities by the types of goods or services produced as outputs, the types of inputs used or consumed, the techniques or models of production employed, and how the outputs are used. By including goods and services, the concept of production is broader than manufacturing. All SNA sectors have distinctive approaches to production.

2.62.  Beyond production, measurement can identify innovations in redistribution, consumption and other activities. These can be relevant to the study of innovation at the household or systemic level, as major system transformations require not only production shifts, but also the development of new consumption habits for recycling, sustainability, etc.

Activities leading to and following from innovations

2.63.  Institutional units can undertake a series of actions with the intention to develop or adopt innovations. This can require dedicated resources and engagement in specific activities, including policies, processes and procedures.

2.64.  Chapter 4 identifies innovation activities that are used by firms to develop innovations. These activities can be characterised by the knowledge they draw upon and generate, or the stage in the innovation process when they are used. They include R&D, engineering, design and other creative activities; marketing and brand equity activities, IP-related activities, employee training activities, software development and database activities, activities related to the acquisition or lease of tangible assets, and innovation management activities.

2.65.  Engagement in these activities can strengthen organisational or individual capabilities for innovation, although most of these activities can be conducted without an explicit innovation objective. For example, R&D, as formally defined, is neither a sufficient nor necessary condition for either innovation activity or innovation to occur.

2.66.  Innovation activities can be organised around explicit innovation projects. ISO 10006 defines a project as a “unique process consisting of a set of co-ordinated and controlled activities with start and finish dates, undertaken to achieve an objective conforming to specific requirements, including constraints of time, cost and resources” (ISO, 2017). The concept of an innovation project, while useful for understanding how innovation takes place, is unlikely to be applied in the same way across all types of organisations or institutional units. Some organisations, especially large firms, will have a broad portfolio of innovation projects at different stages of maturity, while start-ups could devote all of their resources to a single innovation without viewing it as a project. This limits the usefulness of innovation projects as a construct for measurement.

Transactions and assets of relevance to innovation

2.67.  Innovation data users are interested in the magnitude of efforts devoted to innovation activities. In-house expenditures on these activities can be difficult for managers to estimate if the activity is not undertaken within a formal division of the organisation or under narrowly defined cost codes. In comparison, market purchases of goods or services to support innovation activities can often be identified from company accounts. Chapter 4 discusses methods for estimating expenditures on developing or acquiring knowledge used in business innovation activities, including methods for estimating the internal costs of these activities.

2.68.  Innovation activities can produce knowledge-based assets. The SNA defines an asset as a store of value that represents a benefit or series of benefits accruing to the economic owner by holding or using the asset over a period of time. Both financial and non-financial assets are relevant to innovation. Fixed assets are the result of production activities and are used repeatedly or continuously in production processes for more than one year. The SNA treatment of knowledge assets (formally defined as intellectual property products) has evolved over time, with the addition of R&D in 2008. Other types of knowledge assets that the SNA recognises as generated through production and of relevance to innovation include investment in computer software and databases, and entertainment, literary and artistic items.

2.69.  Knowledge assets can be used by their owners in production or sold on the market if use of the knowledge is restricted through legal or other protection mechanisms. The ability to exclude users provides an incentive to invest in innovation, as recognised in theories of innovation and economic growth (Aghion and Howitt, 1992; Romer, 1990).

2.70.  Units in all sectors can develop or acquire knowledge assets (Corrado, Jäger and Jona-Lasinio [eds.], 2016). Because development requires some degree of specialisation, many units, including firms, acquire knowledge assets of value to innovation without engaging in their production.

2.71.  The study of innovation can extend beyond products and processes. In the SNA, the production activities and ownership of assets generates income for institutional units. Units can use their disposable income for the consumption of individual or collective goods to satisfy household needs or wants. Collective consumption services are provided simultaneously to all members or groups of the community. Changes in consumption patterns over time are a potential object of innovation analysis, especially if the focus is on institutional units with final consumption as a defining attribute, as is the case for the Government and Household sectors.

Knowledge flows

2.72.  Knowledge for use in innovation can be exchanged through market transactions and through non-market means. Relevant channels include knowledge carried in the minds of individuals across different organisational boundaries. Individuals can work temporarily in different organisations without a change in employer, for instance when an employee is seconded to work in an academic institution as part of a collaboration project. Data on the types of networks used, linkages between organisations, and the role of different actors in knowledge creation and diffusion is useful for research on the division of innovation labour across organisations and the creation of innovation value chains. It is difficult, however, to fully trace innovation-relevant linkages due to complex feedback loops and because respondents may not be aware of relevant linkages that extend beyond an immediate partner organisation.

2.73.  Innovations can emerge through linkages between actors within or across different sectors and through a wide range of mechanisms (co-operation, alliances, joint ventures), or as an interactive process involving open innovation or user-producer interactions (OECD, 2013). The conceptualisation and measurement of linkages for innovation in the Business enterprise sector, including the open innovation paradigm, are covered in Chapter 6.

Innovation policies, laws and regulations

2.74.  Understanding the effects of innovation policies on the innovation activities of organisations, especially firms, is of major interest to the policy community. Innovation policies are intended, as a primary or secondary objective, to influence the extent and nature of innovation in an economy. The implementation of innovation policies and practices can be complex and influenced not only by the intention of enabling legislation, but also by their actual use at different organisational and jurisdictional levels. Innovation policies require co-ordination and institutional arrangements that extend beyond science and research ministries to a whole-of-government approach (OECD, 2010a). Typologies of innovation policies, of value for measuring the use of innovation programs by firms, are in continuous development. Chapter 7 discusses methods for assessing the relevance of different policies and policy instruments for the innovation activities of firms.

Innovation outcomes

2.75.  At the level of a society, the ultimate impacts of innovation are the satisfaction of current or future human needs at either the individual or collective level. For a firm, the expectation of outcomes such as an increase in market share, sales, or profits acts as an incentive for innovation. Measuring the extent to which innovation results in social or private outcomes is difficult, but remains a high priority. Furthermore, innovation does not necessarily result in desirable outcomes for all parties.

2.76.  Productivity, profits, jobs, and social and environmental impacts are examples of outcomes of interest to users of innovation data. Innovation outcomes can be widely distributed over time, organisations and individuals. Innovation impacts can be measured directly (e.g. self-reported impacts), or indirectly through the analysis of data on innovation activities, data on outputs (such as different types of innovations) and data on internal or external outcomes (such as profits). Chapter 8 discusses the measurement of outcomes from innovation in the Business enterprise sector.

2.5. General strategies for measuring innovation

2.77.  The choice of which methods to use to measure innovation depends on the quality of the data collected and its intended use. A measurement strategy for innovation must address several issues, such as the choice of a subject or object approach, the collection of qualitative and quantitative data, data sources, and responsibility for data collection.

2.78.  The structure of a measurement strategy can vary over time as user needs and the types of data that can be collected evolve in response to new opportunities or challenges. In addition, different measurement approaches can complement each other. The value to users of innovation data can often be improved by combining several approaches to measurement and by creating opportunities for data linkage and follow-on analysis.

2.5.1. Subject- versus object-based approaches

2.79.  In selecting the unit of analysis, a measurement framework can focus on the phenomena of interest (the object approach) or on the actors that are responsible for the phenomena (the subject approach). It is also possible to combine both approaches: for instance a survey questionnaire can include general questions about strategies and innovation practices (subject), followed by detailed questions focused on a single innovation (object).

2.80.  The most common use of the object-based approach is to collect data on specific innovations, for example innovations reported in trade journals, crowdfunding platforms or, in a survey context, the most important innovation for a given organisation. Other options are to collect data on specific innovation projects or innovation-related transactions or linkages. Object-based approaches can provide a high level of granularity and detail, but can suffer from self-selection or non-representative samples, as when cases are selected from trade journals.

2.81.  The subject approach is commonly used in innovation surveys to collect data on the innovation activities, outputs and outcomes of the respondent’s organisation. Subject-based surveys can benefit from the statistical infrastructure of business registers and other available information at the firm level, including the industry of activity and the number of employees. This permits the drawing of representative samples, analyses at the level of the organisation, and the presentation of results by industry or by region. Another advantage of subject-based surveys is that they can collect data on organisations with no innovations or innovation activities in the reference period, whereas these organisations would not be captured through object-based approaches based on self-reported innovations or innovation activities.

2.82.  Subject- and object-based approaches can converge if it is possible to collect separate data for every innovation introduced by a firm. This is only likely to be feasible for small organisations with only one or two innovations within the observation period. The combined use of subject and object approaches in business innovation surveys is discussed in Chapter 10.

2.5.2. Qualitative and quantitative data

2.83.  Academic and policy users prefer quantitative data for most research purposes. However, survey respondents find it difficult and demanding to report quantitative, interval data for innovation activities or outcomes, such as expenditures, personnel, income generated by innovations, the number and length of collaborations, the number of IP registrations or applications, etc. In addition, many innovation concepts are difficult to quantify, in part because the records and management systems of firms do not align with innovation concepts, or because the concepts only apply to specific contexts.

2.84.  Qualitative measures for innovation activities that cannot be collected on an interval level can be obtained and codified by using questions that ask for nominal or ordinal data, such as the importance of different information sources or categories for the frequency with which these sources are accessed. This type of qualitative data can be used in econometric analysis and to construct indicators.

2.85.  There is considerable scope for using unstructured qualitative data to construct statistics. Examples include self-reported descriptions of an organisation’s most important innovation, or descriptions of innovation strategies in company or organisational reports. These can be codified manually or through machine-based algorithms that use natural language processing techniques. Chapter 9 discusses the collection of qualitative and quantitative data on innovation.

2.5.3. Sources of innovation data

Census and sample statistical surveys

2.86.  Innovation surveys collect data by sending a questionnaire to all firms in a target population that meet predefined inclusion criteria (a census) or to a random sample of the target population. As a census is expensive, representative samples of the population are commonly used. Sample results can be extrapolated to the entire population and differences between subgroups can be tested using statistical inference techniques. However, non-response can reduce the reliability and validity of the results if the respondents are not representative of the entire population and if the size of this effect cannot be accurately measured.

2.87.  Surveys are well-suited for eliciting information that is not available from other sources, provided that respondents have the ability and incentive to report truthfully and accurately. Surveys of organisations face challenges that are not found in surveys where an individual is the subject of interest, as in social surveys. In surveys of complex organisations, designated respondents might be faced with questions they are unable to answer. For instance, an R&D manager could be unaware of the innovation activities of a logistics division or not know the amount spent on the purchase of equipment innovations for production. Accurate answers may only be possible if different people answer different sections of the questionnaire. In contrast, this problem is considerably less likely to occur in small organisations.

Administrative and commercially generated data

2.88.  Data created for administrative purposes or in the course of commercial activities provide a potentially valuable source of information on a range of innovation phenomena.

2.89.  Company filings and published reports can provide detailed information on innovation activities and outcomes, although not always in a structured and comparable fashion. Administrative data can provide detailed information on specific elements of the innovation process, such as applications for different types of IP rights (patents, design registrations, etc.), or on possible outcomes of innovation, such as value added and profits.

2.90.  The increasing digitalisation of economic and social activities provides new and complementary sources of innovation data. Examples include:

• Barcode data signalling product launches and product recalls.

• Data from electronic platforms where individuals or organisations post proposals for innovation projects in order to secure funding and feedback (e.g. Kickstarter). These can provide a measure of user needs and wants.

• Media reports for product launches, joint ventures, collaborations, product reviews, etc.

• Meta-databases such as the Open Knowledge Foundation’s Open Product Data.

2.91.  Internet platforms provide new sources of innovation data derived from diffusion and feedback processes. This is a promising area of future research, although such data must be evaluated for quality and representativeness.

2.5.4. Responsibility for primary source data collection

2.92.  This manual’s guidelines are designed for organisations with expertise in data collection (particularly national statistical organisations [NSOs]), but they can also be useful for other organisations that collect innovation data on a continuous or one-off basis. Other organisations include government agencies, academic and research organisations, international organisations, market research organisations and consultancies.

National statistical organisations

2.93.  NSOs and comparable agencies have the resources, expertise and jurisdictional authority to conduct representative innovation surveys. Comparable agencies include research institutes with delegated data collection responsibilities and quality assurance mechanisms. Many NSOs and comparable agencies can use legislation to compel respondents to answer innovation surveys and can link other administrative information to innovation data. The expertise, independence and reputation of NSOs, plus routines to ensure confidentiality, increase the trust and confidence of survey respondents, thereby helping secure high response rates and high-quality data from representative samples. However, NSOs can face legal or resource constraints that limit the number of questions that can be asked, the ability to link administrative and innovation data, or the use of in-depth innovation surveys that focus on specific topics or parts of the relevant population.

Other organisations

2.94.  Academics and research organisations are regular and frequent users of innovation data collected by NSOs or other comparable agencies. Furthermore, they often self-organise as consortia to conduct one-off or regular surveys of innovation or related topics. Examples include inventor surveys (Giuri et al., 2007), the Division of Innovative Labour survey (Arora, Cohen and Walsh, 2016), and the World Management Survey consortium (http://worldmanagementsurvey.org).

2.95.  Several international organisations have conducted surveys for countries or on topics that were not covered by national innovation surveys. For example, several Eurobarometer surveys, funded by the European Commission, provided in-depth coverage of innovation-related topics such as the effect of public procurement on the innovation activities of firms. Other organisations that have conducted innovation surveys include the World Bank and the European Bank for Reconstruction and Development. A major motivation for international organisations is to obtain microdata on innovation for multiple countries.

2.96.  Market research organisations and consultants can also conduct innovation surveys on behalf of other organisations, including government agencies, foundations, trade bodies, media companies, etc.

2.5.5. Summary of the measurement approach in this manual

2.97.  The Oslo Manual provides guidelines for the statistical measurement of innovation with the following data collection features:

• A target population of business enterprises, which has been progressively extended from manufacturing industries in the first edition to the entire Business enterprise sector in this manual. The Oslo Manual’s guidelines are not expressly designed to measure innovation in other SNA sectors, but research shows that many of the concepts can be applied to them (Gault, 2018).

• A subject approach focused on the innovation activities of a firm. However, this manual provides recommendations for the collection of data on specific innovation objects, such as the most important innovation or innovation project (see Chapter 10).

• Compatibility with censuses or surveys that are representative of the target population and linkable to other data sources (see Chapters 9 and 11).

• Guidelines designed for use by NSOs or delegated agencies that conduct innovation surveys under some degree of public authority. As an open standard, the guidelines can also be used by international organisations, research institutes, academics, and any other groups with an interest in measuring innovation.

• A focus on serving policy user needs through providing guidance for the construction of indicators and for analysis (see Chapter 11).

2.98.  Although not all measurement strategies are sufficiently mature for inclusion in this manual, the intention is to encourage the development of complementary approaches as well as research on questions that are not covered in this manual. Further research and experimentation are necessary to respond to changes in user demand and to improve existing research practices.

2.6. Measuring innovation beyond the Business sector

2.99.  Innovation activities occur in all four SNA sectors. Consequently there is a need for a general definition of innovation that is applicable to all institutional units or entities, while retaining consistency with the definition in Chapter 3 for business enterprises. The general definition of an innovation for all types of units is as follows:

An innovation is a new or improved product or process (or combination thereof) that differs significantly from the unit’s previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process).

2.100.  Processes include policies that provide an overall strategy that drives a unit’s activities, activities that transform inputs into outputs, and procedures that govern the detailed steps for activities to transform inputs into outputs.

2.101.  Newly established entities such as firms or organisations do not have previous products or processes for comparison. In this case the comparison group for defining an innovation is what is available in the relevant market. Therefore, a product or process of a newly established entity is an innovation if it differs significantly from products available in the relevant market or processes that are currently in use by other entities in the relevant market.

2.102.  Specific innovations can involve the participation of multiple actors across sectoral boundaries. These units can be linked through various methods, such as funding mechanisms, hiring of human resources, or informal contacts.

2.6.1. Innovation in the General government sector

2.103.  Government units are established by political processes with legislative, judicial or executive authority and occur at the national, regional and local administrative levels. Public corporations are part of the Business sector. The key difference between a government unit and a public corporation is that the former do not charge economically significant prices for their goods or services. In order to analyse the full engagement of government in innovation in an economy, it can be useful to collect and report data at the level of the entire public sector, which includes all general government units and all public corporations.

2.104.  The range of goods and services provided by government, and the prices charged, are based on political and social considerations rather than on profit-maximisation or related business objectives. This influences the types of product innovations developed by institutional units within the Government sector and made available to households, non-profits or business enterprises. Many process innovations in the Government sector draw on or are similar to innovations in the Business enterprise sector, but public service innovations often pursue redistributive or consumption-related goals that are unique to government. Common characteristics of innovation in the Government sector include the frequent use of collaboration, including with organisations in other SNA sectors, and the co-production of innovations.

2.105.  The presence or absence of a market is frequently cited as the major difference between the Business and Government sectors (Bloch and Bugge, 2013; Gault, 2012; Lægreid, Roness and Verhoest, 2011). The absence of a market alters both the incentives for innovation and the methods for measuring innovation outcomes compared to the business sector. Without data on the cost or price paid for government services, outcome measurement has relied on subjective, self-reported measures, such as an increase in efficiency or improved user satisfaction (Bloch and Bugge, 2013). It is also difficult to provide aggregated economic outcome measures (financial measures of cost savings or benefits) or external validity measures for outcomes. High-quality outcome measures are generally only available for specific innovations. Examples include the cost and benefits of new treatments or protocols in hospitals or new educational methods in schools.

2.106.  The study of innovation within government and the public sector more broadly has attracted a growing body of empirical research, motivated in part by the increasing demand for benchmarking the efficiency and quality of public services as well as identifying the factors that contribute to desirable innovation outputs and outcomes. Many of these studies have adapted the guidelines in the previous edition of this manual to develop surveys of innovation in public administration organisations (APSC, 2011; Arundel and Huber, 2013; Bloch and Bugge, 2013; OECD, 2015c), but more recent surveys have added questions that are explicitly designed for the Government sector. This shift was driven by the need to collect data to support public sector innovation policy (Arundel, Bloch and Ferguson, 2016). Other research has used various methodologies to examine innovation in education, health and social care services (Windrum and Koch [eds.], 2008; Osborne and Brown [eds.], 2013). The OECD has supported extensive testing of questions on public sector innovation and interim guidelines for measurement OECD (2015c).

2.6.2. Innovation and non-profit institutions

2.107.  Non-profit institutions (NPIs) produce or distribute goods or services, but do not generate income or profit for the units that control or finance them. NPIs that are not part of the Government or Business enterprise sectors are classified as NPISHs. They are often non-governmental social institutions. The assignment of an NPI to the NPISH sector can change, due to an increase in the role of government or business representatives in decision-making or funding. NPISHs can also spin out businesses or exert control over business enterprises in order to serve social objectives.

2.108.  Many NPISHs seek to implement “social innovations”, defined by their objectives to improve the welfare of individuals or communities (Mulgan, Joseph and Norman 2013; Young Foundation, 2012). The same issues for measuring innovation outcomes in the General government sector apply to the NPISH sector.

2.6.3. Innovation, households and individuals

2.109.  People drive the innovation process at many levels and consequently policies often encourage individuals and collective groups in all SNA sectors to engage in innovation (OECD, 2010a). Households, including individuals and unincorporated enterprises, play a critical role for innovation from both a supply and demand perspective.

2.110.  Individuals are the ultimate providers of human and financial resources for production activities including innovation processes. As employees, individuals contribute directly to innovations attributed to their employers and can be engaged in reporting innovation data. Members of one or more households can be involved in innovations for which they are solely responsible as individuals. This can occur outside of regular employment, or through their work on a self-employed basis in unincorporated enterprises for which they are the sole or joint owner.

2.111.  Self-employed individuals, in the Household or Business sector, can have considerable involvement in innovations, although their status can also be highly transient because a promising idea can quickly lead to incorporation, potentially resulting in a transition from the Household to the Business sector. Individuals can also benefit from policy interventions such as direct funding or tax support for innovation that can lead to incorporation or other forms of registration.

2.112.  Historically, individuals have played a leading role in the development of new ideas and subsequent solutions. With the rise in research specialisation and the growth of the industrial corporation, households and individuals came to be viewed as passive consumers of innovations incorporated in purchased goods and services, rather than developers of innovations (von Hippel, 2017, 2005; von Hippel, Ogawa and de Jong, 2011). While individuals lack the organisational support to develop innovations requiring considerable investment, empirical research indicates that there is a non-negligible proportion of individuals who develop concepts and ideas into early prototypes or models, which they either make available to others or pursue further by themselves.

2.113.  Technological developments such as the Internet, 3-D printing and crowdfunding platforms can potentially support the innovation activities of individuals, although technical and commercial success is likely to result in a transition from the Household to the Business sector. Individuals can also finance the innovation activities of other members of the Household sector or start-ups, for instance through crowdfunding platforms. In many of these cases, individual funders can receive the product before it is widely marketed, becoming lead users.

2.114.  Understanding and managing the impact of innovation on individuals in their roles as employees (OECD, 2014; OECD, 2010b), asset owners, and consumers is a policy priority. Measurement could provide policy-relevant data on a range of topics, such as the effect of innovation on skills obsolescence, the willingness of individuals to trade personal data for access to free apps and networks, and factors that support trust and empower consumers to make well-informed purchasing decisions that benefit their interests. Data on the use of innovations by final consumers is also of value to business managers and policy makers. Individuals can contribute useful data for the design of new products and processes, for example behavioural data through their digital online footprint and the use of connected devices, as well as through feedback and review mechanisms. These examples point to the value of innovation measurement in the Household sector.

References

Aghion, P. and P. Howitt (1992), “A model of growth through creative destruction”, Econometrica, Vol. 60/2, pp. 323-351.

APSC (2011), State of the Service Report 2010-11, State of the Service Series, Australian Public Service Commission (APSC), Commonwealth of Australia, Canberra.

Arora, A., W.M. Cohen and J.P. Walsh (2016), “The acquisition and commercialization of invention in American manufacturing: Incidence and impact”, Research Policy, Vol. 45/6, pp. 1113-1128.

Arrow, K. (1962), “Economic welfare and the allocation of resources for inventions”, in The Rate and Direction of Inventive Activity: Economic and Social Factors, Princeton University Press, Princeton.

Arundel, A., C. Bloch and B. Ferguson (2016), “Methodologies for measuring innovation in the public sector”, conference paper for the OECD Blue Sky Forum 2016, Ghent (Belgium), 19-21 September.

Arundel, A. and D. Huber (2013), “From too little to too much innovation? Issues in measuring innovation in the public sector”, Structural Change and Economic Dynamics, Vol. 27, pp. 146-159.

Arundel, A. and K. Smith (2013), “History of the Community Innovation Survey”, in Handbook of Innovation Indicators and Measurement, Edward Elgar, Cheltenham, pp. 60-87.

Bloch, C. and M. Bugge (2013), “Public sector innovation – From theory to measurement”, Structural Change and Economic Dynamics, Vol. 27, pp. 133-145.

Cameron, L. and C. Bazelon (2013), “The impact of digitization on business models in copyright-driven industries: A review of economic issues”, Brattle Group paper for the US National Research Council, http://sites.nationalacademies.org/cs/groups/pgasite/documents/webpage/pga_063398.pdf.

Corrado, C., K. Jäger and C. Jona-Lasinio (eds.) (2016), SPINTAN Manual: Measuring Intangible Capital in the Public Sector, www.spintan.net/manual-and-reports/ (accessed 30 July 2018).

Dosi, G. (1982), “Technological paradigms and technological trajectories: A suggested interpretation of the determinants and directions of technical change”, Research Policy, Vol. 11/3, pp. 147-162.

EC et al. (2009), System of National Accounts 2008, United Nations, New York, https://unstats.un.org/unsd/nationalaccount/docs/sna2008.pdf.

Freeman, C. (1987), Technology Policy and Economic Performance: Lessons from Japan, Pinter, London.

Gault, F. (2018), “Defining and measuring innovation in all sectors of the economy”, Research Policy, Vol. 47/3, pp. 617-622, https://doi.org/10.1016/j.respol.2018.01.007.

Gault, F. (2013), “Innovation indicators and measurement: An overview”, in Handbook of Innovation Indicators and Measurement, Edward Elgar, Cheltenham, pp. 3-37.

Gault, F. (2012), “User innovation and the market”, Science and Public Policy, Vol. 39/1, pp. 118-128.

Giuri, P. et al. (2007), “Inventors and invention processes in Europe: Results from the PatVal-EU survey”, Research Policy, Vol. 36/8, pp. 1107-1127.

Griliches, Z. (1986), “Productivity, R&D, and basic research at the firm level in the 1970s”, American Economic Review, Vol. 76/1, pp. 141-154.

ISO (2017), Quality Management: Guidelines for Quality Management in Projects, ISO 10006:2017, ISO/TC 176/SC 2, International Organization for Standardization (ISO), Geneva, https://www.iso.org/standard/70376.html.

Kemp, R., J. Schot and R. Hoogma (1998), “Regime shifts to sustainability through processes of niche formation: The approach of strategic niche management”, Technology Analysis & Strategic Management, Vol. 10/2, pp. 175-198.

Kline, S. and N. Rosenberg (1986), “An overview of innovation”, in The Positive Sum Strategy: Harnessing Technology for Economic Growth, National Academies Press, Washington, DC.

Lægreid, P., P. Roness and K. Verhoest (2011), “Explaining the innovative culture and activities of state agencies”, Organization Studies, Vol. 32/10, pp. 1321-1347.

Lundvall, B.-Å. (ed.) (1992), National Innovation Systems: Towards a Theory of Innovation and Interactive Learning, Pinter, London.

Mulgan, G., K. Joseph and W. Norman (2013), “Indicators for social innovation”, in Handbook of Innovation Indicators and Measurement, Edward Elgar, Cheltenham.

National Research Council (1986), The Positive Sum Strategy: Harnessing Technology for Economic Growth, National Academies Press, Washington DC.

Nelson, R. (ed.) (1993), National Innovation Systems. A Comparative Analysis, Oxford University Press, New York/Oxford.

Nelson, R. and S. Winter (1982), An Evolutionary Theory of Economic Change, Harvard University Press, Cambridge, MA.

OECD (2016), “System innovation”, in OECD Science, Technology and Innovation Outlook 2016, OECD Publishing, Paris, https://doi.org/10.1787/sti_in_outlook-2016-9-en.

OECD (2015a), Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, OECD Publishing, Paris, http://oe.cd/frascati.

OECD (2015b), The Innovation Imperative: Contributing to Productivity, Growth and Well-Being, OECD Publishing, Paris, https://doi.org/10.1787/9789264239814-en.

OECD (2015c), The Innovation Imperative in the Public Sector: Setting an Agenda for Action, OECD Publishing, Paris, https://doi.org/10.1787/9789264236561-en.

OECD (2014), Measuring Innovation in Education. A New Perspective, OECD Publishing, Paris, https://doi.org/10.1787/9789264215696-en.

OECD (2013), “Knowledge networks and markets”, OECD Science, Technology and Industry Policy Papers, No. 7, OECD Publishing, Paris, https://doi.org/10.1787/5k44wzw9q5zv-en.

OECD (2010a), The OECD Innovation Strategy: Getting a Head Start on Tomorrow, OECD Publishing, Paris, https://doi.org/10.1787/9789264083479-en.

OECD (2010b), Innovative Workplaces: Making Better Use of Skills within Organisations, OECD Publishing, Paris, https://doi.org/10.1787/9789264095687-en.

OECD (1997), National Innovation Systems, OECD, Paris, https://www.oecd.org/science/inno/2101733.pdf.

Osborne, S. and L. Brown (eds.) (2013), Handbook of Innovation in Public Services, Edward Elgar, Cheltenham.

Rogers, E. (1962), Diffusion of Innovations, Free Press, New York.

Romer, P.M. (1990), “Endogenous technological change,” Journal of Political Economy, Vol. 98/5, Part2, pp. S71-S102.

Schumpeter, J. (1934), The Theory of Economic Development: An Inquiry into Profits, Capital, Credit, Interest and the Business Cycle, Harvard University Press, Cambridge, MA.

Simon, H. (1982), Models of Bounded Rationality: Behavioral Economics and Business Organization, Vol. 2, MIT Press, Cambridge, MA.

Simon, H. (1969), The Sciences of the Artificial, MIT Press, Cambridge, MA.

Smith, K. (2006), “Measuring innovation”, in The Oxford Handbook of Innovation, Oxford University Press, Oxford.

UN (2008), International Standard Industrial Classification of All Economic Activities (ISIC), Revision 4, United Nations, New York, https://unstats.un.org/unsd/publications/catalogue?selectID=396.

Verganti, R. (2009), Design-Driven Innovation: Changing the Rules of Competition by Radically Innovating What Things Mean, Harvard Business Press, Boston, MA.

von Hippel, E. (2017), Free Innovation, MIT Press, Cambridge, MA.

von Hippel, E. (2005), Democratizing Innovation, MIT Press, Cambridge, MA.

von Hippel, E., S. Ogawa and J.P.J. de Jong (2011), “The age of the consumer-innovator”, MIT Sloan Management Review, Sloan School of Management, Vol. 53/1, pp. 27-35.

Windrum, P. and P. Koch (eds.) (2008), Innovation in Public Sector Services: Entrepreneurship, Creativity and Management, Edward Elgar, Cheltenham.

Young Foundation (2012), “Social innovation overview: Part I – Defining social innovation”, a deliverable of the project “The theoretical, empirical and policy foundations for building social innovation in Europe” (TEPSIE), European Commission, Brussels, http://youngfoundation.org/wp-content/uploads/2012/12/TEPSIE.D1.1.Report.DefiningSocialInnovation.Part-1-defining-social-innovation.pdf.