- •Foreword
- •Table of Contents
- •1.1. A preliminary word to the user of R&D data
- •1.2. Coverage of the Manual and the uses of R&D statistics
- •Table 1.1. OECD methodological manuals
- •1.4. R&D input and output
- •1.5. R&D and related activities
- •1.5.1. Research and experimental development (R&D)
- •1.5.2. Scientific and technological activities (STA)
- •1.5.3. R&D and technological innovation
- •1.5.4. The identification of R&D in software, social sciences and service activities
- •1.5.5. R&D administration and other supporting activities
- •1.6. R&D in all fields of science and technology is covered
- •1.7. Measures of R&D inputs
- •1.7.1. R&D personnel
- •1.7.2. R&D expenditures
- •1.7.3. R&D facilities
- •1.7.4. National R&D efforts
- •1.9. Classification systems for R&D
- •1.9.1. Institutional classifications
- •1.9.2. Functional distribution
- •1.10. R&D surveys, reliability of data and international comparability
- •1.11. Government budget appropriations or outlays for R&D (GBAORD)
- •1.12. Topics of special interest
- •1.13. A final word to the user of R&D data
- •2.1. Research and experimental development (R&D)
- •2.2. Activities to be excluded from R&D
- •2.2.1. Education and training
- •2.2.2. Other related scientific and technological activities
- •2.2.3. Other industrial activities
- •2.2.4. Administration and other supporting activities
- •2.3. The boundaries of R&D
- •2.3.1. Criteria for distinguishing R&D from related activities
- •2.3.2. Problems at the borderline between R&D and education and training
- •Table 2.2. Borderline between R&D and education and training at ISCED level 6
- •2.3.3. Problems at the borderline between R&D and related scientific and technological activities
- •2.3.4. Problems at the borderline between R&D and other industrial activities
- •Table 2.3. Some cases at the borderline between R&D and other industrial activities
- •2.3.5. Problems at the borderline between R&D administration and indirect supporting activities
- •2.4.1. Identifying R&D in software development
- •2.4.2. Identifying R&D in the social sciences and humanities
- •2.4.3. Special problems for identifying R&D in service activities
- •3.1. The approach
- •3.2. The reporting unit and the statistical unit
- •3.2.1. The reporting unit
- •3.2.2. The statistical unit
- •3.3. Sectors
- •3.3.1. Reasons for sectoring
- •3.3.2. Choice of sectors
- •3.3.3. Problems of sectoring
- •3.4. Business enterprise sector
- •3.4.1. Coverage
- •3.4.2. The principal sector sub-classification
- •3.4.3. Other institutional sub-classifications
- •3.5. Government sector
- •3.5.1. Coverage
- •3.5.2. The principal sector sub-classification
- •3.5.3. Other institutional sub-classifications
- •3.6.1. Coverage
- •3.6.2. The principal sector sub-classification
- •Table 3.2. Fields of science and technology
- •3.6.3. Other institutional sub-classifications
- •3.7. Higher education sector
- •3.7.1. Coverage
- •3.7.2. The principal sector sub-classification
- •3.8. Abroad
- •3.8.1. Coverage
- •3.8.2. The principal sector sub-classification
- •3.8.3. Other institutional sub-classifications
- •3.8.4. Geographic area of origin or destination of funds
- •4.1. The approach
- •Table 4.1. Utility of functional distributions
- •4.2. Type of R&D
- •4.2.1. Use of distribution by type of R&D
- •4.2.2. The distribution list
- •4.2.3. Criteria for distinguishing between types of R&D
- •Table 4.2. The three types of research in the social sciences and humanities
- •4.3. Product fields
- •4.3.1. Use of distribution by product fields
- •4.3.2. The distribution list
- •4.3.3. Criteria for distribution
- •4.4. Fields of science and technology
- •4.4.1. Use of distribution by field of science and technology
- •4.4.2. The distribution list
- •4.4.3. The criteria for distribution
- •4.5. Socio-economic objectives
- •4.5.2. Minimum recommended breakdown
- •4.5.3. The distribution list
- •4.5.4. The criteria for distribution
- •5.1. Introduction
- •Table 5.1. R&D and indirect support activities
- •5.2. Coverage and definition of R&D personnel
- •5.2.1. Initial coverage
- •5.2.2. Categories of R&D personnel
- •5.2.3. Classification by occupation
- •5.2.4. Classification by level of formal qualification
- •5.2.5. Treatment of postgraduate students
- •5.3. Measurement and data collection
- •5.3.1. Introduction
- •5.3.2. Headcount data
- •5.3.3. Full-time equivalence (FTE) data
- •5.3.4. Recommended national aggregates and variables
- •5.3.5. Cross-classified data by occupation and qualification
- •Table 5.4. R&D personnel classified by occupation and by formal qualification
- •5.3.6. Regional data
- •6.1. Introduction
- •6.2. Intramural expenditures
- •6.2.1. Definition
- •6.2.2. Current costs
- •6.2.3. Capital expenditures
- •6.3. Sources of funds
- •6.3.1. Methods of measurement
- •6.3.2. Criteria for identifying flows of R&D funds
- •6.3.3. Identifying the sources of flows of R&D funds
- •6.4. Extramural expenditures
- •6.6. Regional distribution
- •6.7. National totals
- •6.7.1. Gross domestic expenditure on R&D (GERD)
- •Table 6.1. Gross domestic expenditure on R&D (GERD)
- •6.7.2. Gross national expenditure on R&D (GNERD)
- •Table 6.2. Gross national expenditure on R&D (GNERD)
- •7.1. Introduction
- •7.2. Scope of R&D surveys
- •7.3. Identifying target population and survey respondents
- •7.3.1. Business enterprise sector
- •7.3.2. Government sector
- •7.3.3. Private non-profit sector
- •7.3.4. Higher education sector
- •7.3.5. Hospitals
- •7.4. Working with respondents
- •7.4.2. Operational criteria
- •7.5. Estimation procedures
- •7.5.1. Unit and item non-response
- •7.5.2. Estimation procedures in the higher education sector
- •7.6. Reporting to the OECD or to other international organisations
- •8.1. Introduction
- •8.2. Relationship with other international standards
- •8.3. Sources of budgetary data for GBAORD
- •8.4. Coverage of R&D
- •8.4.1. Basic definition
- •8.4.2. Fields of science and technology
- •8.4.3. Identifying R&D
- •8.5. Definition of government
- •8.6. Coverage of government budget appropriations and outlays
- •8.6.1. Intramural and extramural expenditures
- •8.6.2. Funding and performer-based reporting
- •8.6.3. Budgetary funds
- •8.6.4. Direct and indirect funding
- •8.6.5. Types of expenditure
- •8.6.6. GBAORD going to R&D abroad
- •8.7.1. Criteria for distribution
- •8.7.2. Distribution of budgetary items
- •8.7.3. The distribution
- •8.7.4. Socio-economic objectives – SEO
- •Table 8.1. Standard key between NABS 1992 and previous OECD GBAORD objectives
- •Table 8.2. Standard key between NABS 1992 and Nordforsk GBAORD objectives
- •8.7.5. Principal areas of difficulty
- •8.8. Main differences between GBAORD and GERD data
- •8.8.1. General differences
- •8.8.2. GBAORD and government-financed GERD
- •8.8.3. GBAORD and GERD by socio-economic objectives
- •Table 1. Summary of sectors in the SNA and in the Frascati Manual
- •Table 2. Sectors and producers in the SNA
- •Table 5. Gross output and total intramural R&D
- •Table 1. Identifying health-related R&D in GBAORD
- •Table 2. Health-related R&D from performer-reported data: business enterprise sector
- •Table 3. Identifying health-related R&D by field of science and socio-economic objective
- •Table 2. Current classification of French, UK and US terminology in the Frascati Manual
- •Acronyms
- •Bibliography
- •Index by Paragraph Number
7 SURVEY METHODOLOGY AND PROCEDURES
to be established to capture all R&D, especially for units in the business enterprise sector with little R&D. These are described in more detail below.
7.3.Identifying target population and survey respondents
432.Only in a few member countries can the surveying agency make an exhaustive survey of all possible R&D performers. Generally, there are many constraints on the extent of surveys. For example, the number of respondents may have to be restricted to keep costs down; an R&D survey may have to be taken in conjunction with another survey with acceptable, but not ideal, respondents; surveys of some groups may require the participation of other agencies with different data needs and hence different questions for respondents.
433.It is not possible to make detailed recommendations on survey methods that would be equally relevant to all member countries, as the size and structure of national R&D capacities vary widely. Suggestions are given for the business enterprise, government, private non-profit and higher education sectors, although it is recognised that some countries use different systems of sectoring for surveying and reporting data. For example, some countries undertake surveys of enterprises, institutes and higher education teaching establishments and redistribute the institutes among the four standard sectors of performance.
434.Hospitals/healthcare institutions are a special category. They carry out R&D which may relate to any of the four standard sectors of performance. Healthrelated research has increased significantly in recent years, and it is worth reviewing survey coverage to ensure that R&D data are collected for all hospitals and healthcare units that are liable to undertake such activities, not only university hospitals and other research-intensive hospitals but also general hospitals and other healthcare units (ISIC 8512 and 8519). In several countries, some categories of publicly funded hospitals/healthcare units may not be officially permitted to use their funds for R&D but may nevertheless host some research activities. If possible, the major research-performing units should be surveyed following the standard national timetable. For general hospitals and other healthcare units, benchmark surveys should be made at least every decade and methods of estimation established for the intervening years.
7.3.1. Business enterprise sector
435. The enterprise is recommended as the main statistical unit in the business enterprise sector (see also Chapter 3, Section 3.4.2). Some enterprises perform R&D on a regular basis from year to year, and they may have one or several R&D units. Other enterprises perform R&D only occasionally. They may be engaged in a project in one year and not perform any R&D in the next. This R&D is often performed by people from various parts of the enterprise on a
FRASCATI MANUAL 2002 – ISBN 92-64-19903-9 – © OECD 2002 |
127 |
7 SURVEY METHODOLOGY AND PROCEDURES
project basis without a formal R&D organisation. In the general Frascati Manual definition of R&D, the criterion of “creative work undertaken on a systematic basis” is fulfilled by a project which has specific goals and a budget.
It is recommended that all enterprises performing R&D, either continuously or occasionally, should be included in R&D surveys.
436.There are at least two feasible approaches for establishing the survey population of the business enterprise sector. One is to take a census-based survey of large enterprises and a sample of smaller ones belonging to a certain population (in terms of industry and size class) from the entire sector in order to identify R&D performers and request the information from them. The choice of enterprises should be based on a business register of good quality. In this approach, R&D performed in the past in the enterprise is not considered. This is the approach followed in innovation surveys.
437.Surveys of this kind will cover a large number of enterprises and are expensive if applied to all industries and all enterprises regardless of size. It is therefore necessary to limit the target population in terms of size of enterprise and industries covered. This normally leads to the systematic exclusion of very small enterprises and enterprises in certain less R&D-intensive industries. When the sample size is very small, estimates may be less reliable, owing to raising factors. In practice, no member country follows this approach strictly.
438.In R&D surveys for the business enterprise sector, most member countries use the second approach, i.e. they try to survey all enterprises known or assumed to perform R&D. The survey is based on a register of R&D- performing enterprises. The sources of this register include lists of enterprises receiving government grants and contracts for R&D, lists of enterprises reporting R&D activities in previous R&D surveys, in innovation surveys or other enterprise surveys, directories of R&D laboratories, members of industrial research associations, employers of very highly qualified personnel and lists of enterprises claiming tax deductions for R&D. Several countries only use this kind of information to identify R&D performers.
439.It is very difficult to maintain completely up-to-date registers of enterprises that occasionally perform R&D from these sources. This may lead to undercoverage of R&D in small or medium-sized enterprises. The effect on total business enterprise R&D, however, is not significant, as the large R&D performers are included in any case.
440.To improve the coverage of R&D surveys that use this approach, many countries use a combination of these approaches, i.e. they systematically take a census/sample survey to collect information on R&D from enterprises not
128 |
FRASCATI MANUAL 2002 – ISBN 92-64-19903-9 – © OECD 2002 |
7 SURVEY METHODOLOGY AND PROCEDURES
included in registers of R&D-performing enterprises. For cost reasons, such surveys are limited in terms of industries covered and enterprise size. The limitations mainly concern the services sectors, as there is little experience with surveying their R&D activities. Enterprises with a very low probability of performing R&D should be excluded in order to reduce the response burden. The advantage of this approach is that it considerably reduces the uncertainty in estimating figures for the target population as compared to the pure sampling approach described above, which does not take previous R&D into account. Its disadvantage is the cost, which may make it difficult to apply in bigger countries.
441.It is therefore recommended:
–To include in R&D surveys of the business enterprise sector all firms known or supposed to perform R&D.
–To identify R&D performers not known or supposed to perform R&D by a census/sample of all other firms in the industries listed below. In principle, enterprises in all size classes should be included, but if a cut-off point is necessary, it should be at ten employees.
442.The following industries should be included:
Industry |
ISIC Rev. 3/NACE Rev. 1 |
|
|
Mining |
14 |
Manufacturing |
15-37 |
|
|
Utilities, construction |
40, 41, 45 |
Wholesale |
50 |
|
|
Transport, storage and communication |
60-64 |
Financial intermediation |
65-67 |
|
|
Computer and related activities |
72 |
R&D services |
73 |
|
|
Architectural, engineering and other technical activities |
742 |
|
|
In addition, other sectors, for example agriculture (ISIC Rev. 3, Divisions 01, 02, 05), should be covered in countries with significant amounts of research in these sectors.
7.3.2. Government sector
443.Units to include in surveys are:
–R&D institutes.
–R&D activities of general administrations of central or state government, statistical, meteorological, geological and other public services, museums, hospitals.
FRASCATI MANUAL 2002 – ISBN 92-64-19903-9 – © OECD 2002 |
129 |
7 SURVEY METHODOLOGY AND PROCEDURES
– R&D activities at the municipality level.
The best way to survey is to send questionnaires to all units known or assumed to perform R&D.
444.There may be several ways of updating lists of R&D-performing units, such as business registers, directories of R&D-performing units, research associations, bibliometric sources, requests for updates from administrative bodies, etc.
445.It is especially difficult to identify R&D activities at the municipality level owing to the large number of units, the small number of likely R&D performers and difficulties in the interpretation of the concept of R&D. Lists of R&D performers usually do not include these units. It may be worthwhile to make an effort to identify R&D performers in large cities.
7.3.3. Private non-profit sector
446. The sources for identifying possible survey respondents are mainly the same as for the government sector. Register information may be less comprehensive and could be completed by information from researchers or research administrations. This sector may be more relevant for surveys on R&D funding.
7.3.4. Higher education sector
447.
The surveys and estimation procedures (see below) should cover all universities and corresponding institutions, especially those awarding degrees at the doctorate level. Other institutions in the sector known or assumed to perform R&D should also be included.
448. Identification of these institutions is generally relatively easy. If possible, it is often preferable to use smaller units, such as departments or institutes of the university, as statistical units.
7.3.5. Hospitals
449. Some countries may find it satisfactory to include hospitals and healthcare institutions in regular R&D surveys, using the standard questionnaire for the sector concerned. Indeed, this may be the only option for hospitals and other healthcare units in the business enterprise sector. In this
130 |
FRASCATI MANUAL 2002 – ISBN 92-64-19903-9 – © OECD 2002 |