7 SURVEY METHODOLOGY AND PROCEDURES

serve as a valuable source of hypothetical examples and may help countries to develop more uniform classification practices.

7.5.Estimation procedures

463. In the process of compiling R&D statistics, various estimation procedures are used. Results from sample surveys have to be grossed up, using various methods, to correspond to the total target population. Especially in surveys of the business enterprise and government sectors, there are problems of unit and item non-response. In the higher education sector, statistics in most countries are based on a combination of surveys and estimation procedures.

7.5.1. Unit and item non-response

464.In practice, responses to R&D surveys are often incomplete, irrespective of the survey method used. Two types of missing values can be distinguished: item and unit non-responses. Unit non-response means that a reporting unit does not reply at all. The surveying institute may not be able to reach the reporting unit or the reporting unit may refuse to answer. For item non-response, a unit does answer but leaves at least one question blank or even, in an extreme case, leaves all questions but one blank.

465.Item and unit non-responses would be less of a problem if missing values were randomly distributed over all sampling units and all questions. In reality, however, both types of missing values are biased with respect to certain characteristics of the population and the questionnaire. Item nonresponse is more likely when the question is (or seems to be) difficult. Examples are the breakdown of R&D investments (land and buildings and equipment) or of R&D by type of R&D.

466.These non-responses clearly affect the comparability of the results of national and international R&D surveys. Appropriate methods to overcome this problem have to be developed and used. As different methods may lead to different results, some general recommendations should be followed. Otherwise, differences in results over time and/or among countries may arise from using different concepts to reduce the bias of item and unit nonresponses.

467.For practical as well as theoretical reasons, one recommended way to overcome the problem of item non-response is a group of methods called “imputation methods” for estimating missing values on the basis of additional information. The easiest method is to use the previous answer for the same enterprise. Another possibility is to use statistical techniques such as “hot decking”, using information from the same survey, or “cold decking”, using information from previous surveys.

7 SURVEY METHODOLOGY AND PROCEDURES

468. In the case of unit non-response, past R&D data at firm level can be used to estimate the R&D expenditures for the same firm for the current period. The evolution of sales and or employment can be used to adapt the previous figures. In cases where no previous R&D data at firm level are available, as R&D is a metric variable correlated to a certain degree with sales, a recommended method is to use the relation between the sales of the total population and the sales of the realised sample for each cell in the sample. Another method is to use employment as a variable. This procedure is based on the assumption that the ratios of R&D to sales or R&D personnel to total personnel of responding and non-responding units are identical. This assumption can be tested through non-response analysis of a representative sample of non-responding units. Even if the assumption is wrong, the bias introduced can be disregarded as long as the fraction of non-responding units is fairly small.

7.5.2. Estimation procedures in the higher education sector

469.

It is recommended that information on R&D in this sector should be based on surveys of the perform ing units, supplemented, if necessary, by estimations.

470. Often over half of the funding of R&D is given as general university funds, not earmarked for research but given for the general functioning of the university. The R&D share of these funds is often unknown to the universities themselves. To determine which part should be devoted to R&D, a variety of methods have been used:

–Central estimates not based on empirical knowledge of how time is spent on different activities.

–Time-use surveys/studies concerning the distribution of time by various categories of personnel.

–Time-use surveys/studies based on researchers’ own evaluation of their working time.

471. From the time-use studies, research coefficients are derived for use in calculating full-time equivalents on R&D (FTEs) and R&D labour costs. Other R&D costs should primarily be estimated on the basis of purpose. For example, the acquisition of research equipment and expenditures for a research laboratory should be put under research, while maintenance of teaching facilities should be put under teaching. For expenditures not clearly attributable to either research or teaching, an estimate can be made using the research coefficients as the basis of calculation.