Ординатура / Офтальмология / Английские материалы / Eye Movements A Window on Mind and Brain_Van Gompel_2007

14 R. P. G. van Gompel et al.

Table 4

Most cited articles with ‘eye(-)movement(s)’ in title or keywords in any of the databases in Web of Knowledge

Note: Points: For each respondent N = 44 , the most influential paper counted as 3 points, the second most influential paper as 2 points, and the third most influential paper as 1 point.

Times mentioned as most influential: Number of respondents who indicated that this publication has been most influential (i.e. it received 3 points).

Citation count: Number of citations (including incorrect citations) in all databases in Web of Knowledge, searched on 12 May 2006.

Area: Area according to the classification in Table 1.

The discrepancy between the citation counts and responses to the questionnaire might have several causes. First, these citation counts do not include any publications other than journal articles (and occasionally, conference proceedings), explaining why Yarbus (1967), Rayner and Pollatsek (1989), and Robinson (1975) do not appear in Table 4. Clearly, this is a limitation in the use of this database (and many others). However, it should be noted that most of the publications mentioned by our respondents were indeed journal articles, and this consequently cannot account for all of the discrepancy. Second, citation counts may be more biased towards early publications than the opinions of specialists, since earlier publications have had more chance to accumulate citations. But there is little evidence for this. The mean year of publication in Table 4 is 1985, which is comparable to 1986 in Table 2 and later than 1981 in Table 3. Thus, the dates of the publications mentioned by the respondents and those which have most citations are not that dissimilar, though clearly, we would wish to argue that our survey data is likely to be more “up to date” in terms of current impact. Third, it is possible that the type of research that was most common amongst our group of correspondents differed from the type of research that receives most citations. Most of the articles in Table 4 are concerned with physiology and clinical studies, but as we mentioned before, researchers from these areas were probably underrepresented in our sample. It may be that articles on physiology and clinical studies of eye movements receive a large number of citations because many people work in these fields. Similarly, there were no articles on reading in Table 4, while reading researchers were the largest group among questionnaire respondents.

Since most (though certainly not all) of our respondents were social scientists (rather than clinicians and medical researchers), we might expect to see a clearer correspondence between citation counts and the respondents’ responses if we consider social science publications only. In order to check this, we ran a search in Web of Knowledge using the social sciences database only. The 10 most cited journal articles are shown in Table 5. We now see two publications, Frazier and Rayner (1982) and Rayner (1998), that were frequently mentioned as influential by the respondents to the questionnaire. This suggests that some of the discrepancy between the citation counts and the opinions of our respondents resulted from a “social science bias” on the part of our respondents. However, it nonetheless does not account for all of the discrepancy: Table 5 shows that many of the most cited social science articles were not mentioned by our respondents either.

It seems that none of the above explanations can completely account for the discrepancy between citation counts and responses to our questionnaire. It appears that citation counts may therefore be a rather poor reflection of the actual significance of particular publications. One reason for this may be that some well-cited publications have ‘eye(-)movement(s)’ in their title or keywords, but do not have this as their main focus of their investigation. For example, many researchers would agree that MacDonald, Pearlmutter, and Seidenberg (1994) has been influential in the area of sentence processing but their paper has not been influential for the field of eye-movement research. Other

Note: Points: For each respondent N = 44 , the most influential paper counted as 3 points, the second most influential paper as 2 points, and the third most influential paper as 1 point.

Times mentioned as most influential: Number of respondents who indicated that this publication has been most influential (i.e. it received 3 points).

Citation count: Number of citations (including incorrect citations) in all databases in Web of Knowledge, searched on 12 May 2006.

Area: Area according to the classification in Table 1.

publications may receive many citations because they provide a representative example of a particular type of research, method, or theory, without being the original, groundbreaking work that influenced subsequent research. Finally, of course, there is also the possibility that some publications are often cited because they provide a good example of a particularly outrageous claim, a theory that is falsified or research that is flawed (readers will have to come up with their own examples falling into this category!).

In conclusion, it is clear that while there is some relationship between citations and influence, the most heavily cited journal articles are not necessarily the most influential publications. This suggests that we should be cautious when interpreting citation counts and that citation counts are not necessarily the optimal way of quantifying how influential particular publications may be. Of course, questionnaire studies also have limitations. It is difficult to obtain a sufficiently large and representative sample of respondents. Furthermore, the respondents may not always be entirely objective and may be biased towards their own publications, those from within their research group or area, or those that are part of the theoretical framework in which they are working. When considering the impact of publications, both methods have their value: most researchers would probably agree that the majority of publications listed in Tables 2–5 provide a representative sample of influential publications in the field of eye-movement research.

2.2. Journals

We also asked our respondents to list the journals that they considered best for keeping up to date with developments in eye-movement research. Table 6 shows the top 10 most mentioned journals. It is clear that Vision Research is the favourite journal of most respondents, followed at a distance by two of the Journals of Experimental Psychology (HPP & LMC). We see that Vision Research also has many articles with ‘eye(-)movement(s)’ in the title (in the period: 2001–2005), whereas both Journals of Experimental Psychology have few. All journals in Table 6 have a relatively high impact factor (see Tables 6–8 for the definition), mostly higher than 2.

Table 7 ranks all journals in Web of Knowledge according to the number of articles that have ‘eye(-)movement(s)’ in their title in the period 2001–2005. Investigative Ophthalmology & Visual Science, which has a special section on eye movements, strabismus, amblyopia, and neuro-ophthalmology, contains the most articles. It is closely followed by Perception, with 79. Interestingly, neither journal was mentioned very often by respondents to the questionnaire; in fact, Investigative Ophthalmology & Visual Science was never mentioned. This may be because the majority of our respondents were social scientists, but this does not explain why none of the nine respondents who said they worked on physiology and clinical studies of eye movements mentioned it. Perception has a fairly low impact factor, so this may explain why this journal was seldom mentioned by our respondents.

Table 8 ranks all those journals that are in the social sciences database (but not in the science or arts & humanities databases). All eye-movement articles in Perception were in the social sciences, so this journal now ranks highest. By contrast, none of the eye-movement articles in Investigative Ophthalmology & Visual Science are social

Table 6

Journals considered to be the best for keeping up to date with developments in eye-movement research by respondents to the questionnaire

Note: Times mentioned: Number of respondents N = 44 who mentioned the journal.

‘Eye(-)movement(s)’ in title: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 2001–2005, searched on 15 May 2006.

Impact factor

= Number of times that articles published in the journal in 2003–2004 were cited during 2005 Number of articles published in the journal in 2003–2004

Table 7

Journals with most articles that have ‘eye(-)movement(s)’ in the title during 2001–2005 in all Web of Knowledge databases

Note: Times mentioned: Number of respondents N = 44 who mentioned the journal.

‘Eye(-)movement(s)’ in title: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 2001–2005, searched on 15 May 2006.

Impact factor

= Number of times that articles published in the journal in 2003–2004 were cited during 2005 Number of articles published in the journal in 2003–2004

Table 8

Journals with most articles that have ‘eye(-)movement(s)’ in the title during 2001–2005 in the social sciences database in Web of Knowledge

Note: Times mentioned: Number of respondents N = 44 who mentioned the journal.

‘Eye(-)movement(s)’ in title: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 2001–2005, searched on 15 May 2006.

Impact factor

= Number of times that articles published in the journal in 2003–2004 were cited during 2005 Number of articles published in the journal in 2003–2004

science, so this journal is absent from Table 8. In Table 8 we now see some correspondence between the number of articles with ‘eye(-)movement(s)’ in their title and number of times the journal was mentioned by our respondents, although the correspondence is still not great. One possible reason for this is that the impact factor of some social science journals that contain many eye-movement articles (especially the International Journal of Psychology and, to a lesser extent, Perception) is relatively low. Consequently, the quality of the eye-movement articles published in these journals may not be as high as those in other journals. Table 9 suggests another possible reason. For three 10-year periods, it lists the number of articles with ‘eye(-)movement(s)’ in their title together with the number of articles with eye(-)movement(s) per 1000 articles published for seven journals. We see that the per mille of eye-movement articles is relatively high for the five journals that were most often mentioned by our respondents (see Table 6: Vision Research,

Journal of Experimental Psychology: HPP, Journal of Experimental Psychology: LMC, Journal of Neurophysiology, Perception & Psychophysics). The exception is the Journal of Experimental Psychology: LMC for the period 1976–1985, but this is partly because it did not exist under this name until 1982. By contrast, this measure is quite low for Investigative Ophthalmology & Visual Science and the International Journal of Psychology. This suggests that our respondents assigned more importance to the relative number of eyemovement articles in a journal rather than the absolute number. Certain journals are therefore seen to have a greater affinity for eye-movement research.

It is clear from the above that eye-movement articles are published across a wide variety of journals. This is of course related to the fact that eye-movement research is a

Table 9

Number of articles with ‘eye(-)movement(s)’ in title for different journals for 10-year periods from 1976 onwards.

Note: Number: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 2001–2005, searched on 15 May 2006.

‰: Number of articles with ‘eye(-)movement(s)’ in title per 1000 articles published during each period.

very diverse field which encompasses many different disciplines, with many researchers using eye-movement methodology as a research tool for investigating various different kinds of cognitive and perceptual processes, rather than necessarily focussing on the characteristics of eye movements per se. There is no single journal that includes all these different disciplines. Vision Research, which is the favourite journal for most of the respondents to the questionnaire, contains articles on physiology and clinical studies, attention, scene perception, modelling, and low-level reading processes, and so it includes a fairly substantial part of the full range of eye-movement research. However, it normally does not have any articles that use eye-movements to investigate higher levels of language processing (syntax, semantics, and discourse processing). Of the other journals that were often mentioned by our respondents, both Journals of Experimental Psychology include a wide variety of eye-movement research in experimental psychology, but no physiological and clinical studies involving eye movements. The Journal of Neurophysiology includes physiological studies, but few behavioural studies, while Perception & Psychophysics focuses on particular behavioural areas, but not others.

Until very recently, there was no specialised eye-movement journal that contained research from all the different eye-movement disciplines, but this has changed with the introduction of the online Journal of Eye Movement Research (http://www.jemr.org). One might argue that such a specialised journal is not needed, because eye-movement research is too diverse and the different disciplines are too unrelated to bring them together in a single journal. Many researchers only read articles from their own eye-movement area, and hence articles outside this area may be irrelevant. However, evidence suggests that this assumption is incorrect. First, it is well known that many research areas develop as a consequence of cross-fertilisation between different areas. This is also the case

for eye-movement research. For example, researchers using eye-tracking to investigate spoken language processing have been very interested in eye-movement research investigating attention and scene perception (Part 6 in this book). Eye-movement behaviour during reading and scene perception has interesting commonalities and differences, and is therefore often compared (e.g., Rayner, 1998). Furthermore, eye movements have been modelled for both reading and scene perception, and findings in these two areas can inform one another (see, for example, the comment in the chapter by De Graef, Chapter 7). Finally, many disciplines in eye-movement research have benefited from findings from physiological and clinical studies, and, in turn, physiological and clinical studies have drawn on findings from behavioural studies. In sum, there is significant overlap between the different disciplines in eye-movement research and cross-fertilisation has often been very fruitful. However, because eye-movement research is distributed across different journals and many researchers mainly read journals in their own specialised field, there is a risk that many eye-movement researchers are unaware of much of the relevant research and findings outside their own field. This is likely to inhibit the integration of research across the different disciplines and therefore hold back progress. Perhaps an even greater concern derives from the increased availability of eye-movement technology to researchers with little general background in the area and the consequent risk of their failing to appreciate some of the technical questions and considerations that need to be addressed when planning and analysing their experiments. A specialised eye-movement journal that brings together research from different disciplines would be a place where they could find answers to their questions about the various eye-tracking technologies, experimental methods, and data analysis techniques.

Second, the fact that ECEM conferences have been so successful strongly suggests that bringing together eye-movement researchers from different disciplines is worthwhile. One of the reasons why many researchers go to these conferences is because it enables them to learn about eye-movement research outside their own specialised area. Third, the fact that most ECEM conferences result in an edited volume that contains eye-movement research from all disciplines and the fact that these volumes are read by many people in the field indicate that there is a need to bring together eye-movement research from the different disciplines. A specialised eye-movement journal serves this function. There are, of course, many factors that determine the success of a journal, such as the quality of the publications, speed of the review and publication process, the editorial board, readership, its availability in libraries and on the web, advertising, publicity, inclusion in search databases, and the confidence that researchers have in it. But we hope the Journal of Eye Movement Research will facilitate integration between different eye-movement disciplines and advance eye-movement research in general.

2.3. Development of the area over time

The chapters by Wade, Land, and Westheimer in this book discuss historical developments in eye-movement research in a qualitative way. Using quantitative data from our database

searches and questionnaire, we attempted to gain an insight into the development of eyemovement research as a scientific field of interest in the period from 1970 onwards. The 1970s are often seen as the start of the modern area of eye-movement research, so it seems appropriate to take a closer look at developments in the past 35 years or so.

The first question we addressed is whether the number of journal articles on eye movements has grown during that time period. This would give us an idea of whether research in the area is on the increase (as most of us would like to think) or decline. As mentioned in the previous section, Table 9 shows the number of eye-movement articles for seven representative journals for three 10-year periods. The number of articles with ‘eye(-) movement(s)’ in the title went up in all journals except Perception & Psychophysics, for which it stayed roughly the same. Overall, there appears to be a steady increase across the seven journals. However, when we look at the per mille measure, then there is no clear increase in the proportion of eye-movement articles. Together, this indicates that more and more eye-movement articles are being published in the selected journals, but this appears to be simply a reflection of the fact that these journals accept more and more articles, regardless of whether they involve eye movements.

Of course, a few representative journals do not tell the whole story. The picture may be different for other journals, and over the years, new journals publishing eye-movement research may appear, with a resultant increase in eye-movement articles. Table 10 presents the number and per mille articles with ‘eye(-)movement(s)’ in the title for 5-year periods from 1971 onwards. We have listed the results from all databases in Web of Knowledge (social science, science, and arts & humanities) plus the results from the social science database only. The results from all databases combined show a clear and steady increase in the number of published eye-movement articles. However, again when we look at the per mille of all articles, there is no obvious increase. In fact, the per mille eye-movement

Table 10

Number of articles with ‘eye(-)movement(s)’ in title for 5-year period from 1971 onwards

Note: Nr. eye-movement articles: Number of articles with ‘eye(-)movement(s)’ in title.

‰ of all articles: Number of articles with eye(-)movement(s) in title per 1000 articles published during each period.

articles was highest between 1971 and 1975, and there seems to be a slight dip in the per mille eye-movement articles in the 1980s. We conclude that the number of eye-movement articles is going up, but this increase is proportional to the total number of articles being published. There is no evidence that eye-movement research has become more dominant or productive relative to other areas. However, this consistent pattern does reinforce the notion that eye-movement research is a stable and reliable form of investigation rather than a passing scientific or technological fad.

The picture is slightly different when we consider journal articles in the social sciences only. As before, we see an increase in the number of eye-movement articles since 1971 (though again, there is a slight dip in the 1980s). But we also see that the per mille eye-movement articles in the social sciences has gradually increased since the late 1980s. In particular, the number of eye-movement articles between 2001 and 2005 is much larger than in any previous period. This clearly suggests that in recent years, eye-movement research has become more productive or more popular in the social sciences. The fact that many social scientists responded to our questionnaire and that they are well represented in the chapters in this book may be a further reflection of the gaining popularity of eye-movement research in this area.

We also tried to gain an impression of whether the countries where eye-movement research is conducted have changed over time. Table 11 ranks countries that produced most journal articles with ‘eye(-)movement(s)’ in the title for the period 1976–1980 in all databases in Web of Science. The United States produced by far the largest number

Table 11

Countries producing most articles with ‘eye(-)movement(s)’ in the title. Web of Science, all databases, 1976–1980

Note: 7.3% of articles did not specify the country of origin.

Nr articles: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 1976–1980, searched on 15 May 2006.

% of all articles: Percentage of articles with ‘eye(-)movement(s)’ in title out of all eye-movement articles from all countries.

Nr articles/million population: Number of articles with ‘eye(-)movement(s)’ in title per million population. Population data from Bradfield, Keillor, and Pragnell (1976).

of eye-movement articles during this period: 47.6% of all relevant articles. It is followed at a long distance by West Germany and England. Of course, one might argue that this is not a fair comparison, as the United States has a much larger population than the other countries in the table. If we consider the number of eye-movement articles per million population for all countries in Table 11, both Switzerland and Canada rank higher than the United States.

Table 12 shows the same data for the period 2001–2005. We see that there are not many striking differences with that of the period 1976–1980. The position of the United States has become slightly less dominant, as indicated by the percentages relative to all eye-movement articles (40.2% vs. 47.6% of eye-movement articles in 2001–2005 vs. 1976–1980 respectively), while the percentage of eye-movement articles produced in Germany, Japan, and England has clearly increased. Czechoslovakia and Switzerland have disappeared from the top 10 (Czechoslovakia has, of course, ceased to exist as a country), whereas The Netherlands and Scotland are now included. Interestingly, the number of eye-movement articles per million population is very high for The Netherlands and even more so for Scotland.

Table 13 presents the results for the period 1976–1980 from the social sciences database only. The United States produces over half of all eye-movement articles in the social sciences during this period, followed at a long distance by England and Canada. It is noteworthy that Czechoslovakia ranks fourth in this table. Israel has a high number of eye-movement articles per million population, but it should be noted that its total number

Table 12

Countries producing most articles with ‘eye(-)movement(s)’ in the title. Web of Science, all databases, 2001–2005

Note: 4.2% of articles did not specify the country of origin.

Nr articles: Number of articles with ‘eye(-)movement(s)’ in title in all databases in Web of Knowledge for the period 2001–2005, searched on 15 May 2006.

% of all articles: Percentage of articles with ‘eye(-)movement(s)’ in title out of all eye-movement articles from all countries.

Nr articles/million population: Nr of articles with ‘eye(-)movement(s)’ in title per million population. Population data from Bruinsma, Koedam, Dilworth, and Stuart-Jones (2005).