| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Commentary |
University of Liverpool, Liverpool, L69 3BX UK
 |
Introduction |
|---|
 Top Introduction References |
|---|
Standards of scientific work are most appropriately judged by peer review, meaning scrutiny only by experts in the particular field. When it comes to judging scientific output for a particular purpose, for example resource allocation to and within universities or for individual promotion and awards, the application of rigorous peer review is a time consuming process and true experts with available time are generally a scarce commodity. Quantitative measures of scientific output have therefore been sought as a more accessible alternative. Publication counts alone do not inform about the quality of scientific output but, so one theory goes, may do so when publication is in journals that are generally regarded as being of good repute. This has led to the practice of evaluation by citation rates and impact factors.
While the intent was to provide an objective method for quantifying published research output, the significance of citation rates and journal impact factors has not always been understood, and reactions range from over-reliance to cynical distrust.
In 1955 [1], Garfield suggested that reference counting could measure "impact". The term "impact factor" was coined with the publication in 1963 of the Science Citation Index for 1961. Since that time, a database has been accrued from scientific citations that appear in the reference list of articles from a large number of scientific journals. This database is produced by the Institute for Scientific Information in Philadelphia. The references are listed to show how many times each article has been cited, and by whom, within a given period of time, and the results are published in the Science Citation Index (SCI). The annual citation rate of papers by a particular author can therefore be calculated from the author's list of publications and the SCI. The citation rate of a journal is quantified as its impact factor. The impact factor for a journal in a particular year is the number of citations received in that year to papers published in the journal in the previous 2 years, divided by the number of papers published during those 2 years. For example, if B=cites in 1998 to articles published in a particular journal in 1996 and 1997, and C=number of articles published in 1996 and 1997 in that journal, then the impact factor for the journal in 1998=B/C. A more concise, if not quite as precise, definition of the impact factor of a journal is that it is in essence the number of times the articles it publishes are cited divided by the number of articles that could be cited.
Journal impact factors are published annually in SCI Journal Citation Reports. The consequence is that journal impact factors are widely used as league tables comparing quality of journals within many scientific and medical fields. Is this a valid assumption?
Seglen [2] has pointed out that published articles actually yield a very variable contribution to a journal's impact factor. Evaluating a range of biochemical journals, he found that the most cited half of the articles were cited, on average, ten times as often as the least cited half. This large difference is obscured by the impact factor, which may be solely determined by a relatively few but highly cited articles. Chew and Reylea-Chew [3] found that 10% of papers in a group of radiological journals accounted for 50% of the citations. Using journal impact factors as the sole criterion of an author's published worth is therefore an inaccurate method of assessment.
There are also inherent inconsistencies in calculation of the journal impact factor. The SCI database includes only normal articles, notes and reviews as citable items in the denominator, but citation of all types of articles, such as editorials, letters and meeting abstracts are included in the numerator. It has been conclusively shown [4] that the inclusion of these additional items causes a substantial increase in the impact factor. Editors could raise the impact factor of a journal by frequent reference to their previous editorials, since the database makes no correction for self-citation [5], or by running a large correspondence section. One journal has been accused of manipulating its impact factor by asking authors of submitted articles to increase the number of references to the journal in their article [6]. The inclusion of review articles, which often receive more citations than many original articles, will also improve the impact factor [7]. Longer articles influence the impact factor, the citation rate being proportional to the length of the article [8]. It is widely believed that articles describing methods are cited more than the average and thus increase impact, but this only applies to a few [9]. Multiauthor and multinational articles seem to produce greater impact [9].
The defined practice of collecting citations over only the 2 years post publication has an important influence on impact factors. Journals in highly active and rapidly developing fields of research, such as molecular biology, will tend to have a short mean time from submission to publication and will inevitably have a high level of citations in the first 2 years [10]. Pari passu there will also be a tendency for self-citation. These variables will profoundly influence the impact factor and yet are unrelated to the scientific quality of the journal. In the case of the British Journal of Radiology (BJR), which has an impact factor for 1999 of 1.067, the citation rate is relatively low during the first 2 years but citations continue to accrue over a long period. The cited half-life is the number of journal publication years, going back from the current year, that accounts for 50% of the total citations received by the cited journal in the current year. Many of the papers in journals with long cited half-lives continue to be cited long after they appear in print. The half-life for BJR is 9.1 years, which is considerably longer than that of other journals. This reflects the generally long "shelf-life" of articles in the domain of radiological sciences [3]. As a consequence, papers in the BJR have a much higher number of citations than may be expected from a superficial analysis of impact factor alone. Figure 1
shows that the cumulative percentage of cites to papers published in 1990 increases approximately linearly with time. Only 12% of 1999 cites are from papers published in the previous 2 years. Thus, an impact factor based upon the previous 2 years papers will significantly underestimate the true relevence of papers published in the BJR. As the BJR has a long half-life, the total number of citations for a typical paper will be over six times greater than that deduced from a superficial inspection of the journal's impact factor.
|
Caution is therefore urged when considering, on grounds of prestige and personal advancement, submission of articles to a high impact journal. In reality, the citedness of an article is independent of a journal's impact factor [12].
It would be over optimistic to foresee significant amendments to the huge and cumbersome SCI database. Despite these limitations, is there scope for improving the present system? The usefulness of impact factors would certainly be improved by defining more rigidly the type of article included and by extending the length of time for citing. Authors could help by ensuring that their references are comprehensive and relevant [13]. If we accept that citation rates and impact factors have limitations, what considerations should determine the choice of journal when submitting an article? An important aspect in this decision is to choose a journal that will reach the intended and appropriate audience. The article will most often be written by and for workers in the same specialty. On the other hand, the aim could be to inform another specialty group of a new development relevant to their field but outside their usual ambit. In other instances, the article could be submitted to a national journal because it was especially germane to thepractice in that particular country. The on-linefacility of publications is becoming an increasingly important determinant of choice. This is particularly so if it is accessible, user-friendly—for instance by linking references to Medline—and if articles are immediately made available as soon as they have been accepted for publication and in advance of the printed version. Not withstanding these on-line developments, it is important that the printed form has a short publication time. There are other factors related to the publisher—for instance, their willingness to include colour illustrations, the cheapness of offprints, the absence of page charges, the overall style of presentation and, particularly in imaging journals, the quality of the illustrations. Our policy at the BJR is to increase the attractiveness of the journal to prospective submitters by seeking to improve all these factors.
In summary, the system of impact factors is inherently flawed and there are other considerations when choosing an outlet for publishing scientific work. Brenner [14], quoted by Steel [15], cogently summarized the main priority: "Before we develop a pseudo-science of citation analysis ... what matters absolutely is the scientific content of a paper ... and nothing will substitute for either knowing or reading it."
|
Acknowledgments |
|---|
|
References |
|---|
|
TopIntroductionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  G. Holden, G. Rosenberg, K. Barker, and P. Onghena An Assessment of the Predictive Validity of Impact Factor Scores: Implications for Academic Employment Decisions in Social Work Research on Social Work Practice, November 1, 2006; 16(6): 613 - 624. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| BJR | DMFR | IMAGING | ALL BIR JOURNALS |
