Quantification Quandary
02/12/10 16:04 Filed in: GenomeWeb Daily Scan
Submitted by S. Pelech - Kinexus on Thu, 12/02/2010 - 16:04.
The most frequently used measures of scientific impact tend to be a combination of the general impact factor of the scientific publication in which the work in question appears and the number of times the publication is subsequently cited by other scientists in their own published work. These data are commonly used to determine the scientific impact of an investigator when it comes to career advancement and the awarding of grant-in-aid funds. One significant problem is that it may take several years post-publication before a reasonable citation count can be established for a scientific manuscript, so the general impact score of the journal in which it appears actually predominates the assessment.
In my observations and experiences, the premier scientific journals tend to be very trendy in terms of what they choose to publish. If someone works and publishes something truly novel, it is highly unlikely to appear in such journals. If one is engaged in research in a "hot" area, then there is also a much greater prospect that other scientists will bother to read and perhaps cite a publication in this field. This generates a lot of what I call "me too research" down well trodden paths. While it may seem that this published work is apparently having an impact, the real question is whether it is truly advancing science in new and productive directions.
For most academic scientists, their key objective is to create new knowledge, but this need not have immediate practical outcomes. The expectation of the general public that actually funds research is that there should be some clear practical results with applications in, for just a few examples, medicine, agriculture, and energy production. Such translation of basic research can take decades so the real measure of scientific impact of even a body of work from an investigator will rarely become evident within a period that can have a bearing on their career progression and especially their grant funding. Nobel prizes almost never are awarded to young investigators.
In view of this clear difficulty in assessment of the importance of the scientific contributions of an individual investigator and their proposed studies for allocation of grant funding as well as many other caveats with other aspects of this process, it is probably time for a complete overhaul of the system. The continuing trend has been to support a smaller percentage of the biomedical researchers with larger grants. I advocate that more people should be funded with smaller grants. With smaller allocations, investigators will form more transient, but natural alliances and collaborations that are much more likely to be diversified, fruitful, and require much less counter-productive administration.
Link to the original blog post.
The most frequently used measures of scientific impact tend to be a combination of the general impact factor of the scientific publication in which the work in question appears and the number of times the publication is subsequently cited by other scientists in their own published work. These data are commonly used to determine the scientific impact of an investigator when it comes to career advancement and the awarding of grant-in-aid funds. One significant problem is that it may take several years post-publication before a reasonable citation count can be established for a scientific manuscript, so the general impact score of the journal in which it appears actually predominates the assessment.
In my observations and experiences, the premier scientific journals tend to be very trendy in terms of what they choose to publish. If someone works and publishes something truly novel, it is highly unlikely to appear in such journals. If one is engaged in research in a "hot" area, then there is also a much greater prospect that other scientists will bother to read and perhaps cite a publication in this field. This generates a lot of what I call "me too research" down well trodden paths. While it may seem that this published work is apparently having an impact, the real question is whether it is truly advancing science in new and productive directions.
For most academic scientists, their key objective is to create new knowledge, but this need not have immediate practical outcomes. The expectation of the general public that actually funds research is that there should be some clear practical results with applications in, for just a few examples, medicine, agriculture, and energy production. Such translation of basic research can take decades so the real measure of scientific impact of even a body of work from an investigator will rarely become evident within a period that can have a bearing on their career progression and especially their grant funding. Nobel prizes almost never are awarded to young investigators.
In view of this clear difficulty in assessment of the importance of the scientific contributions of an individual investigator and their proposed studies for allocation of grant funding as well as many other caveats with other aspects of this process, it is probably time for a complete overhaul of the system. The continuing trend has been to support a smaller percentage of the biomedical researchers with larger grants. I advocate that more people should be funded with smaller grants. With smaller allocations, investigators will form more transient, but natural alliances and collaborations that are much more likely to be diversified, fruitful, and require much less counter-productive administration.
Link to the original blog post.