Archive

Posts Tagged ‘ethics in science’

Indices again

April 23, 2011 1 comment

Via Nanopolitan: Current Science carried a letter by Diptiman Sen and S. Ramasesha, two physicists at the IISc, pointing out that the h-index is not a good `scientometric’ index. Unfortunately, the arguments they have used to establish that are not all equally good. In particular they suggest that the Nobel Laureate V. Ramakrishnan has a low h-index. This was picked upon by two other scientists who point out the actual figures are not particularly low. And in between poor arguments and rejoinders, many other important arguments against using the h-index got lost.

Some of these other arguments were given here and here, and many more can be found all over the cyberspace and blogosphere. My arguments against scientometric indices in general, and h-index (and journal impact factor) in particular, are similar to those given in these links, but also try to take into account the conditions special doing science in India. These are, not necessarily in any particular order, are the following.

1. Most (or all?) such indices are based on only the number of citations (total or per year), so no matter how a metric is designed, it is only the number of actual citations which enters the metric — so any of these metrics is ultimately calculated from only one parameter. Or perhaps two, since the rate of growth of citations may be included. Some metrics would include the number of papers (total or per year) as well. That is another parameter, but the number of citations is not completely independent of the number of papers, and the number of papers is usually not a good measure of their quality, so including that number does not improve the quality of the index either.

2. Even if different ways of using the citation count (and paper count) lead to qualitatively different indices, the standard indices are still numbers associated with only one individual (or one institution), the one being evaluated. This cannot make any sense, since high or low values may be systemic. For example, mathematics has fewer papers than  medicine, than even specialized branches of medicine like oncology, and consequently fewer number of citations as well. So any index that can be applied to both mathematics and medicine will have to take into account its behavior specific to that field, and thus require some sort of comparison within the field. This is never done as far as I can gather, either in the construction or in the usage of these indices.

3. Even if we can make a comparative index, for example by taking ratios or percentiles within a field, that is not likely to hold up against historical data.¬† That is, given some index — h, g, or whatever — for some string theorist, we can come up with a `normalized’ one by taking its ratio with the same index for Witten, but the same normalization is not likely to make any sense for Born or Einstein, say. Of course they were not string theorists, and neither is `normal’ a word one should use for any index related to Witten. Still, the explosion of citations is a relatively recent phenomenon, and related to the explosion of papers, so the variation of any index with time — for individuals as well as within fields — need to be taken into account.

4. Many scientists work across disciplines, many more work across subfields. It makes no sense for such people’s work to be evaluated by a single index, as the index may have different ranges in the different fields or subfields. For example, someone working mostly in mathematics and occasionally in string theory may end up with an index which is low compared to string theorists and high compared to mathematicians. How should something like that be used?

5. Indices are used for different purposes at different career stages. So it does not make sense to use the same index for people at different stages of their career.

6. There may be `social’ factors in the rise of citation count of specific papers or individuals — some are obvious and `nearly academic’ ones, like the popularity or currency of a field or a problem — the bandwagon effect. Then there is the `club’ effect — I cite you, you cite me, friends cite friends — which can work wonders in small subfields. There may also be less academic and more career-oriented reasons — it is very likely that papers cite probable referees more often, so that a paper does not come back for revisions simply because `relevant literature was not cited.’ I would not be surprised if this mechanism gets reinforced for people with many collaborators — a paper might be rejected if it did not cite the papers of the referee’s collaborators.

There are also several issues special to Indian science, which have to do with how appointments and promotions are usually effected in India. As noted by G. Desiraju in the letter to Current Science,

it was possible, in the days before we had scientometric indicators, for committees of wise men to simply declare an incompetent as an outstanding scientist.

Unfortunately, it is still possible. But that is another discussion.

BT Brinjal: safe to eat?

September 24, 2010 1 comment

It seems that six Indian academies have declared bt brinjal safe. It is not at all clear if they have done so by making any kind of experiment on the seeds or the plant — my guess would be that they have not. More accurately, my guess is that none of the members of any of these academies have actually done a thorough testing of the seeds and the plants of bt brinjal in their laboratories. They may have asked their students or post-docs to run a series of tests, but I would be doubtful of even that.

More importantly, they have only said, apart from some babble about how “all genetically modified (GM) items pose a risk if the science behind them is flawed” which serves only to protet their own backsides, that they have “found no evidence that the protein used in creating Bt brinjal, Cry1Ac, is unsafe”.

That is not the issue! The genetically modified (GM) crop carry a special gene called a terminator gene, included in crops developed by Monsanto, like this one. The terminator gene ensures that the plant grown from the GM seeds will not be fertile themselves, and the seeds they produce cannot be used to produce the next generation of crops. This infertility is built into the genes, and these genes can migrate to nearby plants, rendering them infertile. It would seem that any gene that makes a plant infertile is not likely to propagate very far. But a new gene does not necessarily manifest itself in one generation. What if the genes transmitted through cross-pollination show up a few generations later? when it may be too late to do anything? And could such genes affect the fertility of animals who eat the plant or its seeds?

The academies do not seem to have said anything about these. I suspect they have no idea about these issues either. In fact I would be very surprised, based on the extrapolation of behaviour of my colleagues, if more than a handful of the scientists who comprise our academies ever think, except in terms of getting more funding, about the relevance of their research to society.

The least the academies can do is to put up the report as a pdf file somewhere, and let the people decide if bt brinjal should be allowed in.