Archive for April, 2011

Indices again

April 23, 2011 1 comment

Via Nanopolitan: Current Science carried a letter by Diptiman Sen and S. Ramasesha, two physicists at the IISc, pointing out that the h-index is not a good `scientometric’ index. Unfortunately, the arguments they have used to establish that are not all equally good. In particular they suggest that the Nobel Laureate V. Ramakrishnan has a low h-index. This was picked upon by two other scientists who point out the actual figures are not particularly low. And in between poor arguments and rejoinders, many other important arguments against using the h-index got lost.

Some of these other arguments were given here and here, and many more can be found all over the cyberspace and blogosphere. My arguments against scientometric indices in general, and h-index (and journal impact factor) in particular, are similar to those given in these links, but also try to take into account the conditions special doing science in India. These are, not necessarily in any particular order, are the following.

1. Most (or all?) such indices are based on only the number of citations (total or per year), so no matter how a metric is designed, it is only the number of actual citations which enters the metric — so any of these metrics is ultimately calculated from only one parameter. Or perhaps two, since the rate of growth of citations may be included. Some metrics would include the number of papers (total or per year) as well. That is another parameter, but the number of citations is not completely independent of the number of papers, and the number of papers is usually not a good measure of their quality, so including that number does not improve the quality of the index either.

2. Even if different ways of using the citation count (and paper count) lead to qualitatively different indices, the standard indices are still numbers associated with only one individual (or one institution), the one being evaluated. This cannot make any sense, since high or low values may be systemic. For example, mathematics has fewer papers than  medicine, than even specialized branches of medicine like oncology, and consequently fewer number of citations as well. So any index that can be applied to both mathematics and medicine will have to take into account its behavior specific to that field, and thus require some sort of comparison within the field. This is never done as far as I can gather, either in the construction or in the usage of these indices.

3. Even if we can make a comparative index, for example by taking ratios or percentiles within a field, that is not likely to hold up against historical data.¬† That is, given some index — h, g, or whatever — for some string theorist, we can come up with a `normalized’ one by taking its ratio with the same index for Witten, but the same normalization is not likely to make any sense for Born or Einstein, say. Of course they were not string theorists, and neither is `normal’ a word one should use for any index related to Witten. Still, the explosion of citations is a relatively recent phenomenon, and related to the explosion of papers, so the variation of any index with time — for individuals as well as within fields — need to be taken into account.

4. Many scientists work across disciplines, many more work across subfields. It makes no sense for such people’s work to be evaluated by a single index, as the index may have different ranges in the different fields or subfields. For example, someone working mostly in mathematics and occasionally in string theory may end up with an index which is low compared to string theorists and high compared to mathematicians. How should something like that be used?

5. Indices are used for different purposes at different career stages. So it does not make sense to use the same index for people at different stages of their career.

6. There may be `social’ factors in the rise of citation count of specific papers or individuals — some are obvious and `nearly academic’ ones, like the popularity or currency of a field or a problem — the bandwagon effect. Then there is the `club’ effect — I cite you, you cite me, friends cite friends — which can work wonders in small subfields. There may also be less academic and more career-oriented reasons — it is very likely that papers cite probable referees more often, so that a paper does not come back for revisions simply because `relevant literature was not cited.’ I would not be surprised if this mechanism gets reinforced for people with many collaborators — a paper might be rejected if it did not cite the papers of the referee’s collaborators.

There are also several issues special to Indian science, which have to do with how appointments and promotions are usually effected in India. As noted by G. Desiraju in the letter to Current Science,

it was possible, in the days before we had scientometric indicators, for committees of wise men to simply declare an incompetent as an outstanding scientist.

Unfortunately, it is still possible. But that is another discussion.

Government obstructing justice

April 22, 2011 Leave a comment

The Supreme Court has come down hard on the government for not disclosing the names of Indian depositors of black money in Swiss banks, and for talking about only Hassan Ali whenever the issue of black money comes up. It has further scolded the government for not taking action about black money despite having the necessary information. Clearly, the government is trying to avoid prosecuting such people, and does not want the court to take any steps either. In other words, the government is obstructing the course of justice by not disclosing information and often by misleading the court.

Is it really sufficient for the court to rap the Solicitor General? If the decision to obstruct justice has been taken at ministerial levels, shouldn’t the court try to find a way to bring those responsible to justice? Even ministers are not above the law. Really.

Teaching without grading

April 19, 2011 Leave a comment

This should really go into a category of real life dark humour or something similar. It seems that some faculty at the Indian Statistical Institute have come up with a truly original position. They taught some courses, set the exam papers for those courses, but then refused to grade the papers, saying that they are not bound to grade papers by their service rules. Since the students were in their final semester and in danger of not getting their B.Stat degrees, a committee was constituted, which conducted and graded another exam on the same subject.

Now of course some other faculty have joined them, or at least expressed an wish to do so. The academic council, a toothless body in any case, has not come up with any idea of what to do if more faculty take this position. It is also not clear what brought this on. Someone suggested that Bhatnagar awards might have caused some kind of madness.

Thoughts on the Jan Lokpal Bill

April 16, 2011 2 comments

Everybody seems to have an opinion about the Jan Lokpal bill (draft available here) these days — after Anna Hazare threatened to starve himself to death if the government didn’t agree to put up the bill in the parliament’s next session. Thousands of people waved little flags and wore the white Gandhi caps worn by Anna Hazare and associated with followers of the Mahatma, including Nehru and almost every leader of Congress (but most probably never worn by the great soul himself). It is perhaps a bit ironic that the anti-corruption marches had so many people wearing the Gandhi caps, when those caps are associated with the picture of corrupt politicians. Shiv Visvanathan’s description of the events as a simulacrum is not without merit.

There are now several articles in the blogosphere spelling out problems with the Jan Lokpal bill, for example by Shuddhabrata Sengupta here and Mihir Sharma here. The criticisms focus on the powers of the Lokpal, which appear to be unbridled, and the mechanisms of appointing or removing the said authority. The comments on Tarunabh Khaitan’s post at Law and Other Things seem to be quite well-reasoned, although restricted to a small group of people.

I have another problem with the `movement’. The movement originates in the failure of our justice system — there would be no such movement if the courts did not allow the rich and the powerful to delay or alter the course of justice, often through official mechanisms. The police is completely controlled by the government, which translates to politicians, and the investigations against politicians are often cursory to nonexistent as a result. And this neglect of duty often extends to those in business and industry who fund politicians.

By mobilising in favour of yet another office — that of the Lokpal — the public is reaffirming its loss of faith in the judiciary, the police, the legislature, and in fact the existing laws. So why do they expect this other office to function any better? If there is no one in all these other offices who can be relied upon, or if there is so much corruption in all courts, all law-enforcement agencies and among all politicians that it smothers all efforts to bring the corrupt officials and politicians to justice, how will the office of the Lokpal remain free of it?

My feeling is that one bill or one office will not serve the purpose of preventing or removing corruption from public offices and officials. The only way that will happen is if the existing laws are enforced properly, justice is served quickly, and action is taken quickly against those who protect politicians or rich people. Some new laws may be needed as well, but those must be easily enforceable.

For example, one way of restricting the movement of cash is to prevent the withdrawal of more than Rs.25000 cash per day from any bank account. There was a proposal by the finance ministry under Chidambaram in 2005 to impose a 0.1% tax on cash withdrawals above Rs.10000. Savings bank accounts were exempted from this soon afterwards. I cannot help but think that the pressure to exempt savings accounts must have come from the beneficiaries of such withdrawals, businesses in the real estate in particular, who ask buyers to provide a significant percentage of the price in cash.

Another way of preventing `official corruption’ — i.e. the kind of corruption which grants contracts at inflated prices — is to ensure that whenever such a case comes before the court, all files concerning the government decisions should be provided to the court without delay. I find it very strange when the government asks a court for time to prepare its defense of its own decision, and even stranger that the courts allow such petitions.

Anyway, the Jan Lokpal bill may be passed, with or without modifications. Some actions will result soon after, but in the long run I think it will undermine the functioning of all other offices of law enforcement, thus creating an effect opposite to what it was supposed to.