Plagiarism Scandal
Today’s Nature has an article by Geoff Brumfiel with more details on the plagiarism scandal described here. At last count it involves 15 authors, 67 papers on the arXiv, of which about 35 were refereed and published, in 18 different journals. The arXiv has set up a special page with information about this. As far as I can tell from checking a few examples, most of the published papers are still available online at the journals, with no indication of their plagiarized nature. One exception is the plagiarized paper at JHEP, which has now been removed, with the notation
This paper has been removed because of plagiarism. We regret that the paper was published.
As far as I know, neither JHEP nor any of the journals has given any indication of an intent to change their refereeing procedures because of this scandal.
Journal of K-theory
The editors of the new Journal of K-theory have issued a public statement, explaining in detail their plans for how to handle papers submitted to the older journal, K-theory, where they had resigned as editors.
It does make me wonder about the quality of the refereeing.
The way the review process is applied in physics is a scandal.
Jon
Then do you have any suggestions on how to improve it? remember: referees are not payed, they take time off from pursuing their own work and, this is the most important point i never see properly expressed, they are there to check the scientific soundness and impact of the work. they are not there to check spelling, grammar, fraud or plagiarism. when i get a paper to referee, i assume that the authors were honest. if i don’t, i wouldn’t even know where to start. should i redo the whole work or what to see if it is reproducible? sure, the key argumentation and drawing of conclusions are tractable, but if someone says, this and that is the outcome of a particular experimental setup or a one year calculation on a supercomputer reveals this number as a result – how can you as a referee challenge that? you have to rely on the honesty the authors.
it is similar with respect to plagiarism. ideally, of course, you should know all the literature in the field of the paper that you are refereeing. but honestly, there are certain topics that are pursued by only one group of people worldwide and unless you want them to self-referee, someone a bit outside has to be chosen. you can’t seriously expect them to dig thru all the literature in search of plagiarism. this is not what referees are supposed to do!
and on a final note: why the revelation of plagiarism such a scandal for the refereeing system? it just shows, that basically it works. the culprits have been identified and you can be quite sure that their career is over. and given the fact that in order to be of any relevance, a paper has first to be noticed, it just absolutely floors me how these people think they could ever get away with it and have some advantage from plagiarizing. as soon as the work really becomes known, the plagiarization is soon uncovered.
the same by the way is true for fraud. remember the guy claiming to have produced element 117 (or was it 118?)? he stated an experimentally falsifiable wrong statement. just by common sense there are only 2 future prospects for such a statement: either nobody cares in which case you have to ask yourself what the motivation was for cooking up the fraud in the first place. or people are interested and will inevitably uncover the fraud.
so i would say that what we witness here is good proof that science is healthy and has self-correcting mechanisms that work.
Look at the webpage of ALİ HAVARE,
http://www.mersin.edu.tr/apbs.php?id=335
one of the persons involved. Most of his (former) students, such as Yetkin, Aydogdu, Salti and Korunur were involved as well. Wonder who was the head of the gang.
The K-Theory journal situation still makes my head hurt, no matter how much I read up on it.
Elisha, can you check the reference number 25, page 343 in K-theory Volume 36 (2005) in the printed version (not online version)
to get an example of the `seriousness’ with which Springer has been treating the manuscripts of K-theory!!
chris,
I don’t see how this is evidence of a healthy system at work. If the authors had bothered to change the wording and notation when they plaigiarized, no one would be the wiser.
One of the main roles of the referee is to determine whether the result claimed in a paper is not just correct, but original. If a referee knows nothing about whether the main result claimed in a paper is something that has been done before, and they don’t want to take the time to look into this, they shouldn’t be refereeing the paper. What this scandal shows is that getting unoriginal (and uninteresting) work published in most theoretical physics journals is almost trivially easy. One reason for this is that the results are of such little interest that no one other than the referee is likely to actually read the paper and notice that it isn’t original.
The journals should either find a way to do the kind of peer review that they are claiming to do (and charging lots of money for), or admit that it just can’t be done any more and give up. In the meantime, it might be a good idea to deal with the current situation by putting warning labels on these journals saying something like “the editors haven’t been able to determine if these papers contain original research or not”.
hi peter,
actually i agree with you. and for all practical purposes, these warning labels are already there i would say. because from personal experience i conclude that what counts (in a positive sense) is the recognition of a paper and not journal reference (on the negative, having a preprint without journal reference is a very big warning sign). i think there is consensus that a lot of published work is not worth the paper it was printed on. but i think there is equal consensus, that it is more worthwhile to push ones own research than debuking others unless they make very strong claims or do something that affects you personally.
i am not saying that this state of affairs is ideal. but i see it as the prize to pay for the ease of information exchange. what it ultimately boils down to is that in order to judge a papers merits you actually have to read and understand it yourself.
so in all honesty, classical journals in my opinion are outdated already. they do have some merit, but the speed of todays research just makes a close to 100% identification rate of ‘good’ papers illusionary. the problem in my opinion comes in when what they do publish is taken as gospel. when selection committees count the number of papers rather than reading the 3 topcited ones. but i see this as a problem on the recipient side. a judgment based on metadata of a persons publication record is just not foolproof. today probably less than decades ago.
for plagiarism specifically i can only repeat my claim, that the only way it can stay undiscovered is when nobody cares about the research and nobody has suffered negative consequences. it is kind of sad that people exist who wish to populate this corner. but i doubt that much more resources of serious scientists should be spent to explicitly search for them in cases where their plagiarism has next to no effect.
but to be constructive, let me ask what you think journals or referees should do?
since refereeing can only be sensibly done by active researchers, this implies that more thorough refereeing will leave less time for research.
Peter,
What makes me feel sad about the peer review system in physics is that good ideas may fail to go through while rubbish, just because is well recognized fashion, can easily get published on the most important journals. What is worst are the reasons for rejecting a paper that are generally unsound, but now, besides the reviewers, publishers tend to hide also the editors. No person takes on responsibility for the large number of errors the system is badly doing in this historical period.
A paper of mine was rejected by a DAE of PRL because “His work has had no response by the community. I do not understand what he is doing”. Other “reasons” like these can be found as I have a large file of published papers and a lot of unpublished ones and it would be really fun to get the reports of such reviewers known, after so long time, to see how badly they turn out to be wrong. I think there is a lot of people out there in similar situations but we are all silent because we fear to have our other papers no more published.
I worked in almost all field of physics but the absolutely worst situation is in particle physics. A referee claimed that “QCD lattice computations do not reflect reality” to reject a paper of mine. So, there is a lot of people out there wasting time, money and resources!
There is again a lot to say. For the moment I stop here. But I would like to discuss what could make a paper publishable and what should mean “important”, a criteria largely questionable and deemed to the taste of the single person. In this way is truly easy to suggest rejection on questionable personal feelings.
Jon
hi jon,
did you try to submit said paper to another journal than prl and did it make it?
honestly, i think that every paper which is not totally meritless will certainly find a journal these days.
and even if that should happen, if the paper reaches 50 or 100 citations it doesn’t matter that much anymore. people will get curious as of why it didn’t make it. and if it doesn’t recach that, well, chances are nobody would have cared anyways.
Then you shouldn’t referee. If you don’t know the field well enough to know whether a result is original or not, accepting to referee is dishonest on your part.
Hi chris,
Yes, I did it and I get it accepted in a few days by the editor. I think (my personal judgement) that the paper does worth that.
Jon
hard gluon said:
“Then you shouldn’t referee. If you don’t know the field well enough to know whether a result is original or not, accepting to referee is dishonest on your part.”
I think your judgment is too harsh. In our age of narrow specialization, there could be only 2 or 3 people in the world (including the author) who know exactly the background of each particular manuscript. Often these people are either collaborators or rivals, in which cases it doesn’t make sense to ask them to review the paper.
For the rest of us it could be almost impossible to read all the references in the reviewed work, to repeat all calculations, or redo the experiment. So, it is impractical to ask the referee to give a 100% “seal of approval”. If that would be possible and only “correct” papers appeared in print, then scientific journals would be 100 times thinner than they are now.
This is not only unrealistic, but also dangerous, because increasing the barrier for publication may prevent novel non-mainstream ideas from being published. My attitude is that “s**t happens”, and that one or two plagiarized papers will not have any effect, except for the embarrassement to their authors. There are things much more dangerous for the health of science. String theory and anthropic groupthinks are high in this list. We should be thankful to Peter for keeping focus on these areas.
Eugene.
Maybe it is, hence my nickname :). Besides, nobody is infallible and plagiarism may be hard to detect. Yet, take for example the paper “Brane-world black holes and energy-momentum vector”. I’m sure there are many people out there who are involved sufficiently in the field of black holes in brane worlds to realize whether that paper is lifting its results from some other recent papers. After all, twenty years back nobody was studying brane worlds.
That is obviously something a referee should never do. If there are obvious (to an expert) mistakes, those should be pointed out. But if the results reported in a paper look consistent and plausible enough to an expert eye, no further checking should be necessary. It’s the author’s business, I think, to take care that their results are correct. It is their professional reputations that is at stake.
Having said this, I must say that once a referee found a factor 1/2 wrong in a complicated equation for a cross section in a manuscript I submitted. He/she thought it was a typo. Well, it wasn’t, it was a calculational mistake, and to this very date I have no idea how that referee caught it… This happened some time ago, I’m not sure that kind of quality refereeing happens very often these days.
Hi Peter,
I happen to be from turkey originally so I know the environment there quite well. These folks in question hardly know any english at all. If you asked them to give a 10 minute presentation in english about anything, they wouldn’t be able to. The system gives promotion based on bean counting in turkey these days so the strategy is “publish as many as you can” without any regard to quality. That is why most of the papers in question are published in obscure journals. Nobody cares where you publish. Turkey as a country is yet to publish one single paper in science or nature.
when i get a paper to referee, i assume that the authors were honest. if i don’t, i wouldn’t even know where to start. should i redo the whole work or what to see if it is reproducible?
One thing I’ve started doing with every paper I review is pick a handful of phrases, from the intro, the discussion, the results, and just run them through Google. It takes less than five minutes but has at least a chance of picking up plagiarism. (So far, thank God, I haven’t found any, and I hate to think of what I’d have to do if I did.)
You’re right, you can’t repeat experiments. If you’re lucky, you have experience with a close-enough system to notice if something is plausible or not. You can also take a few minutes to mentally work through a protocol; there’s a well-known instance where a reviewer caught a case of fraud because he realized the experiment as described would have taken thousands of tissue-culture flasks.
Editors are supposedly looking at figures more closely nowadays, including using some automated techniques that will pick up simple fakes. (I’m in biology, and I don’t know if other fields have similar approaches.) I do look with a skeptical eye at figures, but I’m not sure that I’d pick up any but the crudest forgeries.
Ali,
It’s the same all over the Mediterranean (Spain, Italy, Greece, etc.). Promotion depends on the count of papers weighted by impact factor. The more self-citations the better. Governments and universities looking for `objective’ criteria for evaluation impose such requirements. The commercial journals depend on this to survive; the journal is outrageously expensive, but can count on a large number of mediocre scientists looking for `impact’ to submit their articles. That the publisher produces three journals in the same area is no surprise either; the author and his friends rotate journals, citing each other into the university administration. Once there they see to it that more `objective’ criteria for promotion are imposed, favoring ambitious mediocrities like themselves, because such people are incapable of challenging them scientifically, and are easily made dependent.
hi ian,
“One thing I’ve started doing with every paper I review is pick a handful of phrases, from the intro, the discussion, the results, and just run them through Google.”
that actually is a really good suggestion!
Highly Cited Researcher, we have similar system here in eastern Europe. My opinion is that if bureaucrats wants numbers, then total number of citations and determination in which decile the author is in terms of citations in his subfield would be much much better than what we have, though still far from ideal.
Just like that googling idea, isn’t it natural to suggest a similar service from the arxiv? A referee would get a candidate-to-be-paper, run a quick search there (arxiv), if too many similarities come up, well… then she/he’s done already!
but, wait… there’s really nothing similar already?
because, if there isn’t, well… I guess plagiarism is just being asked for…right?
Highly cited researcher,
I am aware of it. It would be more appropriate if people paid attention to the journal the article was published as well. For instance, in china, government pays 1000 dollars per impact factor of the journal the article was published. If you publish one article in science, you receive 30K from the government so there is an incentive to publish in high reputation journals. In countries like greece and turkey, there is no such thing. People can barely speak english, let alone write articles about black holes.
Coincidentally, I was just asked to referee a paper (not found on the arXiv) by different Turkish authors (although from one of the institutions involved), which was also plagurized. However, in this case, the stealing was obvious merely from following up on the paper’s references. It looked like the authors had no idea they were doing anything wrong! I’m not sure what to make of this, except that all this plagurizing must be emblematic of a serious misapprehension of scientific ethics in some corners of the Turkish physics community.
Ali i take it that you have some idea about the status of the status of the physics community in turkey but please don’t extrapolate it to greece. There are many good greek researchers and people actually do speek english. On the other hand it’s true that the greek ( i guess the turkish too) goverment prefers to buy weapons (20000 million € in the next 10 years) than to seriously invest in research. That’s sad but it will not change soon.
“For instance, in china, government pays 1000 dollars per impact factor of the journal the article was published. ”
Is this true?! Does anybody know?
Pingback: Plaigarism Scandal by Turkish Physicists « Theoretically Speaking!
I agree totally with you about Berlinski. His ‘tour of the calculus’ might be the worst book I’ve ever read. One more reason for me to buy yours…
Peter Woit Says “What this scandal shows is that getting unoriginal (and uninteresting) work published in most theoretical physics journals is almost trivially easy. One reason for this is that the results are of such little interest that no one other than the referee is likely to actually read the paper and notice that it isn’t original.”
What do you expect in a world of “publish or perish”? If people are forced to publish, they will publish. Even if they don’t have anything interesting to say.
Trust me, it’s not just physics that publishes rubbish. It’s a huge problem that needs to be solved. As Ilja implied, publish or perish needs to be changed and it needs to be changed by us. If you don’t have anything worthwhile to say, don’t say it. If we all exercise restraint and only publish what is really worth something (and we all know when that happens, and it doesn’t happen that often) we can start to change things.
Yes, it may mean not landing the coveted “position”, and it may mean teaching in community college or non-tenure positions. But isn’t it worth the price to be able to say ” I may have only published X number of papers, but they really meant something”. I can’t help but feel this is a personal responsibility issue (as most are).