User login

Navigation

You are here

Citation Statistics Report by the IMU

Here is a report on Citation Statistics written by the  International Mathematical Union written in cooperation with the ICIAM and IMS.

Given all the discussion on impact factors and h-indicies, I thought many people may find this report interesting.

Comments

This paper pointed out many very good issues about the citation based evaluation it missed many other deficiencies.

1. If we assume all authors had ideal review, every paper is based on a few key papers and number of other papers. in current citation based system all papers get same credit. and it is not true

2. due to last deficiency if you do some thing which is a little useful for many papers is very better than doing something great for a few papers.

3. there is no limitation for number of references and many low quality papers has too much references and change our citation data base significantly.

3. citation based system have a paradoxical nature. if we call higly cited papers high quality papers and lowly cited papers low quality. must of papers are low quality and since much of references comes from these papers, quality of high quality papers is deteremines from low quality papers.

4. there is no parameter to check the final of influence of a bunch of papers on industry and society. then a network of low quality papers can cite each other.

6. reviewers of many ISI journals dont check even content of the papers good, references are next step.

7. a lot of number of references comes from introduction part and in this part many authors just like to justify their work. many of them use any paper which help them in this justification.

8. H factor is good for select nobel prize winners which has a number of hi quality papers it is not good for ordinary researchers. their citation diagram is very different in act. (i checked!). (attention even Einestein was an ordinary researcher)

9. Some books and review papers summurize papers and then others refer to books and review papers. then many of papers will lost because they summurized in books or review papers.

10. when some author or inistitution becomes famous people follow his works and it gets more citation (halo effect)

 

  

Mike Ciavarella's picture

1) Highlycited scientists are really good, in most cases.

Check the www.isihighlycitedscientists.com database.  You can argue about it, but it is sure that you will find the names of the 250 best in each of the 21 disciplines generally very good --- some are nobel prizes, political leaders and so on

2) it is sad that out of 5250 total, 4014 are from US, and just more than 1200 from the rest of the world!!!   This is a real dominance.... and has effects in many ways.  If you are not publishing from US, it is harder.

3) n.3 country comes UK with about 400, and Switzerland has an impressive 144 for its size...

4) Italy is only at 81, not very good at all.   A group of them was formed in 2003, which has a web site, www.gruppo2003.it,  and I am helping them creating a web online journal 

5) if anyone in imechanica wants to create the group of Engineers highlycited, there are 43 in Europe, and 250 in USA.   I can attach the list it is public, no secret.

6)  something interesting that has NOT been done, is to correlate this list with the more recent H-index. If we do that, it has scientific interest.  Clearly, I have noticed that in the top 81 italians, there are people with 81 or 66, in the range of Nobel prize...

7) Zhigang Suo is not yet included in any discipline.  Maybe a disvantage of his work being spread among different disciplines. I remember him getting upset when I would tell him that in engineering alone is h factor is about 18, whereas the total is maybe 43. Another effect, Zhigang, and here is no story to argue about, is that you are not yet included here. We should all help Zhigang getting "challenged" to get it there, otherwise he would feel bored having achieved so much already!

8) I start with list of ISI highlycited Engineers in Europe (43)

Åström, Karl Johan Lund Institute of Technology/Lund University Sweden

Beven, Keith John Lancaster University England Ecology/Environment Engineering

Biersack, Jochen P. Hahn-Meitner-Institut, Berlin Germany Engineering

Billings, Stephen A. University of Sheffield England Engineering

Bimberg, Dieter  Technische Universität Berlin Germany Engineering Physics

Charpak, Georges  CERN Switzerland Engineering

Chen, Sheng  University of Southampton, Highfield England Engineering

Clarke, David W. University of Oxford England Engineering

Crisfield, Michael A. Imperial College London England Engineering

Crochet, Marcel J. Universite Catholique de Louvain Belgium Engineering

Dagan, Gedeon  Tel Aviv University Israel Engineering Ecology/Environment

Dubois, Didier  Institut de Recherche en Informatique de Toulouse France Computer Science

Eckstein, Wolfgang  Max-Planck-Institut für Plasmaphysik Germany Engineering

Fleck, Norman A. University of Cambridge England Engineering

Glover, Keith  University of Cambridge England Engineering

Groeseneken, Guido  IMEC - Katholieke Universiteit Leuven Belgium Engineering

Hopfinger, Emil J. LEGI France Engineering

Hornik, Kurt  Wirtschaftsuniversität Wien Austria Engineering

Isidori, Alberto  Università degli Studî di Roma La Sapienza Italy Engineering

Jones, Kevin C. Lancaster University England Ecology/Environment Engineering

Kemmer, Josef  KETEK GmbH Germany Engineering

Lutz, Gerhard  Max-Planck-Institut fnr Physik Germany Engineering

Mallat, Stéphane G. École Polytechnique France Engineering Computer Science

Morari, Manfred  ETH Zürich Switzerland Engineering

Payne, David N. University of Southampton England Engineering

Philipps, V.  EURATOM Association Germany Engineering

Prade, Henri  CNRS France Computer Science Engineering

Rodi, Wolfgang Alfons Universität Karlsruhe Germany Engineering

Roth, Joachim  Max-Planck-Institut für Plasmaphysik Germany Engineering

Sauli, Fabio  CERN Switzerland Engineering

Schwarzenbach, René P. Vorsteher Institut für Gewässerschutz undWassertechnologie Switzerland Engineering Ecology/Environment

Sharir, Micha  Tel Aviv University Israel Engineering Computer Science

Sigmund, Peter  University of Southern Denmark Denmark Engineering

Stoica, Peter  Uppsala University, Department of Information Technology Sweden Engineering

Tvergaard, Viggo  Technical University of Denmark Denmark Engineering

Unser, Michael  École Polytechnique Fédérale de Lausanne (EPFL) Switzerland Engineering

Vetterli, Martin  École Polytechnique Fédérale de Lausanne (EPFL) Switzerland Engineering

Weilhammer, Peter  CERN Switzerland Engineering

Winter, Jörg  Ruhr-Universität Bochum Germany Engineering

Wold, Svante  Umeå University Sweden Engineering

Wright, Richard F. Norwegian Institute for Water Research Norway Ecology/Environment Engineering

Wyatt, Richard    England Engineering

Zienkiewicz, Olgierd Cecil University of Wales Swansea Wales, UK Engineering
 

Regards,

michele ciavarella
www.micheleciavarella.it

I have one discussion with one of these highly cited researchers. and i asked him about his highly cited paper and he answered me very trustly. i just used a method developed by others to supply needs of others who did not know that. it is just like a movie, when we see a movie we just see the stars but we dont see other people who supply this stars. must of research achivements are evolutionary but when it change the PHASE we acknowledge people we see in the phase change. not the people who had role in that evolution. It is even true about nobel prize winners.

I hope mathematicians find a way to model scientific society with a more exact network model and will prove deficiencies of current system more exactly.  but in very simple mathematics just assume the main results is first funtion and is multiplied by a random function caused by these errors. final functions is what we see know as citation numbers. if peaks be strong enough they remain peaks at final functions but if many of peaks coincide we can not be sure that two function exactly corrolate! in other words i know the most citeds are good are they best?!. are'nt others better than them? this is the question. 

Mike Ciavarella's picture

Roozbeh

to become a start in science fast, the traditional way is that you are a student of a start scientist in science.  That speeds up publication.  Imagine you co-author with a Nobel prize winner.  Who is going to say your paper is bad?

There are similarities with the movie Star industry as well.  With ups and downs. And scandal.  You remember the Korean guy who claimed he cloned first a sheep?

See for example this interesting story:

http://www.wired.com/science/discoveries/news/2002/09/55410

The Undoing of a Star Scientist


Wired News Report


Email

09.26.02


(Editor's note: This story originally was credited as being written by
Michelle Delio. In fact, the Associated Press contributed a significant
portion of the reporting. Ms. Delio added her own, independently
researched material and composed the following report, which now does
not carry her byline).

Jan Hendrik Schön seemed to be a modern day alchemist.

He appeared to be able to manipulate atoms with amazing grace. He
seemed to be able to force electricity to do seemingly impossible
tricks, and claimed he'd found ways to make working molecule-sized
electronic components.

 

The
results of many of his experiments contradicted known laws of physics.
Other scientists were unable to duplicate Schön's work. Some said the
young man must have magic hands.

But not all.

In May,
Lucent Technologies, which runs Bell Labs where Schön worked, responded
to growing queries about Schön by appointing an independent committee
to investigate. The results were released Wednesday, validating what
many physicists already theorized months ago.

The committee believes Schön made up or altered experimental data at least 16 times between 1998 and 2001.

Lucent announced the firing of Schön on Wednesday. But some questions still remain.

Schön, 32, published 80 reports in prestigious journals over the last
two years, more than most scientists produce in a lifetime. Some
scientists think the established scientific peer-review process failed
by allowing some of those papers to be published.

But many others believe that Schön's story proves the merits of standard peer-review procedures.

"Yes, misconduct occurred," said Malcolm Beasley, an applied physics
professor who chaired the committee that investigated Schön. "But the
normal processes of science ferreted it out. The system worked."

But the investigating committee did find that there is a need for
"clear, widely accepted standards" of professional responsibility to
ensure the veracity of published scientific results.

Bell
Labs isn't the only scientific institution to be smacked with charges
of fraud recently. The Lawrence Berkeley National Laboratory announced in June that it had disciplined a researcher for fabricating the results of a published experiment.

"It's a very good thing for people and organizations in science to look
at this," Beasley said. "It's not something that the community has
thought through."

Bell Labs spokesman Saswato Das said
laboratory officials will be taking steps to prevent misconduct,
including encouraging extended review of new research before it is
published and clarifying supervisors' responsibilities for reviewing
all research papers.

The signs of Schön's fraud were writ large, but perhaps are only readable in hindsight.

Schön substituted data in his published papers, supplying fake graphs
that he told investigators "looked better" than the real graphs.

He also used the same graph in a dozen papers on different experiments.
And his data were often far too precise, far beyond reasonable
statistical probability.

Schön acknowledged that the data are
incorrect in many of his papers but said that some problems occurred
due to "honest mistakes."

But investigators noted that "the
recurrent nature of such mistakes suggests a deeper problem. At a
minimum, Schön showed reckless disregard for the sanctity of data in
the value system of science."

Some who worked with Schön at
Bell Labs said he is a quiet, almost shy man -- "totally unpretentious,
totally unlike the egotist you'd imagine would be capable of such
acts," said one research assistant, who spoke on condition of
anonymity.

"People think that he must have cracked under the
pressure ... so much was expected of him," the assistant said. "It's so
sad. All his work is now suspect."

"It's two years that have
to be erased," Princeton University physicist Lydia Sohn said. "A lot
of us had hoped that this was new stuff and that it was going to be
really cool for the future of electronics."

Sohn was one of the scientists who alerted Bell Labs to problems with Schön's work.

Schön did not respond to telephone messages left at his New Jersey home Wednesday night.

In a statement responding to the committee's findings, Schön said, "I
tried to communicate the science that described the experimental
findings and that I was convinced of. Although I have made mistakes, I
never wanted to mislead anybody or to misuse anybody's trust.... I
truly believe that the reported effects are real, exciting and
reproducible."

The Associated Press contributed to this report.

 

 

michele ciavarella
www.micheleciavarella.it

Mike you write, "Imagine you co-author with a Nobel prize winner.  Who is going to say your paper is bad?"

I would say lots of people. 

While it certainly can help to have famous co-authors, i can say from experience on grant review panels and in the paper reviewing process that even Nobels (or other top people) have their work brutally critisized and even rejected.  Sometimes being famous seems to hurt, as people often have a bit of Schadenfreude in taking down famous people in an anoymous manner.  At best, I think having big names in an author list gives you a bit of deference, in that if the reviewer does not understand something in the paper/proposal they will spend extra time trying to understand it before writing a negative comment (that may reveal themselves as less bright than the authors).

 

As an anecdote, I will add the story of Hans Einstein (Albert's son).  Hans was a Civil Engineering professor at Berkeley and at one time wrote a paper dealing with the Navier-Stokes equations.  In the paper, he apparently made a fundamental error.  This paper then became one of the most highly cited papers in the field because many people took joy in being able to point out that an Einstein made an error.  This story is also a bit of caution in assuming that all highly cited papers are good.  (I have never verfied the accuracy of this story but I do trust the person I heard it from.)

Prof. Dr. Sanjay Govindjee
University of California, Berkeley

7. most of references comes from introduction part and in this part many authors just like to justify their work. tell us an story, many of them use any paper which help them in this justification. i think it is better to just calculate those which USED in the paper in citation calculation. 

Subscribe to Comments for "Citation Statistics Report by the IMU"

Recent comments

More comments

Syndicate

Subscribe to Syndicate