Important update (July 6, 2008)

• Wikipedia on peer review --- an excellent article, an example of wikipedia quality, and interesting anyway!
• Science's Editor in Chief and "Friends" Discuss Peer Review and Journal Impact.

Zhigang, I was re-reading the comment from the NSF proposal to fund imechanica originally. Comments like "Your project is high risk".  Your project was "visionary" ---- then how did you do it, even without the NSF money?   If you were visionary, these NSF reviewers were blind!! It "could not work" with the little money they wanted to give you.  It did work without!  ;)

I wonder if any of the review is good at all --- even today, rereading, do we learn anything?  I don't see any good use of it.  As I read the reviews of 2 rejected papers I had (one you know very well in Int J Fatigue, and another, more recent, from JMPS - Caltech Editor), I am thinking about reviews.

Why there is not a feedback on reviews?  One writes a paper, or a proposal, spends a lot of time and effort.  Then somebody, honest or dishonest, correct or incorrect, valid or less valid, kills your project or your proposal.

We can always repost the paper or the proposal, somewhere else.  This is indeed what you did with imechanica, and indeed you are proving successfull even WITHOUT the funding you asked for!!!

This is ALSO what I plan to do with my 2 rejected papers.

But what about, like in your case, you proved the reviewers wrong?  Should there be an entity looking for Reviews of Reviewers?  Should be there a feedback on how peer review is done.  If a reviewer systematically fails to kill what should be killed, and instead kill what should not, should be NOT a reviewer in future?  Should we have a closed feedback loop?

I find reading this very interesting today.....  Should NSF give you the money you asked for now, even without resubmitting?  Should there be a "higher court" or "Supreme Court" of appeal for papers and proposals?

ORIGINAL REVIEW OF THE REJECTED NSF PROPOSAL FOR IMECHANICA: 

Very well written proposal. This project is high value, but also high risk. This is an innovative and even visionary project, but the proposal is weak on management and evaluation, and there is no minority outreach plan. This is a serious weakness. This project could have a lot of impact if successful. Could provide us with some genuinely useful lessons. There is much value in funding this proposal. They are ambitious, one reviewer felt that they have the resources to do it, another felt it was over-ambitious.