Refereeing is a professional service which one is required to do as part of being in the academic community. It is also a reciprocal act considering that when we submit a paper for a conference we expect careful and considerate feedback. For a young researcher, refereeing can be an excellent way to keep up to date with the latest research. I found a couple of excellent articles on guidelines for reviewing a paper.
The first article is from a 1989 edition of ACM SIGACT News. The writer, Ian Parberry presents a hierarchy of quality of papers which I found interesting:
- Breakthrough (solves longstanding open problem)
- Groundbreaking (Opens up the field)
- Reprise (superior proof of an old result)
- Tinkering (more careful analysis)
- Debugging (Corrects some undiscovered mistake)
Parberry also has a separate section on ethics of refereeing which is worth reading.
The second article by Alan Smith was published in IEEE Computer. Smith also has valuable advice on refereeing. There is a separate section on refereeing research or funding proposals. Smith says that “a major difference between research proposals and papers is that a proposal is speculative, and the reviewer has to evaluate what is likely to result.” Smith is also of the view that for a paper review, the reputation of the author should not matter except in matters such as certain ambiguities in the paper. This makes sense.
Both articles are useful to get an overview on refereeing papers. They may also be useful to evaluate one’s own work objectively.