Refereeing

Introduction

These are my own notes from the first time I was asked to review a paper. Much of the advice is taken from Marc P. Bellemare’s book “Doing Economics”.

Short guide to reviewing papers

General notes:

• When you get a request to review a paper, give your answer to the editor fast, as the peer review process is slow as is.
• As a referee, you do not reject or accept a paper, you merely give your advice to the editor.
• Generally, you write a report that is shared with the authors and a cover letter to the editor with your advice.

How to decide on accepting:

• A good rule is to never turn down a referee request unless you really are unable to review it (having a child, conflict of interest, etc). It is a good learning experience.
• Should you review papers that are not in your research field? The simple answer is yes, some editors want to have an opinion from someone not in the field. This will signal if the topic is interesting to people outside the area.

How to write a good review:

• Write constructive reviews that are at the right level for the journal.
• If you recommend reject it is fine to write a half to one page report, but if you recommend revise and resubmit make sure you provide everything you expect the authors to revise before the next submission.
• When you point out a problem, tell the authors how they can address it.
• The quality of the review is only weakly correlated with its length.

Steps of reviewing:

• Read the title and abstract and decide if you can review the paper.
• Once you have accepted: read the title, abstract, introduction, tables (if there are any) and conclusion. If you do not have a good idea of what the authors are doing (e.g., if it is an empirical paper can you understand the tables) or the topic is not interesting: recommend rejection. Authors should work hard to “sell” their paper. Write a report (less than 2 pages) explaining how the authors can improve their paper.
• If you think the paper got a shot, read the whole paper and write a more extensive report (also including typos, missing references, etc.).

Blueprint for report:

• Start by summarizing the paper (not copying the abstract), give your view of what the paper does.
• List major comments (deal breakers) and how the authors can address them. If you do not think the authors can address the comments you should recommend rejection.
• List minor comments of what you think the authors could improve upon (optional).

Blueprint for cover letter:

• Write your recommendation: “reject”, “weak revise and resubmit”/”major revision”, “strong revise and resubmit”/”minor revision”, or “accept”. Motivate your recommendation to the editor.
• In the cover letter you have the potential to write any ethical concerns.