Report: FRO seminar on peer review

Research and Innovation Development Office

Category: Research success

13 June 2017

We held the latest in our popular Research Funding Observatory seminar series on 7 and 8 June, run in a different way from our usual lecture-and-group-work format, with a panel question and answer style.

We wanted to encourage our less-experienced researchers with their applications, and to have more confidence when preparing a bid for funding.

The sessions were broken up into three main sections.

  • Reviewers: Who are the people that review funding bids?
  • Process: What hoops does a bid have to jump through to get funded?
  • Appeal: How do you make a bid appeal to the reviewers? What do they like or dislike?

Lots of questions were asked of the panel, and we've summarised the conversations below. Our panels included reviewers for AHRC, ESRC, MRC, Leverhulme Trust and the European Commission, among others. We had representatives from FHSCE, FMS and ALSS, as well as a guest from University of Cambridge, so our range of expertise was tremendous. Hopefully the notes we've captured will inspire you to submit your own funding bids with confidence in the peer review process.

Reviewers

The panel discussed the types of people who end up reviewing bids. We heard that panels often used to be made up of senior academics only, with their inevitable gender and age biases. Nowadays panel diversity has increased rapidly, right down to (relatively) early career researchers. The panel also discussed the importance of putting your name forward to be a reviewer as early as possible. If you're a reviewer you meet more people, you gain reputation with funders, and you learn about aspects of your field you hadn't yet encountered. RIDO really encourage you to sign up to be a reviewer - get in touch if you want to know how!

We also talked about how the type of reviewer will vary scheme-to-scheme. In, for example, European fellowships, an application will be reviewed by at least two expert reviewers in the field, plus at least one other. This gives a wide range of review, so a really important point is to write very clearly for a mixed audience. Don't assume very specialist knowledge, but do assume that the reviews will be experts in their own area. Some panels contain industrial experts who have a customer-focus, in which case you really need your project to make a difference to their industry.

Process

Are reviewers typically able to see the comments and reviews from other reviewers? Well, probably not. If you're a panel member you'll see all the reviews and scores provided. However most reviewers work in isolation. There can be very large discrepancies between reviewers' scores, and we rely on the funders' strategy for dealing with this phenomenon.

Panel members and reviewers can often disagree and sometimes clash. This is the best system we've got an unfortunately can be down to subjective decision making. A strong panel chair has to steer the discussion to a decision and persuade the panel to agree with a final score or rank order. Luckily there are always policies in place to deal with conflicts of interest, so ulterior motives to fund one grant above another are reduced where possible.

Some funders give applicants the chance to nominate (or exclude) specific reviewers, but caution was advised in using this function. Sometimes the people you know best can give more harsh reviews, and sometimes your nominee (especially if they're at the top of their field) will ask a post doc or PhD student to perform the review, with varying results. However, you can be sure that if a real expert reads your application they will provide useful feedback to help you improve your bid next time.

Appeal

So what are the immediate 'red flags' to a reviewer? What puts them off funding you by the end of page one? The panels gave a few answers to this question.

  • Not following the rules (eg page limits, font size, etc). Things have to conform and be legible to be considered. If not, they'll be excluded without review.
  • No proof-reading. Although allowances can and should be made for researchers from non-English-speaking backgrounds, lack of attention to detail immediately makes the reviewer question the researcher's rigour.
  • Personal formatting choices. As with any subjective review (think of job applications, etc), different readers have strong preferences. If you use devices such as underlining to highlight key text, you risk patronising a reviewer who can work out the key sections for themselves.
  • Not summarising the project effectively on page one. In your initial abstract or summary or first page, you should be able to summarise the research question, the approach and hypothesis, the solution and the impact of your proposed project. If you can explain your work succinctly on page one, you stand a great chance of persuading people to read on.

The panel was asked if an early career researcher (ECR) needs to have lots of more experienced colleagues on their bids, maybe even relegating the ECR to Co-I role instead of PI. The discussion said that while sometimes this can be a good idea, as long as you satisfy the rules of a funding scheme you can apply as a PI as soon as you feel ready. You could aim for fellowships and ECR-specific funds to ensure you're targeting the right calls, but ask RIDO for more information on this! Building a good project advisory board into your bid is a great way to increase a reviewer's confidence in a less experienced PI.

We talked about whether your situation (as an ARU researcher) may bias a reviewer against your project. While the panel acknowledge that sometimes the playing field is not level for all applicants, the discussion was quite positive. Yes you might have to work a little harder, but concentrate on convincing the reviewer that you and your team are the right people to tackle your project, and that your environment is good enough to make it a success. Incorporating partners and collaborators can help make this case for you. Bring the right people into your team and show that you can complete an interdisciplinary project. It can also help to target different (less snobby!) funders.

Final thoughts

We wrapped up the discussion by making sure to note that while impact sections are vital to a successful bid, you must take care to keep your aims feasible but impressive. Impact must involve different target audiences, so think about ways to reach different people.

Sometimes you have chance to respond to your reviews. Make the most of this opportunity to highlight important parts of your bid and see it as a positive stage in the process. Clarify the areas that need clarification, add detail if required, and mostly make your key arguments again.




We hope this helps you with your next bids. The best piece of advice any of the panel members had to learn about peer review is to become a reviewer, so talk to RIDO to make this happen!

Thanks again to all our panel members for making the session so successful, and we hope to hear from you again soon.

Don't forget to sign up to other RFO sessions here.

Disclaimer

The views expressed here are those of the individual and do not necessarily represent the views of Anglia Ruskin University. If you've got any concerns please contact us.