Guidelines for anonymous review of grant applications

Our Office is leading a national trial to study the effects of anonymising grant applications for in-demand scientific equipment, like telescopes, synchrotrons, and supercomputers.  In this trial, grant applicants are asked to remove any identifying information from their applications. Reviewers assess the applications without knowing who the applicants are. This process is called ‘anonymous peer-review.’ The goal is to focus on the science, not the science teams.

The study will assess whether the anonymised review of applications reduces bias — not just for women, but other groups too.

Guidelines for applicants

The anonymous review process requires some changes in the way applicants write their proposals.

Below are the guidelines applicants must use in preparing their grant applications in Anonymous Third Person (A3P). The anonymous review does not mean applications will be accepted from anonymous sources. Applicants will still be required to provide their names, affiliations and information on ‘team expertise and background’ (track-record) in their application. However, this specific identifying information will not be available to the reviewers and review committee during the assessment process.

Applicants should:

  • Not use names or affiliations in text, page headers, footers, diagrams or figures.
  • Not use gender pronouns in the text.
  • Use third-person neutral wording when self-referencing. Ex: replace “as we have shown in our previous work (Doe et al. 2010)” with “as Doe et al. (2010) showed…”
  • Use references to published work, including work citable by a DOI, without including information that that may reveal the identity of the applicant(s). If you cite exclusive access datasets or non-public software that may reveal or strongly imply the investigators on the proposal, use language like “obtained in private communication” or “from private consultation.”
  • Refer to previous grant programs in a non-identifying manner. Ex: replace “we observed this cluster under grant program #XXXXX,” with “grant program #XXXXX observed this cluster in the past…”
  • Not include acknowledgements or the source of any other grant funding.

 Find example A3P text of anonymised applications. 

 

Team expertise and background

Team expertise and background is provided as supplementary information and should not be made available in the experimental section of the submission.

See this sample document for an example of team expertise and background supplementary information.

Guidelines for reviewers

The anonymous review process requires some changes in the way reviewers evaluate the proposals.  

The reviewers:

  • Will review and score the applications in A3P format — without applicant identifying information.
  • Are asked to resist the urge to spend time guessing who the applicants are.
  • Are encouraged to focus on the scientific merit of the application; the primary objective is to select the best science, not the best science teams.  

During the review committee meeting:

  • A ‘leveller’ is used to make sure it stays focused on the science and not the science teams. The leveller is an external and impartial individual who monitors the committee review meeting. If necessary, the leveller may interject and refocus the discussion.
  • An unblinded post-review round is available as a fail-safe. This optional step involves revealing the identities of the applicants (only after the review) to cross-check the veracity of the team. The committee may disqualify an application based on a significant lack of expertise to carry out the proposed research project.

Compliance

Anonymisation of the applications is mandatory for compliance, and applicants must follow the applicant guidelines.  

If an application is found to quite obviously and blatantly disregard the anonymising guidelines, the application will be deemed non-compliant and may be withdrawn from further consideration. Some of the organisations pre-screen applications. Applicants of non-compliant applications are given the opportunity to make appropriate A3P changes to their application and re-submit before the reviewer’s assessment begins.

Note. The applicant, reviewer and compliance guidelines are closely based on the guidelines employed by NASA for the STScI Hubble Space Telescope (HST).

Stay informed

We are continually updating our progress on our social channels and website. For direct updates on how this project is progressing, please enter your email address here.