Anonymised Review Study
Background
Anonymising applications for access to scientific equipment has been shown to reduce existing bias1,2 for women, early-career researchers3 and other marginalised groups1—specifically, in the astronomy and planetary science sector.
We led a trial across Australia to study the effects of anonymising applications for the use of specialised scientific equipment. The twofold purpose of the study was to observe the degree and existence of disparities, and to evaluate how anonymising applications would affect application scores, success rates and resource allocation based on the gender and career seniority of the lead researcher.
The trial involved four cross-disciplinary research entities that manage access to national scientific facilities:
- Anglo-Australian Telescope (AAT)
- Australian Centre for Neutron Scattering (ACNS)
- Australia Telescope National Facility (ATNF)
- National Computational Merit Allocation Scheme (NCMAS)
The entities implemented anonymisation differently. Our analysis examined three grant application outcomes—application scores, success rates, and allocated resources—before anonymisation and the impacts of anonymisation on them according to the gender of the lead investigator, also accounting for career seniority where those data were available.
Key findings
The introduction of anonymisation led to improved success rates for early-career researchers, while generally maintaining the pre-existing gender equity landscape before anonymisation.
- We found that the introduction of anonymisation boosted the success rates for applications led by early-career researchers at ACNS, irrespective of the applicant’s gender.
- In gender-related outcomes, there was a noteworthy absence of gender differences in application scores, success rates and allocated resources before anonymisation at all four entities.
- The introduction of anonymisation generally maintained the existing gender equity landscape, with one organisation (AAL) experiencing improved success rates for women-led applications.
Improved Early-Career Researcher Outcomes
Anonymisation statistically significantly boosted the success rates for applications led by early-career researchers at ACNS, irrespective of the applicant’s gender.
Conclusions
Anonymising applications for scientific equipment opens doors for early-career researchers, enhancing their chances of success. Since no prior gender gap existed, anonymisation would not be expected to impact gendered outcomes. Our results confirm this.
The implications extend beyond application outcomes, which represent only one piece of the puzzle that contributes to inequity in STEM research4. By enhancing success rates for early career researchers, anonymisation may create a positive ripple effect in the career pipeline, diversifying the research pool, and supporting the broader issue—retaining and advancing researchers facing barriers in STEM research.
The research
Our team
- Dr Isabelle Kingsley, Senior Research Associate at the Office of the Women in STEM Ambassador UNSW Sydney
- Nicholas Ho, Research Assistant at the Office of the Women in STEM Ambassador UNSW Sydney
- Amanda B. Chan, Research Assistant at the Office of the Women in STEM Ambassador UNSW Sydney
- Professor Lisa Harvey-Smith, Australian Government’s Women in STEM Ambassador, Professor of Practice, UNSW Sydney
- Associate Professor Lisa A Williams, School of Psychology, UNSW Sydney
References
- Strolger, L. & Natarajan, P. Doling out Hubble time with dual-anonymous evaluation. Physics Today https://physicstoday.scitation.org/do/10.1063/PT.6.3.20190301a/full (2019) doi:10.1063/pt.6.3.20190301a.
- Johnson, S. K. & Kirk, J. F. Dual-anonymization yields promising results for reducing gender bias: A naturalistic field experiment of applications for hubble space telescope time. Publications of the Astronomical Society of the Pacific 132, (2020).
- Carpenter, J. M. et al. Update on the Systematics in the ALMA Proposal Review Process after Cycle 8. Publications of the Astronomical Society of the Pacific 134, (2022).
- Kingsley, I., Slavich, E., Harvey-Smith, L., Johnston, E. L. & Williams, L. A. Gender differences in Australian research grant awards, applications, amounts, and workforce participation. OSF Preprints https://osf.io/cpvqk/ (2023) doi:10.31219/OSF.IO/CPVQK.
Learn more about our research
We lead two research projects to examine gender equity in grant programs in the Australian research sector.
What works to improve equity in STEM
We have developed tools to guide evaluation efforts across STEM equity initiatives in Australia.