Open Workflows
Preregistration
Preregistration is the practice of registering a study design before data collection commences. It is a detailed plan of your research, providing an opportunity to outline and share your hypotheses, methodology and data analysis processes in a public registry before undertaking your research. Preregistration is a statement of intent and a step towards a more open and transparent way of undertaking research.
Preregistration is often peer-reviewed and can lead to conditional acceptance by journal editors once the study is complete.
Some of the more notable registries include:
- OSF Preregistration General registry by Open Science Framework
- AsPredicted.org General registry from the Wharton Credibility Lab, University of Pennsylvania
- ICTRP Registry Network World Health Organisation (Biomedicine)
Increasingly, journals are now offering preregistration services, including PLOS & American Psychological Association. By submitting your study design to a journal, you receive expert peer review feedback to ensure you are taking the best possible approach to your study. It also gives you an editorial decision based on the research rather than the results. This is considered part one of a two-stage registered report.
Benefits of preregistering
- Demonstrates research integrity
- Improves transparency
- Increases rigour
- Facilitates reproducibility
- Allows for peer review by experts in your field to ensure you take the best possible approach to your study
- Prevents cherry picking or Hypothesising After your Results are Known (HARKing) 10.1207/s15327957pspr0203_4
- Reduces risk of p-hacking (tinkering with data) or data dredging
- Provides an opportunity for you to commit to those plans before conducting the research
- Can lead to conditional acceptance in a journal, regardless of the study’s results
- Improves public confidence in the results
Good preregistration
Preregistration should be a read-only document that outlines the study and analysis. Ideally, it includes a time stamp of when it was created and must be completed before the study. Preregistration should be stored in a repository.
Useful resources
Registered reports
Registered reports are a form of publication where study design is peer reviewed and pre-accepted at the preregistration stage and then followed up with results and discussion. Introduced in 2012, registered reports are viewed as a means of increasing rigour and countering publication bias. They align with scientific values and practices and aim to reduce the likelihood of HARKing 10.1207/s15327957pspr0203_4
By separating the results from the scientific process, registered reports are designed to improve reproducibility and are an essential element of open science.
Stages of registered reports
Stage 1: the author submits their research question, hypotheses, methodology and analysis plans. This is reviewed with any revisions made. Proposals are then given in-principle acceptance (IPA). Authors register their study design in a repository or under temporary embargo.
Stage 2: once the research has been undertaken, authors submit a manuscript which includes the approved study design, results, and discussion. This manuscript is peer reviewed, with a focus on compliance with the study design and the conclusions.
Over 300 journals offer registered reports as an article type. Find a journal who offers registered reports at the Centre for Open Science.
Useful resources
- Peer Community in Registered Reports
- Benefits and workflows of Registered Reports – provides a good overview of Registered Reports including workflows and benefits
- Ten simple rules for writing a Registered Report
“RRs numerically outperformed non-RRs on every criterion, showing statistically robust and large improvements in attributes such as methodological rigour and overall article quality, while being statistically indistinguishable from comparison papers in terms of features such as novelty and creativity.” 10.1038/s41562-021-01193-7
Open peer review
Peer review is the evaluation of your research output by experts in your field. It is a means of validating research and is an assessment of quality. Historically, peer review is performed as anonymised or double-anonymised, but open peer review aims to change this. Open peer review is an umbrella term that aims to bring openness and transparency to what has largely been a very closed process.
Characteristics of open peer review
- Authors and reviewers are aware of each other’s identities
- Review reports are published with the relevant article
- The wider community can contribute to the review process
- Open peer review allows for review and commenting on the final version of record (VoR)
Benefits of open peer review
- Greater transparency which means reviewers are accountable for their assessment
- Can reduce reviewer bias including potential conflicts of interest
- Community peer review can speed up the review process
- Encourages interaction between authors and reviewers
- Credit for areas of the publication workflow that are often not recognised
- Improves reviewer consistency
- Fosters scientific discussion
- Supports a culture of openness
Ways to engage with open peer review:
- If the opportunity is presented to you, consider making your identity known as an author or as a reviewer
- If you are an editor of a publication, discuss ways to make your peer review process more open
- Consider publishing in a journal that offers open peer review or an open peer review service such as F1000Research, PubPeer & ScienceOpen
Useful resources
- PLOS Open peer review - an overview of open peer review including why it’s important and how to engage with it
- Open Peer Review, Metrics, and Evaluation – Part of the Open Science Training Handbook, this webpage provides a good overview of open peer review
- Guidelines for open peer review implementation – BMC article exploring best practice in open peer review
Open notebooks
Open notebook science is the practice of making your research records, journals and workbooks open online to allow reuse and redistribution of the material. They are a means of opening science in the early stages to accelerate scientific discovery. Typically involving an online notebook where information about all stages of the research including raw data is inputted, open lab notebooks are an important part of the open science movement. Sharing information during the early stages of scientific discovery allows others to avoid going down dead ends while also creating opportunities for collaboration with researchers undertaking similar research. In this way, they are a means of increasing efficiency in the research process.
Open notebooks allow researchers to share their experimental data and protocols online as they go rather than waiting for the preprint stage or later.
Benefits of open notebooks
- Reduce duplication of work
- Can lead to opportunities for collaboration
- Greater transparency
- Create a more open working environment
Useful resources
- Openlabnotebooks.org An open notebooks initiative
- Research Data Management Guide A how-to guide on Research Data Management at the University of Waikato
Need help?
The Open Research Team at the University of Waikato is committed to working with academics who wish to make their research open in ways that work for them.