Search
Generic filters

Registered report is a relatively new article type that involves a 2-stage submission process:

  • Stage 1, pre-data collection, initial manuscript submission and review
  • Stage 2, after study completion, full manuscript is submitted for re-review

The main benefits of this two-stage approach are:

  • Once the methods and proposed analyses are provisionally accepted in Stage 1, the journal will commit to publishing the results regardless of the outcome, provided the final study conforms to the initially approved proposal and meets all quality checks. This means that publication bias is reduced as negative results will not prevent publication.
  • Peer review of the research proposal provides an opportunity for the authors to receive constructive critical feedback that may help them to fine-tune the study design prior to conducting the experiment.
  • This process can help reduce researcher bias.
  • This process may enhance the credibility of the work.

Stage 0: Initial Proposal

Authors are strongly encouraged to submit an initial proposal prior to submitting their stage 1 registered report. This proposal will be shared with the senior editors for registered reports for initial assessment. While optional, this extra step is beneficial for authors because it helps ensure the planned study is appropriate for the registered report format and that reviewers are available to expedite the peer review process upon submission of the Stage 1 manuscript.

The initial proposal should contain:

  1. A brief description of the planned study – background, research question, basic proposed methodology (1 paragraph).
  2. A justification for why a Registered Report (RR) is preferred over traditional submission types for the proposed study (1-3 sentences). Authors are encouraged to refer to the likely replication value of the research. High-value replication studies are welcomed in addition to novel studies.
  3. Sufficient detail to evaluate whether the proposed study is likely to be of high quality and appropriate for the RR format. Authors may want to consult the Stage 1 Development Checklist [INSERT LINK] and upload a copy of the completed design planner as part of their initial proposal.
  4. Additional details, including a full list of potential authors, the estimated date when the stage 1 manuscript will be ready to be submitted, and the target date that data collections is expected to begin.  

After receipt of the Stage 0 proposal, the senior editor writes back to invite the author to submit a Stage 1 manuscript, or to recommend the traditional publication route. Decision-making is based on whether the authors make a good case for the study to be conducted as a RR and whether the proposed manuscript is a good fit with the mission and scope of the journal. No commitments are made at this stage. 

Stage 1: Initial Manuscript Submission and Review

At stage 1, the author submits

  • Cover letter
  • Introduction
  • Preliminary data/studies
  • Methods
  • Disclosures

The Stage 1 cover letter should include:

  • A brief scientific case for consideration. Authors are encouraged to refer to the likely replication value of the research. Replication studies are welcome in addition to novel studies.
  • An anticipated timeline for completing the study if the initial submission is accepted.

During Stage 1 submission, authors will be asked to affirm the following:

  •  All necessary support (e.g. funding, facilities) and approvals (e.g. ethics) are in place for the proposed research. Note that manuscripts will be generally considered only for studies that are able to commence immediately; however, authors with alternative plans are encouraged to contact the journal office for advice.
  • The authors agree to share their raw data, any digital study materials, and analysis code as appropriate.
  • Following Stage 1 in principle acceptance, the authors agree to register their approved protocol on the Open Science Framework (https://osf.io/) or other recognized repository, either publicly or under private embargo until submission of the Stage 2 manuscript.
  • If the authors later withdraw their paper, they agree to the Journal of Speech, Language, and Hearing Research publishing a short summary of the pre-registered study under a section “Withdrawn Registrations.”

Following initial screening, the Stage 1 submission will be sent for in-depth peer review. In considering papers at the registration stage, reviewers will be asked to assess:

  1. The importance of the research question(s).
  2. The logic, rationale, and plausibility of the proposed hypotheses.
  3. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
  4. Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
  5. Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks.

After Stage 1 peer review, manuscripts may be offered in principle acceptance (IPA). Following IPA, the authors will then proceed to conduct the study, adhering exactly to the peer-reviewed procedures. Once the study is complete, authors prepare and resubmit their manuscript for re-review (Stage 2). The review process is illustrated in the flow chart below.

Authors are reminded that any deviation from the stated experimental procedures, regardless of how minor it may seem to the authors, could lead to rejection of the manuscript at Stage 2. In cases where the pre-registered protocol is altered after IPA due to unforeseen circumstances (e.g. change of equipment or unanticipated technical error), the authors must consult the editorial board immediately for advice, and prior to the completion of data collection. 

Stage 2: Full Manuscript Review

In stage 2, the author submits the completed manuscript, including results and discussion, which is then sent for an expedited peer review. Please note that no substantive changes can be made to the introduction and methods submitted in Stage 1. Pending quality checks, the journal will commit to publishing the results.

The resubmission will most likely be considered by the same reviewers as in Stage 1, but could also be assessed by new reviewers. In considering papers at Stage 2, reviewers will be asked to decide:

  1. Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)
  2. Whether the Introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission (required)
  3. Whether the authors adhered precisely to the registered experimental procedures
  4. Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative
  5. Whether the authors’ conclusions are justified given the data 

Reviewers are informed that editorial decisions will not be based on the perceived importance, novelty or conclusiveness of the results. Thus, while reviewers are free to enter such comments on the record, they will not influence editorial decisions. Reviewers at Stage 2 may suggest that authors report additional post hoc tests on their data; however, authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 2 review criteria.

Once accepted at this stage, the full manuscript is published.

More details about Registered Reports is available from the Center for Open Science  at https://www.cos.io/initiatives/registered-reports

The Review Process for Registered Reports

Registered Report Q&A

How does the review process for registered reports work?

In Stage 1, reviewers evaluate the importance of the research question and quality of the proposed design. At the conclusion of this stage, the editor may reject the submission, request revisions that may be sent back to reviewers, or accept the submission “in principle.”

If a paper is accepted in principle, the author(s) are asked to register the report on a registry (e.g., ClinicalTrials.gov, OSF) and proceed to conduct the study, data analysis, and write up the results.

In Stage 2, researchers resubmit completed papers that have been accepted in principle after the proposed research has been conducted. Reviewers are asked to evaluate whether the researchers carried out the research in the way they proposed in Stage 1 or adequately justified any divergences from the pre-analysis plan. ASHA ojurnals will make every effort to recruit the same reviewers from Stage 1, but if reviewers are unavailable, we may recruit a new reviewer and provide this person with all of the details from Stage 1. Submissions that clear this stage are accepted for publication and subjected to the same publication process as other submissions (e.g., checks for reproducible results, etc.)

Like a traditional paper, the Stage 1 review process could take 3-6 months and more if extensive revisions are requested, and author(s) should account for the review process when planning their study timeline.

What are the criteria for evaluating registered reports?

Stage 1

In Stage 1, reviewers are asked to review the registered report along five dimensions:

  1. Importance of the research question
  2. Soundness of proposed hypothesis
  3. Soundness and feasibility of proposed design (including statistical power)
  4. Whether proposed experiment offers an adequate and appropriate test of hypothesis
  5. Whether proposed methods are appropriate and sufficiently detailed

Like all manuscripts submitted to ASHA journals, the editors have the option of immediately rejecting the registered report if they believe that it is fundamentally lacking on any of the five dimensions. Registered report sent out for peer review will be considered  in a similar way. The editor may conditionally accept the registered report, ask for revisions, or reject it. Editors are encouraged to set a high bar for registered reports. The proposed revisions should be doable and straightforward as well as promise to be reasonably successful at addressing reviewers’ concerns.

Stage 2

Registered reports that are accepted-in-principle are subject to a second stage review where, to the extent possible, the original reviewers are re-contacted and asked to assess the manuscript along four dimensions listed below.

  1. Research question and rationale for hypothesis did not change from Stage 1.
  2. Experimental procedures detailed in the registered report were followed closely and any departures are noted and justified.
  3. Unregistered post hoc analyses are clearly labeled, justified, methodologically sound, and informative.
  4. Conclusions are justified by the data. (Considerations of data quality fall here).

Reviewers are asked not to consider the perceived importance, novelty, or clarity of the empirical results.

If the reviewers and editor conclude that the manuscript did not adhere to the registered report, the manuscript will be either rejected or, in the case that the editor deems a revision to be eminently doable, given a chance to make revisions.

Accepted registered reports must meet guidelines for publishing other manuscripts in the ASHA journals.

What is the page limit and can I have an appendix?

The suggested maximum page limit for registered reports is 40 manuscript pages. Authors are encouraged to use an online appendix as a way to provide details that reviewers need to evaluate proposed experimental designs and supplemental materials for , as applicable.

How detailed do I need to be when describing the research design in a registered report?

As detailed as possible. Please use an appendix as a way to provide full details of experimental conditions, measures, and the like. Report everything that you would in a traditional paper, including power analyses, experimental stimuli, question wording, details about measurement (e.g., recoding, factor analysis, etc.), rationales for excluding data, details about statistical models, thresholds for inferential statistics, etc.

As is the case with a traditional paper, the burden is on the author(s) to organize the information in a way that is clear and accessible to reviewers.

What if I need to make unforeseen adjustments to experiments in a registered report once I am in the field?

You will need to note and justify in the main text of the registered report any divergences from the pre-analysis plan that was accepted in principle.

If unforeseen circumstances cause the authors to deviate from the design and protocol detailed in the accepted-in-principle registered report, authors are encouraged to reach out to the editorial office for guidance. Senior Editors have the option of consulting with the first stage reviewers.

Can I conduct analyses that I did not register in the first stage of the registered report review process?

It is best to register all of the planned analyses. Nonetheless, if an additional analysis only becomes apparent after the experiment is conducted, it is permissible to do additional post-hoc analyses, as long as you offer a defensible substantive or statistical rationale for the additional analysis and flag these as post hoc when writing up the data analysis. One of the strengths of registered research is that it makes transparent the distinction between confirmatory and exploratory analyses.