ASHA Journals Knowledge Base

What are the criteria for evaluating registered reports?

Stage 1

In Stage 1, reviewers are asked to review the registered report along five dimensions:

  1. Importance of the research question
  2. Soundness of proposed hypothesis
  3. Soundness and feasibility of proposed design (including statistical power)
  4. Whether proposed experiment offers an adequate and appropriate test of hypothesis
  5. Whether proposed methods are appropriate and sufficiently detailed

Like all manuscripts submitted to ASHA journals, the editors have the option of immediately rejecting the registered report if they believe that it is fundamentally lacking on any of the five dimensions. Registered report sent out for peer review will be considered  in a similar way. The editor may conditionally accept the registered report, ask for revisions, or reject it. Editors are encouraged to set a high bar for registered reports. The proposed revisions should be doable and straightforward as well as promise to be reasonably successful at addressing reviewers’ concerns.

Stage 2

Registered reports that are accepted-in-principle are subject to a second stage review where, to the extent possible, the original reviewers are re-contacted and asked to assess the manuscript along four dimensions listed below.

  1. Research question and rationale for hypothesis did not change from Stage 1.
  2. Experimental procedures detailed in the registered report were followed closely and any departures are noted and justified.
  3. Unregistered post hoc analyses are clearly labeled, justified, methodologically sound, and informative.
  4. Conclusions are justified by the data. (Considerations of data quality fall here).

Reviewers are asked not to consider the perceived importance, novelty, or clarity of the empirical results.

If the reviewers and editor conclude that the manuscript did not adhere to the registered report, the manuscript will be either rejected or, in the case that the editor deems a revision to be eminently doable, given a chance to make revisions.

Accepted registered reports must meet guidelines for publishing other manuscripts in the ASHA journals.

Related Questions