terms of reporting statisitical findings) has been chal- lenged by numerous researchers and statisticians [11, 14,22]. The most serious argument against this per- spective relates to the influence that sample size has in determining the significance of any statistical test. Hayes , for example, pointed out that virtually any study can be made to yield statistically significant re- sults if the researcher includes enough subjects. To avoidthepossibilityofmisleadingresearchconsumers, thelatesteditionofthePublicationManualsuggests thatall authorsprovideestimatesofpracticalorclinical significance along with all statistical significance tests reported in the Results section. A quantitative Results section should be limited to
the findings obtained by the researcher(s) in the cur- rent investigation. Speculation concerning what those findings mean in a larger context is reserved for the Discussion section. TheResults sectionsofqualitativelyorientedarticles
displaymuchmorevarietyin thecontentandmannerof presentation than is found in quantitative studies. Be- cause the researcher’ssubjective interpretationshelp to shape the processes and outcomes of qualitative inves- tigations, results areoftenframedinbroad, interpretive contexts. In that regard, the lines between the Results and Discussion sections are often blurred in qualitative research. Researchers(qualitativeandquantitative)commonly
use tables and figures to summarize and/or graphically present their results. There is wide variability in the content and presentation of tables and figures, with the most important universal requirement being easy interpretability for the reader.
TheDiscussionsectionservesas the researcher’s fo- rum to go beyond the current investigationand discuss the contributions of study findings to existing litera- ture, theory, and professional practices. The first part of a thoughtful Discussion is often an analysis of the
study’s results vis a vis the research questions and hy- potheses. Researchers should begin with a discussion of whether the hypotheses were upheld, posit possible explanationsfor thoseoutcomes,anddrawimplications fromthefindingsbackto the researchproblemthatwas identified in the Introduction. If the results provide a warrant for modifying or re-testing the conceptual frameworkuponwhichthe investigationwasbased, the Discussion section is the place to suggest a reformula- tion of the underlying theory. Researchers should also include a statement of the scientific limitations of the currentstudy,alongwith specificrecommendationsfor future research. Finally, the researcher ends the arti- cle with a cogent summary of the conclusions, in the mostgeneralsense, thatcanbedrawnfromthemethods and findings of the current study. Some authors use a separate Conclusion section for this purpose.
The final section of a research article is always a listing of the references that were cited in the body of the text. References are listed in alphabetical order, according to authors’ last names. Most rehabilitation journals requireadherenceto theAmericanPsycholog- ical Association’s  guidelines regarding the compo- sition of the References section.
3. A scale for critiquing research manuscript and articles
Understanding the components, organization, and composition of a research article will help make Work subscribersbetterinformedconsumersas theyreadem- piricallybasedpublications. As readersdigest thecon- tents of research articles and apply them to their prac- tices, the “anatomy” of research reports can serve as a useful rubric for critically analyzing the quality, con- tent, and practical significance of published articles. Table 1 presents specific questions for conducting a section-by-section critique of a rehabilitation research article.
This article examined the components of a research articleandprovidedguidelinesforconductingacritical analysis of published research. Although the descrip- tions of the components of a research article provide
P. Rumrill et al. / Guidelines for evaluating research articles 263
only a skeletal summary of what should be included in a published research article, they should provide the readerenoughinformationtobothpreparemanuscripts forpublicationandevaluate the empirical research that appears in Work and other rehabilitation journals.
 Washington,D.C.,AmericanPsychological Association, Pub- lication manual of the American Psychological Association, (Fourth Edition), 1994.
 Bandura, A., Social foundations of thought and action: A social cognitive theory, Englewood Cliffs, NJ: Prentice-Hall, 1986.
 Bellini, J., Bolton, B. and Neath, J., Rehabilitation counselors assessments of applicants functional limitations as predictors of rehabilitation services provided, Rehabilitation Counseling Bulletin 41(4) (1998), 242–258.
 Bellini, J. and Rumrill, P., Research in rehabilitation counsel- ing: A guide to design, methodology, and utilization, Spring- field, IL: Charles C. Thomas Publishers, 1999.
 Bolton, B. and Brookings, J., Development of a multifaceted definition of empowerment, Rehabilitation Counseling Bul- letin 39(4) (1996), 256–264.
 Diksa,E.andRogers,E.,Employer concernsabout hiringper- sons with psychiatric disability: Results of the employer atti- tude questionnaire, Rehabilitation Counseling Bulletin 40(1) (1996), 31–44.
 Hayes,W.,Statistics for psychologists, NewYork: Holt,Rine- hart, and Winston, 1981.
 Heppner, P.,Kivlighan, D. and Wampold, B., Research design in counseling, Pacific Grove, CA: Brooks/Cole, 1992.
 Heppner, P.,Kivlighan, D. and Wampold, B., Research design in counseling, (2ndEdition), Pacific Grove, CA:Brooks/Cole, 1999.
 Hershenson, D., A systems reformulation of a developmental modelofworkadjustment, Rehabilitation Counseling Bulletin
40(1) (1996), 2–10.  Hunter, J.,Needed: Aban on the significance test, Psycholog-
ical Science 8 (1997), 3–7.  Kazdin, A., Research design in clinical psychology, (2nd Edi-
tion), Needham Heights, MA: Allyn and Bacon, 1998.  Krause, J. and Anson, C., Self-perceived reasons for unem-
ployment cited by persons with spinal cord injury: Relation- ship to gender, race, age, and level of injury, Rehabilitation Counseling Bulletin 39(3) (1996), 217–227.
 McClure, P., Determining the significance of significance: P- values, effect size, and clinical judgement, Journal of Hand Therapy 12 (1999), 40–41.
 McMillan, J. and Schumacher, S., Research in education: A conceptual introduction, (Fourth Edition), New York: Long- man, 1997.
 Munley, P., Sharkin, B. and Gelso, C., Reviewer ratings and agreement on manuscripts reviewed for the Journal of Counseling Psychology, Journal of Counseling Psychology 35 (1988), 198–202.
 Orwell, G.,Politics and the English language, in: A collection of essays, G. Orwell ed., San Diego: Harcourt, Brace, and Jovanovich, 1946, pp. 156–171.
 Ravesloot, C. and Seekins, T., Vocational rehabilitation coun- selors’ attitudes toward self-employment outcomes, Rehabili- tation Counseling Bulletin 39(3) (1996), 189–201.
 Rumrill, P.,Roessler, R.andDenny, G., Increasing confidence in the accommodation request process among persons with multiple sclerosis: A career maintenance self-efficacy inter- vention, Journal of Job Placement 13(1) (1997), 5–9.
 Rumrill, P.,Roessler, R. and Koch, L.,Surveying the employ- ment concerns of people with multiple sclerosis: A participa- tory action research approach, Journal of Vocational Rehabil- itation 12(2) (1999), 75–82.
 Schaller, J. and Parker, R., Effect of graduate research in- struction on perceived research anxiety, research utility, and confidence in research skills, Rehabilitation Education 11(4) (1997), 273–287.
 Thompson, B., AERA editorial policies regarding statistical significance testing: Three suggested reforms, Educational Researcher 25(2) (1996), 26–30.
P. Rumrill et al. / Guidelines for evaluating research articles 261
Table 1 A scale for critiquing research articles
Instructions: Answer the following questions regarding the article, “ ”. Use examples from the article to support your analyses.
A. Title 1. Does the title describe the study? 2. Do the key words of the title serve as key elements of the article? 3. Is the title concise, i.e., free of distracting or extraneous phrases?
B. Abstract 4. Does the abstract summarize the study’s purpose, methods, and findings? 5. Does the abstract reveal the independent and dependent variables under study? 6. Are there any major premises or findings presented in the article that are not mentioned in the abstract? 7. Does the abstract provide you with sufficient information to determine whether you would be interested in reading the entire article?
C. Introduction 8. Is the research problem clearly identified? 9. Is the problem significant enough to warrant the study that was conducted? 10. Do the authors present a theoretical rationale for the study? 11. Is the conceptual framework of the study appropriate in light of the research problem? 12. Do the author’s hypotheses and/or research questions seem logical in light of the conceptual framework and research
problem? 13. Are hypotheses and research questions clearly stated? Are they directional? 14. Overall, does the literature review lead logically into the Method section?
D. Method 15. Is the sample clearly described, in terms of size, relevant characteristics, selection and assignment procedures, and
whether any inducements were used to solicit subjects? 16. Do the instruments described seem appropriate as meausres of the variables under study? 17. Have the authors included sufficient information about the psychometric properties (e.g., reliability and validity) of
the instruments? 18. Are the materials used in conducting the study or in collecting data clearly described? 19. Are the study’s scientific procedures thoroughly described in chronological order? 20. Is the design of the study identified (or made evident)? 21. Do the design and procedures seemappropriate in light of the research problem, conceptual framework, and research
questions/hypotheses? 22. Overall, does the method section provide sufficient information to replicate the study?