This Article

Citations


Creative Commons License
Except where otherwise noted, this work is licensed under Creative Commons Attribution-NonCommercial 4.0 International License.

Readability of Patient Education Materials on the American Association for Surgery of Trauma Website


1 Warren Alpert Medical School, Brown University, Providence, USA
2 Division of Trauma and Surgical Critical Care, Department of Surgery, Rhode Island Hospital, Providence, USA
3 Division of Orthopaedic Trauma, Department of Orthopaedic Surgery, Rhode Island Hospital, Providence, USA
*Corresponding author: Adam E. M. Eltorai , Warren Alpert Medical School, Brown University, Box G-9247, Providence, RI 02903, USA. Tel: +1-4018633330, Fax: +1-4018635096, E-mail: adam_eltorai@brown.edu.
Archives of Trauma Research. 3(2): e18161 , DOI: 10.5812/atr.18161
Article Type: Research Article; Received: Feb 11, 2014; Revised: Mar 28, 2014; Accepted: Mar 30, 2014; epub: May 1, 2014; collection: May 2014

Abstract


Background: Because the quality of information on the Internet is of dubious worth, many patients seek out reliable expert sources. As per the American Medical Association (AMA) and the National Institutes of Health (NIH) recommendations, readability of patient education materials should not exceed a sixth-grade reading level. The average reading skill of U.S. adults is at the eighth-grade level.

Objectives: This study evaluates whether a recognized source of expert content, the American Association for Surgery of Trauma (AAST) website’s patient education materials, recommended readability guidelines for medical information.

Materials and Methods: Using the well-validated Flesch-Kincaid formula to analyze grade level readability, we evaluated the readability of all 16 of the publicly-accessible entries within the patient education section of the AAST website.

Results: Mean ± SD grade level readability was 10.9 ± 1.8 for all the articles. All but one of the articles had a readability score above the sixth-grade level. Readability of the articles exceeded the maximum recommended level by an average of 4.9 grade levels (95% confidence interval, 4.0-5.8; P < 0.0001). Readability of the articles exceeded the eighth-grade level by an average of 2.9 grade levels (95% confidence interval, 2.0-3.8; P < 0.0001). Only one of the articles had a readability score below the eighth-grade level.

Conclusions: The AAST’s online patient education materials may be of limited utility to many patients, as the readability of the information exceeds the average reading skill level of adults in the U.S. Lack of patient comprehension represents a discrepancy that is not in accordance with the goals of the AAST’s objectives for its patient education efforts.

Keywords: Patient Education Materials; Online Health Information; Readability; Comprehension, Flesch-Kincaid Formula

1. Background


Health literacy is the ‘‘capacity to obtain, interpret, and understand basic health information and services and the competence to use such information and services to enhance health’’ (1). An individual’s health literacy is an independent predictor of their health-related quality of life (2) with low health literacy being associated with worse overall health (3), increased hospitalizations (4), increased complications that require hospital attention (3), poor understanding of one’s disease (5), and an overall increase in health-care costs (6).

Patients are increasingly using the internet to acquire health information (7, 8) with over eight million Americans seeking health information from the internet every day (7). However, the ability to utilize online health information to make healthcare decisions depends on the ability to comprehend the material (8). As an expert website, the American Association for Surgery of Trauma (AAST) has an obligation to make sure its content is not only accurate, but also readable.

The reading comprehension level determines the readability that a text must have so that a reader understand the written materials (9). The Flesch-Kincaid Grade Level (FKGL) formula is a validated and common instrument used to determine the readability of written materials in terms of the United States academic grade levels (10-18). The FKGL formula was originally used by the United States Army for assessing the difficulty of technical manuals and became the Department of Defense Military Standard. The higher the FKGL of a text, the more difficult it is to read and comprehend, requiring more advanced reading skills.

The average American adult reads at an eighth-grade level (19, 20). Approximately, 47% of adults in the United States “experience considerable difficulty in performing tasks that required them to integrate or synthesize information from complex or lengthy texts” (21). Nearly one-fifth of adults in America cannot comprehend fourth-grade-level text (20). Doak and Doak investigated the reading level of patients at a public hospital, and found the average patient read at a seventh-grade level, despite having reported high school graduation (22).

Organizations including the American Medical Association (AMA) and the National Institutes of Health (NIH) recommend that the readability of patient education materials should be no greater than a sixth-grade reading level (23-27). Despite these recommendations, several studies suggested that current patient education materials might be at a reading level that is too complex for most patients to comprehend (10, 11, 13, 14, 16, 18, 20, 28).

2. Objectives


To our knowledge, the readability of the online patient information on the AAST website had not previously been assessed. Therefore, the goal of this study was to evaluate the readability of patient education materials on the publicly available AAST website to determine whether it met recommended readability guidelines for medical information. We hypothesized that the readability of these online materials would be a FKGL of > 6.

3. Materials and Methods


This study analyzed the patient education materials on the AAST website (http://www.aast.org/GeneralInterest/Links.aspx). The study was exempt from Institutional Review Board (IRB) review. The website is publicly accessible and was accessed on October 23, 2013. All 16 patient education articles were assessed for this study. No participants were recruited for this study.

Text from the website was copied in plain text format into individual Microsoft Office Word 2010 (Microsoft Corporation, Redmond, WA, USA) documents. Copyright notes, date stamps, author information, hyperlinks, citations, tables, and any other text not directly related to patient education were deleted. To avoid underestimating the readability level, numbers, decimals, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes within a sentence were removed, as recommended by Flesch and others (29, 30).

The FKGLs were obtained for each document by using the readability calculator built into the Word. The FKGL is calculated by the following formula:

[0.39 × (average number of words per sentence)] + [11.8 × (average number of syllables per word)] − 15.59.

Sequentially selecting “File, Options, Proofing, and Show readability statistics” enabled the built-in FKGL calculator on Microsoft Word. The FKGL for each document was automatically displayed after grammar and spelling were checked. Each FKGL was recorded. It was more convenient to use the Word version of the FKGL calculator since the text that was being analyzed was already copied and pasted into a Word document. To compare the mean FKGL with the recommended readability level (the AMA and NIH as well as the average American adult reading level), an unpaired t-test was used. A statistical cutoff of P <0.05 was used for the determination of significance.

4. Results


The mean readability of all 16 patient information pages was grade level 10.9±1.8. Except one, the rest of the articles (93.8%) had a readability score above the sixth-grade level, the maximum level recommended by the AMA and the NIH. The readability of the articles exceeded this level by an average of 4.9 grade levels (95% CI: 4.0-5.8; P < 0.0001) (Table 1). Only one of the articles had a readability score below the eight-grade level, the average reading skill level of the United States adults. The readability of the articles exceeded this level by an average of 2.9 grade levels (95% confidence interval, 2.0-3.8; P < 0.0001) (Figure 1).

Table 1.
The Flesch-Kincaid Grade Level and Levels Above Sixth-Grade for Each Analyzed Article a
Figure 1.
The Flesch-Kincaid Grade Levels of Patient Education Articles Available on the American Association for Surgery of Trauma (AAST) Website

5. Discussion


The majority of the patient education materials on the AAST website has readability levels far above the reading comprehension level of the average patient and thus, may contain information that is too difficult to comprehend for a substantial portion of the patient population. Although the reading skills of the intended audience should be taken into consideration when patient education materials are developed, this must be weighed against the necessity of providing complete and accurate medical information. The readability of patient education materials can be improved by using simpler terms, shorter sentences, and plentiful illustrations (10, 26, 31, 32). According to the National Assessment of Adult Literacy, only 12% of adults have the health literacy skills needed to manage their health and prevent disease (33). Lower health literacy is associated with reduced health-related quality of life, reduced general health, and increased hospitalizations as well as complications. Together, these outcomes yield substantial increases in overall health-care costs. Therefore, improving health literacy may in turn improve patient’s outcomes.

5.1. Limitations of Study

This study had several potential limitations. We did not specifically assess the reading skills of the website’s readers as they may differ from the general population; however, trauma patients likely have similar or potentially lower rates of reading comprehension compared to the general public. A possible additional limitation is that the FKGL only evaluates text (i.e. not diagrams) and does not directly measure comprehensibility. Finally, although we only reviewed one society’s patient education materials, the sample is relevant as surgeons increasingly refer their patients to such professional websites (34-36).

5.2. Important Conclusion

This study found that the AAST patient education website currently contains information at a readability level too advanced for most patients to comprehend. Improving readability of patient education materials may improve patient understanding and thus, positively affect health outcomes. Moreover, a better-informed public would be a powerful ally in the efforts to increase trauma awareness, public funding, and injury prevention.

Acknowledgments

None declared.

Footnotes

Implication for health policy/practice/research/medical education: This study found that the American Association for Surgery of Trauma (AAST) patient education website currently contains information at a readability level too advanced for most patients to comprehend. Improving readability of patient education materials may improve patient’s understanding and thus, positively affect health outcomes. Moreover, a better-informed public would be a powerful ally in the efforts to increase trauma awareness, public funding, and injury prevention.
Authors’ Contributions: Adam E M Eltorai: literature search, study design, data collection, data analysis, data interpretation, writing, critical revision; Soha Ghanian: data collection, data analysis, data interpretation; Charles A Adams: data interpretation, writing, critical revision; Christopher T Born: data interpretation, writing, critical revision; Alan H Daniels: literature search, study design, data analysis, data interpretation, writing, critical revision.

References


  • 1. US Department of Health and Human Services . Healthy People 2010. Office of Disease Prevention and Health Promotion; 2000.
  • 2. Wang C, Li H, Li L, Xu D, Kane RL, Meng Q. Health literacy and ethnic disparities in health-related quality of life among rural women: results from a Chinese poor minority area. Health Qual Life Outcomes. 2013;11:153. [DOI] [PubMed]
  • 3. Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, et al. Association of health literacy with diabetes outcomes. JAMA. 2002;288(4):475-82. [PubMed]
  • 4. Baker DW, Gazmararian JA, Williams MV, Scott T, Parker RM, Green D, et al. Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health. 2002;92(8):1278-83. [PubMed]
  • 5. Williams MV, Baker DW, Parker RM, Nurss JR. Relationship of functional health literacy to patients' knowledge of their chronic disease. A study of patients with hypertension and diabetes. Arch Intern Med. 1998;158(2):166-72. [PubMed]
  • 6. Weiss BD, Palmer R. Relationship between health care costs and very low literacy skills in a medically needy and indigent Medicaid population. J Am Board Fam Pract. 2004;17(1):44-7. [PubMed]
  • 7. Fox S. Online Health Search. Pew Internet and American Life Project. Online Health Search 2006. 2006; Available from: http://www.pewinternet.org/~/m...
  • 8. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285(20):2612-21. [PubMed]
  • 9. Department of Health and Human Services . Pretesting in health communications. National Institutes of Health, Bethsda: DHHS Publication; 1982.
  • 10. Albright J, de Guzman C, Acebo P, Paiva D, Faulkner M, Swanson J. Readability of patient education materials: implications for clinical practice. Appl Nurs Res. 1996;9(3):139-43. [PubMed]
  • 11. Cooley ME, Moriarty H, Berger MS, Selm-Orr D, Coyle B, Short T. Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum. 1995;22(9):1345-51. [PubMed]
  • 12. Bluman EM, Foley RP, Chiodo CP. Readability of the Patient Education Section of the AOFAS Website. Foot Ankle Int. 2009;30(4):287-91. [DOI] [PubMed]
  • 13. Badarudeen S, Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 2008;90(1):199-204. [DOI] [PubMed]
  • 14. Sabharwal S, Badarudeen S, Unes Kunju S. Readability of online patient education materials from the AAOS web site. Clin Orthop Relat Res. 2008;466(5):1245-50. [DOI] [PubMed]
  • 15. Yi PH, Ganta A, Hussein KI, Frank RM, Jawa A. Readability of arthroscopy-related patient education materials from the American Academy of Orthopaedic Surgeons and Arthroscopy Association of North America Web sites. Arthroscopy. 2013;29(6):1108-12. [DOI] [PubMed]
  • 16. Wang SW, Capo JT, Orillaza N. Readability and comprehensibility of patient education material in hand-related web sites. J Hand Surg Am. 2009;34(7):1308-15. [DOI] [PubMed]
  • 17. Polishchuk DL, Hashem J, Sabharwal S. Readability of online patient education materials on adult reconstruction Web sites. J Arthroplasty. 2012;27(5):716-9. [DOI] [PubMed]
  • 18. Vives M, Young L, Sabharwal S. Readability of spine-related patient education materials from subspecialty organization and spine practitioner websites. Spine (Phila Pa 1976). 2009;34(25):2826-31. [DOI] [PubMed]
  • 19. Doak CC, Doak LG, Friedell GH, Meade CD. Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA Cancer J Clin. 1998;48(3):151-62. [PubMed]
  • 20. Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills. 2nd ed. Philadelphia, PA: JB Lippincott; 1996.
  • 21. Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: a first look at the results of the National Adult Literacy Survey. National Center for Education Statistics; 1993; Available from: http://files.eric.ed.gov/fullt...
  • 22. Doak LG, Doak CC. Lowering the silent barriers to compliance for patients with low literacy skills. Promot Health. 1987;8(4):6-8. [PubMed]
  • 23. Weiss BD. Health literacy: A manual for clinicians. Chicago, IL: American Medical Association Foundation and American Medical Association; 2003.
  • 24. National Institutes of Health. [cited October 21, 2013]; Available from: http://www.nlm.nih.gov/medline...
  • 25. Weiss BD, Coyne C. Communicating with patients who cannot read. N Engl J Med. 1997;337(4):272-4. [DOI] [PubMed]
  • 26. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health. 2005;30(3):213-9. [PubMed]
  • 27. Doak LG, Doak CC, Meade CD. Strategies to improve cancer education materials. Oncol Nurs Forum. 1996;23(8):1305-12. [PubMed]
  • 28. Murero M, D'Ancona G, Karamanoukian H. Use of the Internet by patients before and after cardiac surgery: telephone survey. J Med Internet Res. 2001;3(3):E27. [DOI] [PubMed]
  • 29. Flesch RF. How to Write Plain English: A Book for Lawyers, Consumers. 1st ed. New York, NY: Barnes and Noble; 1981.
  • 30. Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33(3):352-73. [DOI] [PubMed]
  • 31. Centers for Disease Control and Prevention. Scientific and technical information simply put. 2d ed. Atlanta, Georgia: Department of Health and Human Services; 1999.
  • 32. Jackson RH, Davis TC, Bairnsfather LE, George RB, Crouch MA, Gault H. Patient reading ability: an overlooked problem in health care. South Med J. 1991;84(10):1172-5. [PubMed]
  • 33. National Center for Education Statistics. Washington, DC: U.S. Department of Education; 2006.
  • 34. Rozental TD, Bozentka DJ, Beredjiklian PK. Patient education through the Internet: academic and private practice sites. Clin Orthop Relat Res. 2004;(421):50-3. [PubMed]
  • 35. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am. 2000;82-A(11):1540-3. [PubMed]
  • 36. Purvis JM, Alexander AH, Einhorn TA, Griffin LY. American Orthopaedic Association symposium: Evaluating the flood of orthopaedic media and marketing information. J Bone Joint Surg Am. 2005;87(6):1392-8. [DOI] [PubMed]

Table 1.

The Flesch-Kincaid Grade Level and Levels Above Sixth-Grade for Each Analyzed Article a

Article Title Readability Grade Level (FKGL) Grade Levels Above Recommended (FKGL–6.0)
“Aspiration in Trauma” 12.0 6.0
“Blunt Cardiac Injury” 12.0 6.0
“Blunt Splenic Trauma” 12.0 6.0
“Child Passenger Safety” 8.5 2.5
“Cost of Injury” 12.0 6.0
“Critical Care Illness” 12.0 6.0
“Discharge Instructions for Wound Care” 6.0 0.0
“Epidemiology and Injury Prevention” 11.9 5.9
“Field Triage of the Injured Patient” 12.0 6.0
“Mechanical Ventilation in the Intensive Care Unit” 12.0 6.0
“Pelvis Injuries” 10.4 4.4
“Rib Fractures” 10.9 4.9
“Sports Concussions” 9.4 3.4
“Thromboembolic Disease” 12.0 6.0
“Trauma Systems” 11.0 5.0
“Traumatic Brain Injury Rehabilitation” 12.0 6.0
Mean 10.9 5.0
a Abbreviations: FKGL, Flesch-Kincaid grade level.

Figure 1.

The Flesch-Kincaid Grade Levels of Patient Education Articles Available on the American Association for Surgery of Trauma (AAST) Website
Eleven of the 16 articles had a readability grade level between 11 and 12, three of the articles had a readability level between nine and ten, one of the articles had a readability level between seven and eight, and only one of the articles had a readability level between four and six.