Readability of dermatology patient education materials: a comparative analysis of Turkish Dermatology Association brochures and texts generated by ChatGPT and Gemini ​

OpenAlex İngilizce

Background/aim: In modern healthcare, patient education materials (PEMs) are essential for bridging the gap between clinical knowledge and patient understanding. However, in Türkiye, where 64.6% of the adult population possesses inadequate or problematic health literacy, traditional medical brochures often exceed recommended reading levels. This study aimed to evaluate and compare the readability of Turkish dermatology PEMs prepared by the Turkish Dermatology Association (TDA) with those generated by artificial intelligence (AI) models. Materials and methods: A total of 68 TDA PEMs were analyzed. Readability was assessed using the Ateşman Readability Index. Responses were generated by AI models to questions extracted from these brochures under two conditions: "default" (no specific instructions) and "low-literacy prompted" (instruction to write at a sixth-grade level). Statistical comparisons were performed using the Kruskal-Wallis and Mann-Whitney U tests to evaluate differences in readability scores between the original TDA brochures and the four AI-generated groups. Effect size was calculated manually for each pairwise comparison and reported as r values. Results: The original TDA brochures had a mean readability score of 59.08 ± 5.80 (95% CI: 57.69-60.44), categorized as "medium difficulty." All AI-generated groups demonstrated significantly higher readability scores compared with the TDA brochures (p < 0.05). Effect sizes for TDA versus AI group comparisons ranged from r = 0.242 to r = 0.753. In the default condition, Gemini Pro 2.5 produced more readable and consistent texts than ChatGPT-4.5 (66.63 versus 62.66; p = 0.004). While specific prompting significantly improved ChatGPT performance (p < 0.001), no significant change was observed in Gemini scores, which were already high in the default condition. Prompted Gemini achieved the highest overall mean score (68.98 ± 4.66). Conclusion: The linguistic complexity of TDA brochures may create a barrier for patients with limited health literacy. AI models, particularly when prompted with specific instructions, can generate dermatological PEMs with significantly higher readability levels.