Utilizing Artificial Intelligence to Prepare Structured Abstracts in Brain Tumor Clinical Practice Guidelines in Japan – A Case Stud - Abstract
Background: Developing clinical practice guidelines requires significant time and effort, especially in the systematic review process to create structured
abstracts, and large language models (LLMs) have the potential to help this process. This study describes the use of artificial intelligence (AI) to create structured
abstracts for the Brain Tumor Clinical Practice Guideline 2025 by the Japan Society for Neuro-Oncology.
Methods: Ten papers on palliative care and quality of life for diffuse glioma patients were selected in this study. First, the abstracts were retrieved from
PubMed and translated into Japanese using ChatGPT 4o. Then, the structured abstracts were refined manually. Second, ChatGPT 4 Turbo processed the full
text PDFs to optimize the content. Then, the similarity between the original and AI-generated abstracts was assessed using sentence-bidirectional encoder
representations from transformers (S-BERT).
Results: The AI successfully generated structured abstracts within a few minutes. The subjective evaluation showed that the AI-generated translations were
useful. The S-BERT analysis improved the similarity scores from the primary phase (0.581) to the secondary phase (0.662), with a p-value of 0.01. Note that
no significant correlation was observed between the S-BERT scores and the document length.
Conclusion: AI-assisted structured abstract generation proved to be a practical and time-saving approach in guideline development. However, human
reviews remain crucial to ensure sufficient quality. Future research should focus on blind evaluations and utilize larger datasets for further validation.