Introduction
Large language models (LLMs) show great potential to shape health care and medical education at large. Activities through which LLMs can augment clinical practice include serving as an ambient scribe to assist with documentation of clinical encounters, generating differential diagnoses, and formulating preliminary treatment plans. Research is just beginning to explore the numerous applications of LLMs in the field of psychiatry. A systematic review suggests that LLMs have the potential to transform the field through activities such as detecting mental health conditions via parsing clinical notes and social media posts, formulating psychodynamic profiles, and creating therapeutic materials.1 Within child and adolescent psychiatry, LLMs might be applied to create interactive digital interventions2 for mental health awareness among children, generate educational materials1 for teachers and community leaders, and produce practice guidelines for pediatricians and primary care providers.
Ongoing research suggests that there are numerous opportunities for LLMs in all phases of medical education. ChatGPT passes Steps 1, 2, and 3 with greater than 85% accuracy,3,4 suggesting a high level of medical content expected of early-career trainees represented within its knowledge. We have previously proposed use cases for LLMs within the preclinical curriculum to reinforce the learning of core Association of American Medical Colleges (AAMC) Entrustable Professional Activities, including formulating differential diagnoses, providing interactive clinical cases, and facilitating multiple-choice question review.5 Numerous use cases also exist for later stages of training, including helping clerkship students identify an appropriate initial workup and supporting board exam preparation. LLMs are a learning tool we are only just beginning to explore.
The Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Child and Adolescent Psychiatry6 outline competencies for trainees in child and adolescent psychiatry. One domain of these competencies (IV.B.1.b) is Patient Care and Procedural Skills. This domain includes skill development in evaluating and treating children from diverse backgrounds across the full spectrum of psychiatric disorders utilizing a wide array of techniques, as well as documenting these encounters. It is expected that trainees will primarily develop and demonstrate the aforementioned patient care skills through treating patients; standardized patient experiences are not typically implemented across training sites. Feedback is mostly preceptor-based, and there is not necessarily a uniform way to assess whether someone has developed competency in a particular skill. Given the dialogic nature of general medical practice,7 which has been more specifically rarefied in psychiatry,8 dialogic simulation offers a clear opportunity to operationalize the polyphonic dialogue that characterizes mental health. LLMs, particularly interfaces such as ChatGPT that function as dialogic AI, provide the opportunity to implement simulated patient interactions to promote both standardization of requirements and strengthening of patient care skills.
One of the specific competencies, IV.B.1.b).(1).(a).(ii), under Patient Care and Procedural Skills involves building an understanding of and developing clinical skills in the major treatment modalities used in child and adolescent psychiatry, including psychodynamic psychotherapy, cognitive-behavioral therapy, and crisis intervention. Here, we showcase how we used the voice feature of the ChatGPT app to practice crisis intervention (Figure 1). Using this feature, we were able to simulate a patient interaction in which the trainee had to respond live. Note that we give ChatGPT specific instructions, including defining the trainee’s role and objectives as well as defining ChatGPT’s role.
Discussion
In this simulation, ChatGPT gives a plausible patient scenario to facilitate the practice of crisis intervention techniques by providing basic instructions. These instructions are flexible; we can change aspects of the scenario, such as the circumstances under which the patient calls the psychiatrist (eg, suicidal ideation vs panic attack) or characteristics of the patient (eg, adolescent vs child). Furthermore, ChatGPT is able to take on a coaching role; it gives appropriate suggestions as to how a trainee might navigate the conversation in a way that mimics bedside precepting. Using ChatGPT to practice crisis intervention is also inherently less risky than in a real patient scenario and more cost-effective than hiring standardized patients.
With more specific prompt engineering, it would be possible to produce even more realistic scenarios and responses that are in line with a particular practice or style that instructors may want to emphasize. For example, a program director could give an LLM specific scenarios as guidelines and predetermined appropriate responses on which to base feedback. Thus, although LLMs are inherently stochastic, prompt engineering enables confines to be placed around the interaction, enabling trainees to practice with their clinical context GPT in a more standardized way than chance clinical encounters; programs can control the variability of the prompts more readily than the variability of patients presenting to a service. This structured approach allows trainees to practice adapting clinical reasoning to different patient contexts in a controlled manner, ensuring exposure to specific scenarios to support a more systematic approach to personalized care.
Crisis intervention is certainly not the only potential ACGME competency whose attainment can be augmented by LLMs. Any of the major treatment modalities could be practiced using the voice feature of ChatGPT. As has been shown comprehensively with knowledge-based undergraduate learning,10 LLMs can also be used to facilitate the acquisition of medical knowledge in psychiatry, per IV.B.1.c. For example, LLMs can be harnessed to build competency in understanding the basic neurobiological, psychological, and clinical sciences relevant to psychiatry, in accordance with IV.B.1.c).(1).(a), through facilitating dynamic interaction with research papers via OpenEvidence, Google Notebook, and SciSpace, among others.
There are potential downsides to incorporating artificial intelligence tools such as LLMs into attaining ACGME competencies. Some skills are best learned through interacting with other humans, for example conducting a physical or neurological examination, per IV.B.1.b).(1).(a).(iv), or a procedure, per IV.B.1.b).(2). Regarding simulated patient scenarios, although an LLM may be able to recapitulate some nuanced communication through prosody, voice apps cannot represent nonverbal communication. Additionally, because these models learn from corpora that embed historical social inequities, their outputs can potentially perpetuate or even amplify harmful biases, particularly toward marginalized groups.11,12 Careful prompt design, post hoc review, and ongoing evaluation of equity impacts are therefore essential when integrating LLMs into clinical training. More generally, it is important that the trainee not rely entirely on these tools for clinical knowledge and judgment. Caution must be exercised to ensure that these tools are used to complement rather than supplant clinical skills and judgment.
In addition to supporting trainees and programs in their attainment of ACGME competencies, interacting with LLMs is inherently important as further applications of LLMs inevitably arise in the clinical practice of child and adolescent psychiatry. It is essential that trainees build fluency in this technology to learn its strengths and weaknesses to promote responsible use. It behooves programs educating the next generation of psychiatrists to account for the fact that as of the past year, medical students will never have interacted with patients without having access to ChatGPT. Much as the dynamics of personal electronics changed the use of resources such as UpToDate from static CD libraries to a pocket search engine, ChatGPT offers a software-driven evolution in the information and learning tools available to the physician.
About the Authors
Anne Elizabeth Sidamon-Eristoff, AB, is a third-year MD–PhD student at Yale School of Medicine in the Interdepartmental Neuroscience Program. She actively researches both functional genomics in psychiatric disorders and the impact of LLMs on medical education. https://orcid.org/0000-0001-7422-9703
Conrad Safranek, BS, is a third-year medical student at Yale School of Medicine with interests in clinical informatics, medical education, and artificial intelligence in health care. His research focuses on leveraging AI, including large language models, to enhance clinical decision-making, promote health equity, and improve medical education. https://orcid.org/0000-0003-1985-9432
Andrés Martin, MD, PhD, is the Riva Ariella Ritvo Professor at the Child Study Center, Yale School of Medicine. He served as JAACAP Editor-in-Chief (2008–2017) and cofounded JAACAP Connect in 2014. As a faculty member of the Center for Medical Education and of the training programs at the YCSC, he is involved in undergraduate and graduate medical education. https://orcid.org/0000-0001-6425-5158
David Chartash, PhD, is a lecturer at the Department of Biomedical Informatics and Data Science, Yale School of Medicine. He has an active research portfolio in both clinical informatics and medical education and has written significantly on the impact of AI on undergraduate medical education. https://orcid.org/0000-0002-0265-330X
Funding
This research was funded by National Institutes of Health grant T32GM136651. http://dx.doi.org/10.13039/100000002
Disclosure
The authors have reported no biomedical financial interests or potential conflicts of interest.