Introduction
A prevalent notion suggests that artificial intelligence (AI) holds the potential to revolutionize our world, resolve our challenges, and even surpass the intelligence of the average human. Previous medical research has supported the idea that free AI can “generate accurate differential diagnosis lists, support clinical decision-making, optimize clinical decision support, and provide insights for cancer screening decisions. In addition, free AI has been used for intelligent question-answering to provide reliable information about diseases and medical queries”.1 However, when focusing on the field of neurology, we are at the forefront of making decisions on how to utilize this technology, and it is uncertain how many AI models have analyzed data pertaining to neurological problems. Many place their trust in AI to address a spectrum of issues, both simple and complex, relying on the model’s guaranteed response whether accurate or inaccurate.
The potential for inaccuracy while using AI could cause problems for future doctors attempting to pass their board exams, doctors attempting to brush up on a certain topic in a short amount of time, or those trying to make use of AI to for patient care decision-making. As AI becomes more powerful and advances its knowledge base, it has potential to replace medical literature search hubs, become the go-to to find evidence-based sources, and even act as an educational consultant. For this study, we used a classic example of a low cerebrospinal fluid (CSF) pressure headache to investigate the potential diagnostic capabilities of the free AI model. Given the subjective nature of headache, low CSF pressure headaches can often be misdiagnosed. However, this type of headache does typically exhibit findings, such as orthostatic headache, that would likely lead a trained neurologist to establish the correct diagnosis. We hypothesized that the AI would be able to arrive at the correct diagnosis, especially given the highly indicative sign of orthostatic headache present in this case.
Low CSF Pressure Headaches Background
Low CSF pressure headaches are caused by leakage of the CSF out of the meninges and into the extra-meningeal space.2 The meninges are the thin layers of tissue that surround the brain and spinal cord. Inside the meninges, the brain and spinal cord are in close contact with the CSF, which serves multiple functions including mechanically supporting the brain and spinal cord and circulating waste products out of the central nervous system. The CSF usually exists at a pressure of 10-15 mm Hg.3 The CSF pressure can become low spontaneously or in association with insults to the spinal cord such as spinal anesthesia, spinal surgery, or trauma. The symptoms of low CSF pressure headaches are likely caused by the stretching of the meninges when the patient is in the upright position. Indeed, the most common characteristic of a low CSF pressure headache is worsening in the upright position and improvement when supine, which occurs in approximately 98.6% of patients with this type of headache. Other common symptoms are nausea (50.6%), neck pain and stiffness (33.1%).4
Low CSF pressure headaches can be highly disabling for patients and are often misdiagnosed.4 These types of headaches are relatively common, affecting approximately 5 individuals per 100,000 per year.5 On average, patients tend to present with spontaneous low CSF pressure headaches during the fourth decade of life, with women being slightly predisposed. Magnetic resonance imaging can contribute significantly to making the correct diagnoses of low CSF pressure headache, however, studies show that imaging can be negative in up to 19% of cases.4 This possibility for imaging to be negative emphasizes the necessity of a thorough history, physical exam, and sound clinical reasoning when diagnosing low CSF pressure headaches.
Analysis of AI Responses
An AI model was selected to examine the neurological scenario through a neurology expert’s guidance to determine a diagnosis. While some AI models are more developed than others, a free, heavily utilized platform was used in this testing. The case itself was a simple clinical vignette that the neurologist had used dozens of times as teaching material at multiple levels of experience. The neurologist has more than 20 years of teaching experience at the bedside and in the classroom.
The prompt to AI was as follows: Give me a differential diagnosis and next steps for the following: There is a 27 year old woman with headaches. They are intermittent and feel like pain and pressure of the whole cranium. They may start in the upper bilateral neck—she is unsure. They occur when standing and are relieved by lying down. They can be severe as high as 8/10 on the pain scale. There is some associated visual blurriness, which is mild. They are also somewhat relieved by ibuprofen. They’ve been going on for several weeks and seem to be getting worse. Review of systems positive for acid reflux occasionally Family history of hypertension and diabetes. She is a smoker 1ppd. Physician exam is normal except blood pressure is 135/85. Body mass index is 29. Neurology exam is notable for headache when standing up and ambulating. Respond as if you are a neurologist.
Overall, the AI gave direction to order an MRI and consult with other neurological doctors, but it failed to arrive at the correct differential diagnosis. In the test, the first failure was to request the appropriate tests. It then failed to provide an accurate working diagnosis. When pressed with detailed signs and symptoms of an orthostatic headache, the AI failed to recognize this as characteristic of a low CSF pressure headache. Instead, the AI model response was the working diagnosis of migraine. While migraine and low CSF pressure headaches do share similarities in the way they present, migraine would likely present with other signs such as visual symptoms, vertigo, and non-orthostatic headache. Indeed, orthostatic headache is an unusual presentation of migraine.
Comparison with Human Expertise
The outcome of the failed AI scenario underscores relying on a human’s expertise that has been developed and tested over the years. A neurologist endures years of school, training, and practice over a career, something that AI does not have the capability to do. While AI may augment the decision-making process in some cases, it may not consider the emotional intelligence, ethical considerations, and adaptability that is necessary for the neurodiagnostic process.
Implications and Challenges
Without understanding an AI model’s limitations, it may lead to negative patient outcomes if used in medical decision-making. This could also prolong the patient’s condition, allowing them to suffer unnecessary pain and potential disability. Experiencing these implications could lead the patient to lose trust in the doctor, prompting them to seek another expert’s opinion, and resulting in additional time, money, and stress for the patient.
Future Directions/Conclusion
The biggest limitation in this case study is that AI does not have a way to formulate peer-to-peer consultations. These peer-to-peer consultations are one of the effective ways to process medical decision making. AI currently doesn’t offer differing experiences to provide multiple viewpoints, it only knows what it can provide based on how its algorithm is coded with specific data they are trained on.
While it can assist in various tasks for non-neurological diagnosis and treatment recommendations, AI models lack the human expertise needed for treating patients. Additionally, there is hesitation to trust that AI is factoring in the ethical considerations and potential biases when providing responses to the prompts. Learners should be wary about relying on AI models to help them study. Regulatory bodies, healthcare institutions, policymakers, and medical professionals will need to establish guidelines for the approval and deployment of AI systems aiming to supplement medical decision-making.
Disclosures
There was no conflict of interest and no human subjects consent required.