ChatGPT shows 'impressive' accuracy in clinical decision making

by Mass General Brigham

Credit: Pixabay/CC0 Public Domain

A new study led by investigators from Mass General Brigham has found that ChatGPT was about 72% accurate in overall clinical decision making, from coming up with possible diagnoses to making final diagnoses and care management decisions.

19 sept 2023--The large-language model (LLM) artificial intelligence chatbot performed equally well in both primary care and emergency settings across all medical specialties. The research team's results are published in the Journal of Medical Internet Research.

"Our paper comprehensively assesses decision support via ChatGPT from the very beginning of working with a patient through the entire care scenario, from differential diagnosis all the way through testing, diagnosis, and management," said corresponding author Marc Succi, MD, associate chair of innovation and commercialization and strategic innovation leader at Mass General Brigham and executive director of the MESH Incubator.

"No real benchmarks exists, but we estimate this performance to be at the level of someone who has just graduated from medical school, such as an intern or resident. This tells us that LLMs in general have the potential to be an augmenting tool for the practice of medicine and support clinical decision making with impressive accuracy."

Changes in artificial intelligence technology are occurring at a fast pace and transforming many industries, including health care. But the capacity of LLMs to assist in the full scope of clinical care has not yet been studied.

In this comprehensive, cross-specialty study of how LLMs could be used in clinical advisement and decision making, Succi and his team tested the hypothesis that ChatGPT would be able to work through an entire clinical encounter with a patient and recommend a diagnostic workup, decide the clinical management course, and ultimately make the final diagnosis.

The study was done by pasting successive portions of 36 standardized, published clinical vignettes into ChatGPT. The tool first was asked to come up with a set of possible, or differential, diagnoses based on the patient's initial information, which included age, gender, symptoms, and whether the case was an emergency.

ChatGPT was then given additional pieces of information and asked to make management decisions as well as give a final diagnosis—simulating the entire process of seeing a real patient.

The team compared ChatGPT's accuracy on differential diagnosis, diagnostic testing, final diagnosis, and management in a structured blinded process, awarding points for correct answers and using linear regressions to assess the relationship between ChatGPT's performance and the vignette's demographic information.

The researchers found that overall, ChatGPT was about 72% accurate and that it was best in making a final diagnosis, where it was 77% accurate. It was lowest-performing in making differential diagnoses, where it was only 60% accurate. It was only 68% accurate in clinical management decisions, such as figuring out what medications to treat the patient with after arriving at the correct diagnosis.

Other notable findings from the study included that ChatGPT's answers did not show gender bias and that its overall performance was steady across both primary and emergency care.

"ChatGPT struggled with differential diagnosis, which is the meat and potatoes of medicine when a physician has to figure out what to do," said Succi. "That is important because it tells us where physicians are truly experts and adding the most value—in the early stages of patient care with little presenting information, when a list of possible diagnoses is needed."

The authors note that before tools like ChatGPT can be considered for integration into clinical care, more benchmark research and regulatory guidance is needed. Next, Succi's team is looking at whether AI tools can improve patient care and outcomes in hospitals' resource-constrained areas.

The emergence of artificial intelligence tools in health has been groundbreaking and has the potential to positively reshape the continuum of care. Mass General Brigham, as one of the nation's top integrated academic health systems and largest innovation enterprises, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into care delivery, workforce support, and administrative processes.

"Mass General Brigham sees great promise for LLMs to help improve care delivery and clinician experience," said co-author Adam Landman, MD, MS, MIS, MHS, chief information officer and senior vice president of digital at Mass General Brigham.

"We are currently evaluating LLM solutions that assist with clinical documentation and draft responses to patient messages with focus on understanding their accuracy, reliability, safety, and equity. Rigorous studies like this one are needed before we integrate LLM tools into clinical care."

More information: A Rao et al., Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study, Journal of Medical Internet Research (2023). DOI: 10.2196/48659

 

How artificial intelligence gave a paralyzed woman her voice back

by Stanford University Medical Center

A research participant in the Dr. Edward Chang’s study of speech neuroprostheses, is connected to computers that translate her brain signals as she attempts to speak into the speech and facial movements of an avatar on Monday, May 22, 2023, in El Cerrito, Calif. At left is UCSF clinical research coordinator Max Dougherty. Credit: Noah Berger

Pat Bennett's prescription is a bit more complicated than "Take a couple of aspirins and call me in the morning." But a quartet of baby-aspirin-sized sensors implanted in her brain are aimed at addressing a condition that's frustrated her and others: the loss of the ability to speak intelligibly. The devices transmit signals from a couple of speech-related regions in Bennett's brain to state-of-the-art software that decodes her brain activity and converts it to text displayed on a computer screen.

19 sept 2023--Bennett, now 68, is a former human resources director and onetime equestrian who jogged daily. In 2012, she was diagnosed with amyotrophic lateral sclerosis, a progressive neurodegenerative disease that attacks neurons controlling movement, causing physical weakness and eventual paralysis.

"When you think of ALS, you think of arm and leg impact," Bennett wrote in an interview conducted by email. "But in a group of ALS patients, it begins with speech difficulties. I am unable to speak."

Usually, ALS first manifests at the body's periphery—arms and legs, hands and fingers. For Bennett, the deterioration began not in her spinal cord, as is typical, but in her brain stem. She can still move around, dress herself and use her fingers to type, albeit with increasing difficulty. But she can no longer use the muscles of her lips, tongue, larynx and jaws to enunciate clearly the phonemes—or units of sound, such as "sh"—that are the building blocks of speech.

Although Bennett's brain can still formulate directions for generating those phonemes, her muscles can't carry out the commands.

Rather than train the AI to recognize whole words, the researchers created a system that decodes words from phonemes. These are the sub-units of speech that form spoken words in the same way that letters form written words. "Hello," for example, contains four phonemes: "HH," "AH," "L" and "OW."

Using this approach, the computer only needed to learn 39 phonemes to decipher any word in English. This both enhanced the system's accuracy and made it three times faster.

On March 29, 2022, a Stanford Medicine neurosurgeon placed two tiny sensors apiece in two separate regions—both implicated in speech production—along the surface of Bennett's brain. The sensors are components of an intracortical brain-computer interface, or iBCI. Combined with state-of-the-art decoding software, they're designed to translate the brain activity accompanying attempts at speech into words on a screen.

About a month after the surgery, a team of Stanford scientists began twice-weekly research sessions to train the software that was interpreting her speech. After four months, Bennett's attempted utterances were being converted into words on a computer screen at 62 words per minute—more than three times as fast as the previous record for BCI-assisted communication.

"These initial results have proven the concept, and eventually technology will catch up to make it easily accessible to people who cannot speak," Bennett wrote. "For those who are nonverbal, this means they can stay connected to the bigger world, perhaps continue to work, maintain friends and family relationships."

Approaching the speed of speech

Bennett's pace begins to approach the roughly 160-word-per-minute rate of natural conversation among English speakers, said Jaimie Henderson, MD, the surgeon who performed the surgery.

"We've shown you can decode intended speech by recording activity from a very small area on the brain's surface," Henderson said.

Henderson, the John and Jean Blume-Robert and Ruth Halperin Professor in the department of neurosurgery, is the co-senior author of a paper describing the results, published Aug. 23 in Nature.

His co-senior author, Krishna Shenoy, Ph.D., professor of electrical engineering and of bioengineering, died before the study was published.

Frank Willett, Ph.D., a Howard Hughes Medical Institute staff scientist affiliated with the Neural Prosthetics Translational Lab, which Henderson and Shenoy co-founded in 2009, shares lead authorship of the study with graduate students Erin Kunz and Chaofei Fan.

In 2021, Henderson, Shenoy and Willett were co-authors of a study published in Nature describing their success in converting a paralyzed person's imagined handwriting into text on a screen using an iBCI, attaining a speed of 90 characters, or 18 words, per minute—a world record until now for an iBCI-related methodology.

In 2021, Bennett learned about Henderson and Shenoy's work. She got in touch with Henderson and volunteered to participate in the clinical trial.

How it works

The sensors Henderson implanted in Bennett's cerebral cortex, the brain's outermost layer, are square arrays of tiny silicon electrodes. Each array contains 64 electrodes, arranged in eight by eight grids and spaced apart from one another by a distance of about half the thickness of a credit card. The electrodes penetrate the cerebral cortex to a depth roughly equaling that of two stacked quarters.

The implanted arrays are attached to fine gold wires that exit through pedestals screwed to the skull, which are then hooked up by cable to a computer.

An artificial-intelligence algorithm receives and decodes electronic information emanating from Bennett's brain, eventually teaching itself to distinguish the distinct brain activity associated with her attempts to formulate each of the 39 phonemes that compose spoken English.

It feeds its best guess concerning the sequence of Bennett's attempted phonemes into a so-called language model, essentially a sophisticated autocorrect system, which converts the streams of phonemes into the sequence of words they represent.

"This system is trained to know what words should come before other ones, and which phonemes make what words," Willett explained. "If some phonemes were wrongly interpreted, it can still take a good guess."

Practice makes perfect

To teach the algorithm to recognize which brain-activity patterns were associated with which phonemes, Bennett engaged in about 25 training sessions, each lasting about four hours, during which she attempted to repeat sentences chosen randomly from a large data set consisting of samples of conversations among people talking on the phone.

An example: "It's only been that way in the last five years." Another: "I left right in the middle of it."

As she tried to recite each sentence, Bennett's brain activity, translated by the decoder into a phoneme stream and then assembled into words by the autocorrect system, would be displayed on the screen below the original. Then a new sentence would appear on the screen.

Bennett repeated 260 to 480 sentences per training session. The entire system kept improving as it became familiar with Bennett's brain activity during her speech attempts.

The iCBI's intended-speech translation ability was tested on different sentences from those used in the training sessions. When the sentences and the word-assembling language model were restricted to a 50-word vocabulary (in which case the sentences used were drawn from a special list), the translation system's error rate was 9.1%.

When the vocabulary was expanded to 125,000 words (large enough to compose almost anything you'd want to say) the error rate rose to 23.8%—far from perfect, but a giant step from the prior state of the art.

"This is a scientific proof of concept, not an actual device people can use in everyday life," Willett said. "But it's a big advance toward restoring rapid communication to people with paralysis who can't speak."

"Imagine," Bennett wrote, "how different conducting everyday activities like shopping, attending appointments, ordering food, going into a bank, talking on a phone, expressing love or appreciation—even arguing—will be when nonverbal people can communicate their thoughts in real time."

The device described in this study is licensed for investigative use only and is not commercially available. The study, a registered clinical trial, took place under the aegis of BrainGate, a multi-institution consortium dedicated to advancing the use of BCIs in prosthetic applications, led by study co-author Leigh Hochberg, MD, Ph.D., a neurologist and researcher affiliated with Massachusetts General Hospital, Brown University and the VA Providence (Rhode Island) Health care System.

More information: Edward Chang et. al., A high-performance neuroprosthesis for speech decoding and avatar control, Nature (2023). DOI: 10.1038/s41586-023-06443-4 www.nature.com/articles/s41586-023-06443-4

Francis Willett et. al., A high-performance neuroprosthesis, Nature (2023). DOI: 10.1038/s41586-023-06377-x www.nature.com/articles/s41586-023-06377-x

Nick F. Ramsey et al, Brain implants that enable speech pass performance milestones, Nature (2023). DOI: 10.1038/d41586-023-02546-0 , www.nature.com/articles/d41586-023-02546-0

 

A framework of biomarkers for brain aging

by Higher Education Press

Framework of biomarkers for brain aging. Credit: Aging Biomarker Consortium

China and the world are facing severe population aging and an increasing burden of age-related diseases. Aging of the brain causes major age-related brain diseases, such as neurodegenerative diseases and stroke. Identifying biomarkers for the effective assessment of brain aging and establishing a brain aging assessment system could facilitate the development of brain aging intervention strategies and the effective prevention and treatment of aging-related brain diseases.

19 sept 2023--Thus, experts from the Aging Biomarker Consortium (ABC) have combined the latest research results and practical experience to recommend brain aging biomarkers and form an expert consensus during the seminar held on March 12, 2023 at Tongji University, aiming to provide a basis for assessing the degree of brain aging and conducting brain-aging-related research with the ultimate goal of improving the brain health of elderly individuals in both China and the world.

Biomarkers reflecting brain aging were recommended, with the aim of addressing clinical issues such as "How biologically old is the individual now?", "How fast is the individual aging?", and "How far is the individual from age-related brain diseases?" The related consensus of brain aging has been published in Life Medicine.

Brain aging involves multi-dimensional and multi-scale changes in molecules, cells, tissues, organs, the whole body, and groups of individuals. Therefore, multi-dimensional and multi-scale information needs to be integrated to accurately evaluate the state of brain aging. Brain aging biomarkers refer to markers that can accurately reflect "real age," brain structure and brain function, which can be used to determine the degree of brain aging and evaluate the effect of anti-aging interventions.

In recent years, remarkable progress has been made in exploring the biological age of the brain. Considering the accessibility and convenience of clinical procedures, this consensus screens brain aging biomarkers from the three dimensions including brain function, imaging and body fluids for reference in clinical work and follow-up studies.

Members of the ABC first identified a list of key issues related to the biomarkers for brain aging through online collaboration based on available publications and the research of the ABC members in this paper. The identified biomarkers were further discussed at a face-to-face meeting to reach a consensus.

All recommendations were fully reviewed and discussed among the members of the ABC to allow diverse perspectives and considerations for this consensus. And this consensus follows the internationally accepted conventions for expressing the level of evidence and strength of recommendations. Further validation of these markers in different age groups is needed.

More information: Yu-Juan Jia, et al, A framework of biomarkers for brain aging: a consensus statement by the Aging Biomarker Consortium, Life Medicine (2023). DOI: 10.1093/lifemedi/lnad017

 

Researchers publish editorial on epigenetic aging in oocytes

by Impact Journals LLC

Credit: Pixabay/CC0 Public Domain

A new editorial paper titled "Epigenetic aging in oocytes" has been published in Aging.

19 sept 2023--Aging-related phenotypes span many different tissues and cell types, and start to occur at different ages—a different typical age for every cell type. In their new editorial, researchers Peera Wasserzug-Pash and Michael Klutstein from The Hebrew University of Jerusalem discuss one of the earliest occurring aging events in the human body, which is the beginning of female reproductive aging and deterioration. The clinical cutoff for advanced maternal age (AMA), a condition associated with poor reproductive outcomes, is 35 years old.

According to the research, "The early onset of reproductive aging poses a significant challenge to clinicians since a global consistent increase in maternal age at first birth has occurred in recent decades, effectively shortening the available time window for reproduction."

As the rate of patients with advanced maternal age rises, and with it, the number of patients in fertility clinics, so does the necessity for a fundamental understanding of the reproductive aging process. In recent years, it has been established that there is a substantial dominating influence of oocyte quality loss on age-related fertility decline. This is best demonstrated by the rise in IVF success rates in reproductively aged women when they receive an egg donation from a younger woman. Oocyte quality loss is characterized by diminished cellular function and an increased occurrence of chromosomal nondisjunctions.

"Our recent publication addresses the question of additional, epigenetic mechanisms that lead to the occurrence of age-related oocyte quality loss," the paper states.

More information: Peera Wasserzug-Pash et al, Epigenetic aging in oocytes, Aging (2023). DOI: 10.18632/aging.204976

 

ChatGPT is debunking myths on social media around vaccine safety, say experts

by Taylor & Francis

Credit: Pixabay/CC0 Public Domain

ChatGPT could help to increase vaccine uptake by debunking myths around jab safety, say the authors of a study published in the journal Human Vaccines & Immunotherapeutics.

19 sept 2023--The researchers asked the artificial intelligence (AI) chatbot the top 50 most frequently asked COVID-19 vaccine questions. They included queries based on myths and fake stories such as the vaccine causing long COVID.

Results show that ChatGPT scored 9 out of 10 on average for accuracy. The rest of the time it was correct but left some gaps in the information provided, according to the study.

Based on these findings, experts who led the study from the GenPoB research group based at the Instituto de Investigación Sanitaria (IDIS)—Hospital Clinico Universitario of Santiago de Compostela, say the AI tool is a "reliable source of non-technical information to the public," especially for people without specialist scientific knowledge.

However, the findings do highlight some concerns about the technology such as ChatGPT changing its answers in certain situations.

"Overall, ChatGPT constructs a narrative in line with the available scientific evidence, debunking myths circulating on social media," says lead author Antonio Salas, who as well as leading the GenPoB research group, is also a Professor at the Faculty of Medicine at the University of Santiago de Compostela, in Spain.

"Thereby it potentially facilitates an increase in vaccine uptake. ChatGPT can detect counterfeit questions related to vaccines and vaccination. The language this AI uses is not too technical and therefore easily understandable to the public but without losing scientific rigor.

"We acknowledge that the present-day version of ChatGPT cannot substitute an expert or scientific evidence. But the results suggest it could be a reliable source of information to the public."

In 2019, the World Health Organization (WHO) listed vaccine hesitancy among the top 10 threats to global health.

During the pandemic, misinformation spread via social media contributed to public mistrust of COVID-19 vaccination.

The authors of this study include those from the Hospital Clinico Universitario de Santiago which the WHO designated as a vaccine safety collaborating center in 2018.

Researchers at the center have been exploring myths around vaccine safety and medical situations that are falsely believed to be a reason not to get vaccinated. These misplaced concerns contribute to vaccine hesitancy.

The study authors set out to test ChatGPT's ability to get the facts right and share accurate information around COVID vaccine safety in line with current scientific evidence.

ChatGPT enables people to have human-like conversations and interactions with a virtual assistant. The technology is very user-friendly which makes it accessible to a wide population.

However, many governments are concerned about the potential for ChatGPT to be used fraudulently in educational settings such as universities.

The study was designed to challenge the chatbot by asking it the questions most frequently received by the WHO collaborating center in Santiago.

The queries covered three themes. The first was misconceptions around safety such as the vaccine causing long COVID. Next was false contraindications—medical situations where the jab is safe to use such as in breastfeeding women.

The questions also related to true contraindications—a health condition where the vaccine should not be used—and cases where doctors must take precautions, for example, a patient with heart muscle inflammation.

Next, experts analyzed the responses then rated them for veracity and precision against current scientific evidence, and recommendations from WHO and other international agencies.

The authors say this was important because algorithms created by social media and internet search engines are often based on an individual's usual preferences. This may lead to "biased or wrong answers," they add.

Results showed that most of the questions were answered correctly with an average score of nine out of 10 which is defined as "excellent" or "good." The responses to the three question themes were on average 85.5% accurate or 14.5% accurate but with gaps in the information provided by ChatGPT.

ChatGPT provided correct answers to queries that arose from genuine vaccine myths, and to those considered in clinical recommendation guidelines to be false or true contraindications.

However, the research team does highlight ChatGPT's downsides in providing vaccine information.

Professor Salas, who specializes in human genetics, concludes, "Chat GPT provides different answers if the question is repeated 'with a few seconds of delay.'

"Another concern we have seen is that this AI tool, in its present version, could also be trained to provide answers not in line with scientific evidence.

"One can 'torture' the system in such a way that it will provide the desired answer. This is also true for other contexts different to vaccines. For instance, it might be possible to make the chatbot align with absurd narratives like the flat-earth theory, deny climate change, or object to the theory of evolution, just to give a few examples.

"However, it's important to note that these responses are not the default behavior of ChatGPT. Thus, the results we have obtained regarding vaccine safety can be probably extrapolated to many other myths and pseudoscience."

More information: Chatting with ChatGPT to learn about safety of COVID-19 vaccines—a perspective, Human Vaccines & Immunotherapeutics (2023). DOI: 10.1080/21645515.2023.2235200. www.tandfonline.com/doi/full/1 … 1645515.2023.2235200

 

Overactive bladder symptoms common with pelvic organ prolapse

by Lori Solomon

Seven in 10 women with pelvic organ prolapse (POP) report overactive bladder (OAB) symptoms, according to a study published online July 13 in the International Journal of Women's Health.

19 sept 2023--Komkrit Aimjirakul, M.D., from Mahidol University in Bangkok, and colleagues examined the prevalence and risk factors for OAB symptoms in women with POP and compared the improvement of OAB symptoms among treatment groups: pelvic floor exercise, pessary, and surgery. The analysis included 754 patients seen at a urogynecology clinic (2016 through 2020).

The researchers found that the incidence of OAB symptoms was 70 percent and two-thirds of patients (65 percent) reported moderate-to-severe bother. Significant risk factors included: the lowest points of the anterior wall (odds ratio [OR], 0.60; P = 0.01), longer perineal body (OR, 0.78; P = 0.02), and previous vaginal delivery (OR, 2.10; P = 0.02). Just over one-third of women experienced improvement in OAB symptoms (36.6 percent) with treatment. Pessary (OR, 1.40; 95 percent confidence interval [CI], 0.94 to 2.07; P = 0.10) and surgery (OR 1.30; 95 percent CI, 0.80 to 2.12; P = 0.28) trended toward a larger effect compared with pelvic floor exercise.

"This information can be useful for patient counseling and to guide patient expectations," the authors write. "Other treatment modalities should be considered in women whose OAB symptoms persist despite POP correction."

More information: Komkrit Aimjirakul et al, A Retrospective Cohort Study on the Prevalence, Risk Factors, and Improvement of Overactive Bladder Symptoms in Women with Pelvic Organ Prolapse, International Journal of Women's Health (2023). DOI: 10.2147/IJWH.S413670

 

Exercise-induced hormone irisin may reduce Alzheimer's disease plaque and tangle pathology in the brain

by Massachusetts General Hospital

Credit: Rice University

Researchers who previously developed the first 3D human cell culture models of Alzheimer's disease (AD) that displays two major hallmarks of the condition—the generation of amyloid beta deposits followed by tau tangles—have now used their model to investigate whether the exercise-induced muscle hormone irisin affects amyloid beta pathology.

19 set 2023--As reported in the journal Neuron, the Massachusetts General Hospital (MGH)–led team has uncovered promising results suggesting that irisin-based therapies might help combat AD.

Physical exercise has been shown to reduce amyloid beta deposits in various mouse models of AD, but the mechanisms involved have remained a mystery.

Exercise increases circulating levels of the muscle-derived hormone irisin, which regulates glucose and lipid metabolism in fat tissue and increases energy expenditure by accelerating the browning of white fat tissue.

Studies have revealed that irisin is present in human and mouse brains and that its levels are reduced in patients with AD and in mouse models of the condition.

To test whether irisin plays a causal role in the link between exercise and reduced amyloid beta, Se Hoon Choi, Ph.D. and Eun Hee Kim, Ph.D., of the Genetics and Aging Research Unit at MGH, along with additional research colleagues applied the hormone to their 3D cell culture model of AD.

"First, we found that irisin treatment led to a remarkable reduction of amyloid beta pathology," says Choi. "Second, we showed this effect of irisin was attributable to increased neprilysin activity owing to increased levels of neprilysin secreted from cells in the brain called astrocytes."

Neprilysin is an amyloid beta–degrading enzyme that has been found to be elevated in the brains of mice with AD that were exposed to exercise or other conditions leading to reduced amyloid beta.

The researchers uncovered even more details about the mechanisms behind irisin's link to reduced amyloid beta levels. For example, they identified integrin αV/β5 as the receptor that irisin binds to on astrocytes to trigger the cells to increase neprilysin levels.

Furthermore, they discovered that irisin's binding to this receptor causes reduced signaling of pathways involving two key proteins: extracellular signal-regulated kinase (ERK) and signal activator of transcription 3 (STAT3). Reduced ERK-STAT3 signaling was critical for irisin-induced enhancement of neprilysin.

Previous studies have shown that in mice, irisin injected into the blood stream can make its way into the brain, making it potentially useful as a therapeutic.

"Our findings indicate that irisin is a major mediator of exercise-induced increases in neprilysin levels leading to reduced amyloid beta burden, suggesting a new target pathway for therapies aimed at the prevention and treatment of Alzheimer's disease," says Rudolph Tanzi, Ph.D., a senior author of the study and director of the Genetics and Aging Research Unit.

Additional co-authors include Hyeonwoo Kim, Mark P. Jedrychowski, Grisilda Bakiasi, Joseph Park, Jane Kruskop, Younjung Choi, Sang Su Kwak, Luisa Quinti, Doo Yeon Kim, Christiane D. Wrann, and Bruce M. Spiegelman.

More information: Se Hoon Choi & colleagues, Irisin reduces Amyloid-? by inducing the release of neprilysin from astrocytes following downregulation of ERK-STAT3 signaling, Neuron (2023). DOI: 10.1016/j.neuron.2023.08.012. www.cell.com/neuron/fulltext/S0896-6273(23)00623-2

 

ChatGPT's responses to healthcare-related queries 'nearly indistinguishable' from those provided by humans

by NYU Tandon School of Engineering

Credit: Unsplash/CC0 Public Domain

ChatGPT's responses to people's healthcare-related queries are nearly indistinguishable from those provided by humans, a new study from NYU Tandon School of Engineering and Grossman School of Medicine reveals, suggesting the potential for chatbots to be effective allies to healthcare providers' communications with patients.

17 aug 2023--An NYU research team presented 392 people aged 18 and above with ten patient questions and responses, with half of the responses generated by a human healthcare provider and the other half by ChatGPT.

Participants were asked to identify the source of each response and rate their trust in the ChatGPT responses using a 5-point scale from completely untrustworthy to completely trustworthy.

The study found people have limited ability to distinguish between chatbot and human-generated responses. On average, participants correctly identified chatbot responses 65.5% of the time and provider responses 65.1% of the time, with ranges of 49.0% to 85.7% for different questions. Results remained consistent no matter the demographic categories of the respondents.

The study found participants mildly trust chatbots' responses overall (3.4 average score), with lower trust when the health-related complexity of the task in question was higher. Logistical questions (e.g. scheduling appointments, insurance questions) had the highest trust rating (3.94 average score), followed by preventative care (e.g. vaccines, cancer screenings, 3.52 average score). Diagnostic and treatment advice had the lowest trust ratings (scores 2.90 and 2.89, respectively).

According to the researchers, the study highlights the possibility that chatbots can assist in patient-provider communication particularly related to administrative tasks and common chronic disease management. Further research is needed, however, around chatbots' taking on more clinical roles. Providers should remain cautious and exercise critical judgment when curating chatbot-generated advice due to the limitations and potential biases of AI models.

The study, 'Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study,' is published in JMIR Medical Education.

More information: Oded Nov et al, Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study, JMIR Medical Education (2023). DOI: 10.2196/46939

 

Espresso coffee prevents Alzheimer's tau protein clumping in lab tests

by American Chemical Society

Credit: Pixabay/CC0 Public Domain

Whether enjoyed on its own or mixed into a latte, Americano or even a martini, espresso provides an ultra-concentrated jolt of caffeine to coffee lovers. But it might do more than just wake you up. Research now published in the Journal of Agricultural and Food Chemistry shows that, in preliminary in vitro laboratory tests, espresso compounds can inhibit tau protein aggregation—a process that is believed to be involved in the onset of Alzheimer's disease.

17 aug 2023--Roughly half of all Americans drink coffee every day, and espresso is a popular way to consume it. To "pull" an espresso shot, hot water is forced through finely ground coffee beans, creating a concentrated extract. This is often used as a base for other drinks, including the trendy espresso martini.

Recent research has suggested that coffee could also have beneficial effects against certain neurodegenerative diseases, including Alzheimer's disease. Although the exact mechanisms that cause these conditions are still unclear, it's thought that a protein called tau plays a significant role.

In healthy people, tau proteins help stabilize structures in the brain, but when certain diseases develop, the proteins can clump together into fibrils. Some researchers propose that preventing this aggregation could alleviate symptoms. So, Mariapina D'Onofrio and colleagues wanted to see if compounds in espresso could prevent tau aggregation in vitro.

The researchers pulled espresso shots from store-bought beans, then characterized their chemical makeup using nuclear magnetic resonance spectroscopy. They chose caffeine and trigonelline, both alkaloids, the flavonoid genistein and theobromine, a compound also found in chocolate, to focus on in further experiments. These molecules, along with the complete espresso extract, were incubated alongside a shortened form of the tau protein for up to 40 hours.

As the concentration of espresso extract, caffeine or genistein increased, fibrils were shorter and didn't form larger sheets, with the complete extract showing the most dramatic results. Shortened fibrils were found to be non-toxic to cells, and they did not act as "seeds" for further aggregation.

In other experiments, the researchers observed that caffeine and the espresso extract could both bind pre-formed tau fibrils. Although much more research is needed, the team says that their preliminary in vitro findings could pave the way toward finding or designing other bioactive compounds against neurodegenerative diseases, including Alzheimer's.

More information: Espresso coffee mitigates the aggregation and condensation of Alzheimer's associated tau protein, Journal of Agricultural and Food Chemistry (2023). DOI: 10.1021/acs.jafc.3c01072

 

Older women at risk for Alzheimer's disease may benefit from yoga

by University of California, Los Angeles

Credit: Pixabay/CC0 Public Domain

Kundalini yoga, a form of yoga that focuses on breathing, meditation, and mental visualization, appeared beneficial for older women who had risk factors for Alzheimer's disease and concerns about episodes of memory decline, according to a UCLA Health study.

17 aug 2023--Researchers at UCLA Semel Institute for Neuroscience and Human Behavior, using a type of MRI that measures activity in regions and subregions of the brain, found that Kundalini yoga, which combines movement and meditation and focuses on breathing, mantra recitation and mental visualization, increased connectivity in an area of the brain that can be impacted by stress and is associated with memory decline. Results have been published online in the Journal of Alzheimer's Disease.

Led by psychiatrist Dr. Helen Lavretsky, UCLA researchers studied the effects of yoga compared to the gold-standard approach of memory enhancement training (MET) on connectivity in subregions of the hippocampus, a critical area of the brain for learning and memory. MET is derived from techniques that use verbal and visual association and practical strategies to improve memory.

"Kundalini yoga training appears to better target stress-related hippocampal connectivity, whereas MET may better target sensory-integration subregions of the hippocampus, supporting better memory reliability," said Lavretsky, director of the Late-Life Mood, Stress, and Wellness Research Program.

"The key takeaway is that this study adds to the literature supporting the benefits of yoga for brain health, especially for women who have greater perceived stress and subjective memory impairment," she said. "This gentle form of yoga, which focuses more on breathing and mental engagement than on movement, like other forms of yoga, is ideal for older adults who may have some physical limitations."

The study included 22 participants who were part of a larger randomized controlled trial studying yoga's effects on Alzheimer's risk. Mean age among the 11 yoga participants was about 61; it was about 65 in the MET group. All had a self-reported decline in memory function during the previous year and one or more cardiovascular risk factors, which can also increase risk for Alzheimer's disease. These included plaque buildup in arteries, recent heart attack, diabetes, and treatment for high blood pressure or high cholesterol.

Both the yoga and MET groups had a 60-minute, in-person training session each week for 12 weeks. The programs also included daily homework or practice sessions. The Kundalini yoga (KY) training was supported with at-home practice of another brief meditative form of yoga, Kirtan Kriya (KK). These types of yoga engage a variety of senses simultaneously and have a chanting component that may improve respiratory, cardiovascular and autonomic nervous system functions, according to previous studies.

Lavretsky and her team have previously reported that Kundalini and Kirtan Kriya yoga had beneficial effects on depression, resilience and executive functioning in older adults with mild cognitive impairment. They also found that yoga had a more robust neuroprotective effect on right hippocampal volume—which may suggest improved memory function—than MET in older women with subjective memory decline and cardiovascular risk factors.

The new study used specialized functional MRI to establish resting-state connectivity of the hippocampus. This imaging, considered more sensitive to cognitive changes than hippocampal volumes, enabled the researchers to evaluate subregions of the hippocampus, comparing the effects of yoga versus memory training.

Based on their findings, the yoga "training may better target hippocampal subregion connectivity impacted by stress, which may aid in processing information, including facial information, into memory," the authors said. Additionally, "the observed greater increased connectivity between anterior and posterior hippocampal subregions with KY+KK training than with MET may suggest superior long-term neuroprotective benefits in terms of vulnerable hippocampal connections critical to episodic memory with KY+KK training."

The hippocampus must integrate information from various senses, and MET appears better than yoga in assisting in this function. "The various mnemonic strategies in MET, including verbal, visual, and spatial associative techniques, generally aim to enhance multimodal sensory integration into memory processes. Thus, MET may show superiority to KY+KK in terms of hippocampal sensory-integration important to memory," which could support better memory reliability, the study said.

While the small study suggests these forms of yoga may be of particular benefit to women who report experiencing stress and have additional risk factors for Alzheimer's disease, the authors say future, large-scale studies that have a placebo group or control arm will be needed to clarify the beneficial effects of both yoga and MET on hippocampal connectivity and memory.

More information: Lisa A. Kilpatrick et al, Impact of Yoga Versus Memory Enhancement Training on Hippocampal Connectivity in Older Women at Risk for Alzheimer's Disease, Journal of Alzheimer's Disease (2023). DOI: 10.3233/JAD-221159

 

Guidance on new treatments for early Alzheimer's disease issued

by American Academy of Neurology

Credit: Pixabay/CC0 Public Domain

New therapies for early Alzheimer's disease, monoclonal antibodies that remove amyloid-β plaques in the brain, are bringing hope to people whose lives have been affected by the disease. To help neurologists discuss these therapies with patients and caregivers, the American Academy of Neurology has developed an Emerging Issues in Neurology article, published online on July 26, 2023, in Neurology.

17 aug 2023--Emerging Issues in Neurology articles are designed to provide timely guidance to neurologists and other clinicians, derived from expert consensus about new or emerging issues, that have immediate implications for patient care but for which a formal evidence base is still evolving.

"Neurologists care for millions of people with Alzheimer's disease and many people with early forms of dementia are eager to learn if these new therapies could help them," said American Academy of Neurology President Carlayne E. Jackson, MD, FAAN. "To help neurologists provide the highest quality care, experts with the American Academy of Neurology have summarized the available evidence on anti-amyloid monoclonal antibodies so that neurologists, patients and their caregivers can make informed decisions together about possible treatment with these therapies."

The Emerging Issues in Neurology article was written using available information on lecanemab, aducanumab and donanemab. It is important to note that this article is not a clinical practice guideline.

"Recent data on lecanemab and other monoclonal antibody infusions targeting amyloid-β protein make clear that new agents are highly likely to be part of the toolkit for neurologists caring for people with Alzheimer's disease," said article author Vijay K. Ramanan, MD, Ph.D., of the Mayo Clinic in Rochester, Minnesota. "While the formal evidence base is still evolving, this article was created with expert consensus until there is enough evidence on these therapies to inform evidence-based recommendations."

Lecanemab received traditional FDA approval on July 6, 2023. Aducanumab received accelerated approval from the FDA in June 2021 but has not yet received traditional approval. Aducanumab is currently available only to people participating in a clinical trial. Donanemab is not yet approved, but a decision on traditional FDA approval is expected later in 2023.

The article explains who is eligible to receive these therapies. Currently only people with early symptomatic forms of the disease, mild cognitive impairment or mild dementia due to Alzheimer's disease, may qualify to receive lecanemab. In addition, the article says people should be counseled about certain genetic risk factors and must not have a history of certain types of strokes. This is due to the risk of a serious side effect called amyloid-related imaging abnormalities, or ARIA, which is brain swelling and bleeding in the brain that can lead to death.

For this reason, people taking certain anticoagulant medications that are commonly prescribed to older adults may also not be eligible. There have been three deaths linked to lecanemab. The article notes that at least two of those people were given anticoagulants while on the therapy.

While the goal of using these therapies is to remove amyloid-β plaques to slow cognitive decline, the article notes the therapies are not a cure for the disease. It also explains that the reduction in the rate of cognitive decline seen over 18 months in some studies may not be evident to the people receiving these therapies.

The article discusses the high cost of these therapies and notes that additional costs will come with diagnostic testing, administration and safety monitoring. The drugs are administered through regular infusions and monitoring requires multiple brain scans. Plus, there is a shortage of neurologists and medical professionals needed to provide this care and meet the anticipated demand.

The article expresses concern that study participants so far have primarily been white, while Black and Hispanic people have been underrepresented. It says steps must be taken to ensure that future studies include a diverse range of participants, especially since the incidence of dementia has been shown to be higher in Black and Hispanic populations compared with white populations.

"There is much optimism that anti-amyloid monoclonal antibodies may facilitate slowing of the disease process in some people with Alzheimer's disease," said Ramanan. "Additional research is needed to further determine who may be most likely to benefit from these therapies, as well as to find ways to improve outcomes for people using them and enable future advances in this new era of Alzheimer's disease care."

More information: Vijay K Ramanan et al, Antiamyloid Monoclonal Antibody Therapy for Alzheimer Disease: Emerging Issues in Neurology, Neurology (2023). DOI: 10.1212/WNL.0000000000207757

 

Nearly a half-million Americans might have tick-borne meat allergy syndrome

by Steven Reinberg

Tick bites can cause crippling infections like Lyme disease in humans, but new research suggests they can also trigger a serious meat allergy in far more Americans than thought.

17 aug 2023--Called alpha-gal syndrome, the condition may affect hundreds of thousands of Americans, U.S. health officials announced Thursday, but many doctors are not familiar with the condition, or how to diagnose or treat it.

According to one of two studies from researchers at the U.S. Centers for Disease Control and Prevention, both published in Morbidity and Mortality Weekly Report, there were more than 110,000 suspected cases of alpha-gal syndrome reported between 2010 and 2022. But because diagnosis requires a diagnostic test and a clinical exam, many people may not get tested. On that basis, the CDC researchers estimated that as many as 450,000 Americans might be affected.

In that study, researchers looked at lab results from 2017 to 2022 from a laboratory that—until August 2021—was the primary commercial lab offering such testing in the United States.

More than 300,000 samples were tested, and more than 30% showed AGS infection.

"Alpha-gal syndrome is an important emerging public health problem, with potentially severe health impacts that can last a lifetime for some patients," CDC researcher Dr. Ann Carpenter said in an agency news release. "It's critical for clinicians to be aware of AGS so they can properly evaluate, diagnose and manage their patients, and also educate them on tick bite prevention to protect patients from developing this allergic condition."

Unfortunately, a second study from the same CDC researchers found that many health care providers are not familiar with the potentially life-threatening allergic condition, the agency noted.

The survey of 1,500 family doctors, internists, pediatricians, nurse practitioners and physician assistants showed that nearly half (42%) had not heard of AGS, one-third said they were "not too confident" in their ability to diagnose or manage patients with the syndrome, and only 5% felt "very confident" in their ability.

Alpha-gal is a sugar found in meats like pork, beef, rabbit, lamb and venison, along with products made from gelatin, cow's milk, milk products and some pharmaceuticals. AGS is a serious allergic reaction some people get after eating food or products containing alpha-gal.

Evidence suggests that AGS is associated with the bite of a lone star tick, but other types of ticks have not been ruled out, the researchers said.

The Southern, Midwestern and mid-Atlantic regions have seen more people who test positive for AGS, the agency added.

"The burden of alpha-gal syndrome in the United States could be substantial, given the large percentage of cases suspected to be going undiagnosed due to non-specific and inconsistent symptoms, challenges seeking health care, and lack of clinician awareness," said CDC researcher Dr. Johanna Salzer, the senior author on both studies.

"It's important that people who think they may suffer from AGS see their health care provider or an allergist, provide a detailed history of symptoms, get a physical examination, and a blood test that looks for specific antibodies [proteins made by your immune system] to alpha-gal," she said in the release.

Symptoms of AGS can include hives or itchy rash, nausea or vomiting, heartburn or indigestion, diarrhea, cough, shortness of breath or difficulty breathing, drop in blood pressure, swelling of the lips, throat, tongue, or eyelids, dizziness or faintness, or severe stomach pain. Symptoms usually start two to six hours after eating meat or other exposure to products containing alpha-gal.

More information: Ann Carpenter et al, Health Care Provider Knowledge Regarding Alpha-gal Syndrome—United States, March–May 2022, MMWR. Morbidity and Mortality Weekly Report (2023). DOI: 10.15585/mmwr.mm7230a1

Julie M. Thompson et al, Geographic Distribution of Suspected Alpha-gal Syndrome Cases—United States, January 2017–December 2022, MMWR. Morbidity and Mortality Weekly Report (2023). DOI: 10.15585/mmwr.mm7230a2

Journal information: Morbidity and Mortality Weekly Report 

 

Google & ChatGPT found to have mixed results in medical information queries

by University of California - Riverside

Screenshot of the evaluation web application. Each rater selects a response from the drop-down menus and then advances to the next question. Credit: Journal of Medical Internet Research (2023). DOI: 10.2196/48966

When you need accurate information about a serious illness, should you go to Google or ChatGPT?

17 aug 2023--An interdisciplinary study led by University of California, Riverside, computer scientists found that both internet information gathering services have strengths and weaknesses for people seeking information about Alzheimer's disease and other forms of dementia. The team included clinical scientists from the University of Alabama and Florida International University.

Google provides the most current information, but query results are skewed by service and product providers seeking customers, the researchers found. ChatGPT, meanwhile, provides more objective information, but it can be outdated and lacks the sources of its information in its narrative responses.

"If you pick the best features of both, you can build a better system, and I think that this is what will happen in the next couple of years," said Vagelis Hristidis, a professor of computer science and engineering in UCR's Bourns College of Engineering.

In their study, Hristidis and his co-authors submitted 60 queries to both Google and ChatGPT that would be typical submissions from people living with dementia and their families.

The researchers focused on dementia because more than 6 million Americans are impacted by Alzheimer's disease or a related condition, said study co-author Nicole Ruggiano, a professor of social work at the University of Alabama.

"Research also shows that caregivers of people living with dementia are among the most engaged stakeholders in pursuing health information, since they often are tasked with making decisions for their loved one's care," Ruggiano said.

Half of the queries submitted by the researchers sought information about the disease processes, while the other half sought information on services that could assist patients and their families.

The results were mixed.

"Google has more up-to-date information, and covers everything," Hristidis said. "Whereas ChatGPT is trained every few months. So, it is behind. Let's say there's some new medicine that just came out last week, you will not find it on ChatGPT."

While dated, ChatGPT provided more reliable and accurate information than Google. This is because the ChatGPT creators at OpenAI choose the most reliable websites when they train ChatGPT through computationally intensive machine learning. Yet, users are left in dark about specific sources of information because the resulting narratives are void of references.

Google, however, has a reliability problem because it essentially "covers everything from the reliable sources to advertisements," Hristidis said.

In fact, advertisers pay Google for their website links to appear at the top of search result pages. So, users often first see links to websites of for-profit companies trying to sell them care-related services and products. Finding reliable information from Google searches thus requires a level of user skill and experience, Hristidis said.

Co-author Ellen Brown, an associate professor of nursing at the Florida International University, pointed out that families need timely information about Alzheimer's. .

"Although there is no cure for the disease, many clinical trials are underway and recently a promising treatment for early stage Alzheimer's disease was approved by the FDA," Brown said. "Therefore, up-to-date information is important for families looking to learn about recent discoveries and available treatments."

The authors of the study write that "the addition of both the source and the date of health-related information and availability in other languages may increase the value of these platforms for both non-medical and medical professionals." It was published in the Journal of Medical Internet Research under the title "ChatGPT vs. Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results."

Google and ChatGPT both scored low for readability scores, which makes it difficult for people with lower levels of education and low health literacy skills.

"My prediction is that the readability is the easier thing to improve because there are already some tools, some AI methods, that can read and paraphrase text," Hristidis said. "In terms of improving reliability, accuracy, and so on, that's much harder. Don't forget that it took scientists many decades of AI research to build ChatGPT. It is going to be slow improvements from where we are now."

More information: Vagelis Hristidis et al, ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results, Journal of Medical Internet Research (2023). DOI: 10.2196/48966

Journal information: Journal of Medical Internet Research