Virtual puppets developed by kinetic imaging professor help older adults feel more comfortable telling their stories

VoicingElder avatars mimic the motions of users. Credit: Virginia Commonwealth University

Semi Ryu had performed "Parting on Z"—her play about a farewell between lovers—a couple of times before that 2013 night in London. Ryu doesn't know what made this performance different, but something unexpected happened: She found herself sobbing in the middle of it.

09 mar 2017--"It was not acting," she said. "Something was touching me so deeply about it."
Ryu, an associate professor of kinetic imaging in the Virginia Commonwealth University School of the Arts, played both parts in the play about a man and woman ending their affair because of a difficult family disagreement. She physically played the female lover and spoke through an avatar, a virtual puppet, as the male lover.
"Sometimes I act as the female lover and then after that I activate a microphone on the avatar and then my voice becomes the male lover's voice," she said. "So we exchanged a dialogue of farewell and we drink together ... and then I took off my ring and placed it [in his hand to] keep it forever for your memory.
"It was extremely sad and I was really crying," she said. "I didn't expect to cry in front of this live audience and live performance setting and I [felt] so unprofessional. … I had to control myself but I couldn't control my emotions because there is something really overwhelming in this situation—[the] relationship between [the] virtual body and me."
She felt as if she was saying farewell to a part of herself, a symbolic farewell that made her think about how the self has different aspects.

VCUarts Associate Professor Semi Ryu, right, demonstrates how the on-screen avatar mimics the user’s motions. Credit: Virginia Commonwealth University

"There are multiple selves, but what is left out is maybe my social self," Ryu said. "Something defined by [society] as a role, as a woman … but I had to [say farewell to] my virtual body. … It was really about self-exploration and it made me cry."
She was embarrassed, but couldn't stop. The audience, equally moved, started crying as well. As the play ended, Ryu realized her primary emotion was no longer embarrassment, but rather relief. Speaking through the avatar had exposed suppressed feelings and given her emotional release.

'The puppet is speaking to me'

Having grown up in a Korean Confucian society, Ryu was discouraged from expressing emotion. Somehow this particular performance had revealed a lifetime of suppressed feelings. Ryu attributes those feelings of repression to han, the Korean cultural concept that encompasses feelings of oppression and grief with a strong hope and desire to overcome the impossible. Using the virtual puppet had allowed a different side of Ryu to emerge in the storytelling.
"I felt like something unresolved was resolved and then I felt this could also benefit … somebody else who has some kind of communication dilemma in society," Ryu said, realizing that han can be universal.
She immediately thought of the older adult population. As people get older, their image of themselves differs from how other people see them.
"They have ageless bodies happening in their mind but biologically it's degeneration and downgrading," she said. "There is an interesting tension happening, a kind of dilemma between what they want to be [and how they have been portrayed in the] media as powerless, hopeless and helpless individuals. Those images never fit [how] older adults perceive themselves. In this sense, they have han, and maybe we can … process a kind of ritual with older adults so they can start to release what they want to express."
Ryu was inspired by her brother, an occupational therapist who volunteered at area nursing homes and regularly talked about his amazing experiences working with older adults. She applied for the VCU Presidential Research Quest Fund in 2015 and started collaborating with Tracey Gendron, Ph.D., an associate professor in the VCU Department of Gerontology who also serves as the department's director of community engagement and research. Yet Ryu's motivation was always as an artist.
After years of using technology to explore the relationship between herself as the user and the virtual puppet, she became interested in what happens psychologically to users when they speak through the avatar.
"I was always curious about users' psyche…," Ryu said. "What kind of psychological relationship is possible between the user and avatar? That has been my primary research focus for a long time—more than 10 years. It seems to me it's such a different experience when I talk with or without 'puppet me.' [With the puppet], the puppet is speaking to me and so a different part of myself is emerging during the process of storytelling."

Developing VoicingElder

For years, Ryu has collaborated with Stefano Faralli of University of Manheim, Germany, to develop the interactive technology for her virtual puppetry. Together, they developed VoicingElder, an avatar system with four different development stages: childhood, teenage years, young adulthood and older adulthood, with four female and four male avatars. They implemented lip-sync technology so the avatars can speak in real time, giving the illusion that the puppet is the speaker. Users are then in the position of listening to their own stories even as they are telling them.

The VoicingElder program uses avatars from four different development stages: childhood, teenage, young adulthood and older adulthood. Credit: Virginia Commonwealth University

"It's an interesting psychology, looking at the puppet and the puppet is actually telling the story to them. They're going to get some distance from their story so they can easily talk about an emotional story … with more safe distance," Ryu said.
They also created a sentiment analysis algorithm, which detects emotional content from live speech and provides a virtual background to match. Happy stories have bright sunlight, while thunderstorms accompany angry stories. Other emotions detected are sadness, anxiety, fear and laugher.
"The VoicingElder project uses an artistic framework to combine technology and storytelling in an innovative way that actively engages older adult users in a virtual platform that provides the opportunity to share their life stories and experiences," said Gendron.
In March 2016, Ryu was ready to put the system to use. She worked with 10 residents of the Gayton Terrace Senior Living Center for three months, visiting every week or two. At the beginning, the seniors brought scripts with prepared stories, because they were self-conscious about speaking in front of each other. But around the fifth week, they began improvising and volunteering stories.
"It's wonderful because mainly your face is not there," said Connie Cole, 86. "It's a caricature there, so that takes the personal part away from it. … I think the fact that you're using the caricature and our voices and our stories blend beautifully for someone who's having dementia or some other elderly problem. … The fact the avatar's disconnected from me, it was easier for me to talk."
Vikki Fleming, resident program director at the center, said the project is a tool for residents to connect emotionally with important stories that are meaningful to them.

Connie Cole gestures as she tells a story through the VoicingElder system. Credit: Virginia Commonwealth University

"It's an interesting concept and it's fun to see residents experimenting with technology and getting to try new things because they want to continue to learn new things and this is one way of doing that," she said.
Ryu recorded the avatars at each session and at the end of the three months screened the film for residents, who all asked for a copy to share with their families.
"This can contribute to intergenerational relationships, so some story of the grandparents can be delivered to the grandsons … and it can work as a family legacy," she said.
Ryu is applying for a National Endowment for the Arts grant this spring and is pursuing National Institutes of Health grants as well. She has already presented the VoicingElder project at national and international conferences, including the National Center for Creative Aging in Washington, D.C., and Arts in Society as well as gerontology and drama therapy conferences and an upcoming Human-Computer Interaction International Conference. Many attendees have introduced the prototype system—which is very low-tech, requiring a simple setup: a computer with a decent processor, a connected camera and a wireless microphone—into their programs.
Ryu foresees extending the program to other populations and has begun collaboration with Tony Gentry, Ph.D., in the VCU Department of Occupational Therapy to apply the program to people with memory loss and PTSD.
"It's not just for older people but for [those who are] disabled, autistic [or have] communications difficulties," she said. "It can also be used to address diversity in society to increase awareness of racial, cultural and gender issues. It can help us be more empathetic to each other."


Provided by Virginia Commonwealth University

Occupational therapy reduces hospital readmissions

An independent study published in Medical Care Research and Review found that "occupational therapy is the only spending category where additional hospital spending has a statistically significant association with lower readmission rates" for the three health conditions studied: heart failure, pneumonia, and acute myocardial infarction.

16 sept 2016--"The findings of this important study highlight just one of the many roles occupational therapy practitioners are playing in improving quality and reducing healthcare costs," said Frederick P. Somers, Chief Executive Officer of the American Occupational Therapy Association. "Occupational therapy practitioners are proving to be an essential member of any interprofessional team successfully addressing the changing demands of the health care delivery system."
Researchers Andrew Rogers (Johns Hopkins University), Ge Bai (Johns Hopkins), Robert A. Levin (University of Maryland School of Medicine), and Gerard F. Anderson (Johns Hopkins) used Medicare claims and cost data to examine the association between hospital spending for specific services and 30-day admission rates for heart failure, pneumonia, and acute myocardial infarction. They evaluated 19 distinct spending categories (including occupational therapy) in 2,791 hospitals for the heart failure analysis; 2,818 hospitals for the pneumonia analysis; and 1,595 hospitals for the acute myocardial infarction analysis.
The researchers point out that occupational therapy "focuses on a vital issue related to readmission rates—can the patient be discharged safely into her or his environment?" If not, occupational therapistsaddress issues from physical barriers to daily function to support networks. They cite six particular interventions provided by occupational therapists that could lower readmissions:

1. Provide recommendations and training for caregivers.
2. Determine whether patients can safely live independently, or require further rehabilitation or nursing care.
3. Address existing disabilities with assistive devices so patients can safely perform activities of daily living (e.g., using the bathroom, bathing, getting dressed, making a meal)
4. Perform home safety assessments before discharge to suggest modifications.
5. Assess cognition and the ability to physically manipulate things like medication containers, and provide training when necessary.
6. Work with physical therapists to increase the intensity of inpatient rehabilitation.

The researchers identify occupational therapy as "one spending category that affects both the clinical and social determinants of health" and note that "investing in occupational therapy has the potential to improve care quality without significantly increasing overall hospital spending."
The authors point out that "occupational therapy places a unique and immediate focus on patients' functional and social needs, which can be important drivers of readmission if left unaddressed."
More than 213,000 occupational therapy practitioners across the U.S. help people across the lifespan to be as independent as possible, despite illness, injury, or mental disability. Occupational therapy is the only profession that addresses the things patients want and need to do through the therapeutic use of everyday activities (occupations).

More information: A. T. Rogers et al. Higher Hospital Spending on Occupational Therapy Is Associated With Lower Readmission Rates, Medical Care Research and Review (2016). DOI: 10.1177/1077558716666981


Provided by American Occupational Therapy Association

Occupational therapy gets people with osteoarthritis moving

U-M pilot study finds occupational therapy approach combined with exercise increases physical activity for most patients with hip and knee osteoarthritis

Ann Arbor, 02 oct 2008 – Physical activity is the cornerstone of any healthy lifestyle – and especially for people with osteoarthritis as exercise helps maintain good joint health, manage their symptoms, and prevent functional decline. Osteoarthritis, however, often makes physical activity, such as exercise, and even performing daily activities, a challenge.
But an occupational therapist-led approach – called activity strategy training – could provide patients with knee and hip osteoarthritis the opportunity to lead more active lives and even improve their overall health, according to a new study led by researchers at the University of Michigan Health System.
In the pilot study, the researchers found that patients who engaged in activity strategy training along with regular exercise increased their physical activity, more so than those patients who only took part in exercise and health education sessions. Study results are now online and are set to appear in the October issue of Arthritis & Rheumatism.
"Occupational therapy is really the missing link in promoting wellness of people with hip and knee osteoarthritis," says study lead author Susan L. Murphy, Sc.D., OTR, assistant professor in the Department of Physical Medicine and Rehabilitation at the U-M Medical School and Research Health Science Specialist at the VA Ann Arbor Healthcare System.
"Most people with osteoarthritis want to be active, but often find that there are personal or physical barriers in the way. For instance, people with osteoarthritis often deal with pain and fatigue, which makes engaging in regular physical activity more difficult. In addition, there are often barriers in people's homes and communities that make physical activity difficult," she continues.
For that reason, Murphy says the pilot study used activity strategy training in groups and in places – such as senior housing facilities – where barriers could be addressed and potential solutions discovered.
Taught by occupational therapists, this structured rehabilitation program is designed to educate patients about joint protection, proper body mechanics, activity pacing, and environmental barriers. For example, patients with joint pain caused by osteoarthritis learn techniques for walking around the house or outdoors, or even getting in and out of a car. For the pilot study, the activity strategy training included education, group discussion, a home visit, and demonstration and practice of techniques to facilitate activity.
Activity strategy training, however, is not commonly prescribed to patients with hip or knee osteoarthritis, a degenerative disease that causes the breakdown of the cartilage in joints. Most physical activity programs for these patients only offer structured exercise, which has been shown to have short-term positive effects on arthritis pain and physical disability. But these effects usually fade soon after participation in the program ends.
In this study, both groups participated in the same structured exercise program. However, only participants who received the activity strategy training were found to have increased the intensity of their physical activity at the end of the study compared to those who received health education.
While the results are promising, Murphy says more research needs to be conducted to replicate with larger groups and to examine long-term effects before this technique can be applied to patient care.
Regardless, Murphy encourages patients with hip or knee osteoarthritis to seek out opportunities now to enhance and expand their daily physical activity, and improve overall health behaviors.
"People with osteoarthritis tend to know more about surgical options, and less about how they can take an active role in promoting their own health and well-being," Murphy explains. "People with osteoarthritis need to be their own agents of change. They can do so much to manage symptoms and stave off functional decline caused by osteoarthritis just by being physically active. The bottom line is to find ways to help people create and maintain these healthy habits."