TY - GEN
T1 - An Intelligent Augmented Lifelike Avatar App for Virtual Physical Examination of Suspected Strokes
AU - Yao, Kevin
AU - Wong, Kelvin K.
AU - Yu, Xiaohui
AU - Volpi, John
AU - Wong, Stephen T.C.
N1 - Funding Information:
We appreciate Dr. Rebecca Danforth for proofreading the manuscript.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - An intelligent-augmented lifelike avatar mobile app (iLAMA) that integrates computer vision and sensor readings to automate and streamline the NIH Stroke Scale (NIHSS) physical examination is presented. The user interface design is optimized for elderly patients while the app showcases an animated lifelike 3D model of a friendly physician who walks the user through the exam. The standardized NIHSS examination included in iLAMA consists of five core tasks. The first two tasks involve rolling the eyes to the left and then to the right, and then smiling as wide as the user can. The app determines facial landmarks and analyzes the palsy of the face. The next task is to extend the arm and hold the phone at the shoulder level, and the smart phone gyroscope is used to detect acceleration to determine possible weakness in the arm. Next, the app tracks the location of the hand keypoints and determines possible ataxia based on the precision and accuracy of the locations of the touches. Finally, the app determines the user's forward acceleration in walking and possible imbalances using the accelerometer. The app then sends analyzed results of these tasks to the neurologist or stroke specialist for review and decisions.Clinical Relevance - The physical examination of a stroke patient is a time consuming and repetitive process, and there is a lack of infrastructure and resource to monitor patient in post-stroke recovery after they leave the hospital for home or rehabilitation facilities. iLAMA app aims to automate a subset of the NIHSS physical examinations in measuring motor function recovery and also allows individual patients to track their performance over time. It will be an essential component in monitoring rehabilitation recovery and therapy effectiveness after hospitalization and can easily scaled to lo help millions of patients at a fraction of the cost.
AB - An intelligent-augmented lifelike avatar mobile app (iLAMA) that integrates computer vision and sensor readings to automate and streamline the NIH Stroke Scale (NIHSS) physical examination is presented. The user interface design is optimized for elderly patients while the app showcases an animated lifelike 3D model of a friendly physician who walks the user through the exam. The standardized NIHSS examination included in iLAMA consists of five core tasks. The first two tasks involve rolling the eyes to the left and then to the right, and then smiling as wide as the user can. The app determines facial landmarks and analyzes the palsy of the face. The next task is to extend the arm and hold the phone at the shoulder level, and the smart phone gyroscope is used to detect acceleration to determine possible weakness in the arm. Next, the app tracks the location of the hand keypoints and determines possible ataxia based on the precision and accuracy of the locations of the touches. Finally, the app determines the user's forward acceleration in walking and possible imbalances using the accelerometer. The app then sends analyzed results of these tasks to the neurologist or stroke specialist for review and decisions.Clinical Relevance - The physical examination of a stroke patient is a time consuming and repetitive process, and there is a lack of infrastructure and resource to monitor patient in post-stroke recovery after they leave the hospital for home or rehabilitation facilities. iLAMA app aims to automate a subset of the NIHSS physical examinations in measuring motor function recovery and also allows individual patients to track their performance over time. It will be an essential component in monitoring rehabilitation recovery and therapy effectiveness after hospitalization and can easily scaled to lo help millions of patients at a fraction of the cost.
UR - http://www.scopus.com/inward/record.url?scp=85122511370&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85122511370&partnerID=8YFLogxK
U2 - 10.1109/EMBC46164.2021.9629720
DO - 10.1109/EMBC46164.2021.9629720
M3 - Conference contribution
C2 - 34891620
AN - SCOPUS:85122511370
T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
SP - 1727
EP - 1730
BT - 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2021
Y2 - 1 November 2021 through 5 November 2021
ER -