detecting linguistic characteristics of alzheimer's...
TRANSCRIPT
![Page 1: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/1.jpg)
Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models
Sweta KarlekarComputer Science, UNC Chapel Hill
Graduate Student: Tong NiuResearch Mentor: Dr. Mohit Bansal
![Page 2: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/2.jpg)
Alzheimer’s Disease (AD)
• Most common form of Dementia
• Caused by cortical degeneration
• Decline in language comprehension and ability
• Medication can slow or halt progression
Image courtesy of National Institute on Aging, National Institutes of Health, U.S. Department of Health and Human Services
![Page 3: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/3.jpg)
Evaluation TechniquesMental Status and Mood Testing
Physical and Neurological Exams
Extensive Medical History
Brain Imaging
![Page 4: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/4.jpg)
The Task
Transcripts of Spoken Languages
samples
Binary Classification of
AD+ or AD-
![Page 5: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/5.jpg)
But first, let’s look at the methodology.
![Page 6: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/6.jpg)
ML vs. DL
Image from: https://codeutsava.in/blog/40
![Page 7: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/7.jpg)
Natural Language Processing
Artificial Intelligence LinguisticsNLP
GOAL: Have computers understand natural language to perform useful tasks.
![Page 8: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/8.jpg)
NLP + Deep Learning
Image from: https://www.xenonstack.com/blog/data-science/overview-of-artificial-intelligence-and-role-of-natural-language-processing-in-big-data
![Page 9: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/9.jpg)
Back to the task…
![Page 10: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/10.jpg)
Dataset
• Dementia Bank dataset
• Transcripts and speech samples
• Non-AD + AD Patients
• Includes POS tags
• Noun, verb, adjective, adverb, present participle, determiner, etc.
Image from: https://www.researchgate.net/figure/The-Cookie-theft-picture_fig1_317095410
![Page 11: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/11.jpg)
Previous WorksAuthor ML vs. DL Description Accuracy
Rudzicz et al. Machine Learning Extracted over 200+ lexical features 67.0%
Orimaye et al. Machine LearningUsed syntactic,
lexical, and n-gram features
86.1%
Konig et al. Machine Learning Analyzed speech audio 87.0%
Orimaye et al. Deep Learning Deep Neural + Language Model 87.5%
![Page 12: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/12.jpg)
Neural Models
Recurrent Neural Network (RNN)
Convolutional Neural Network (CNN)
Convolutional/Recurrent Neural Network (CNN-RNN)
Images from: http://colah.github.io
![Page 13: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/13.jpg)
ResultsAuthor Model Description Accuracy
Rudzicz et al. Machine Learning 200+ lexical features 67.0%
Orimaye et al. Machine Learning Syntactic, lexical, and n-gram features 86.1%
Konig et al. Machine Learning Speech audio 87.0%
Orimaye et al. Deep Language Model Transcripts 87.5%
- CNN Transcripts 82.8%
- RNN Transcripts 83.7%
- CNN-RNN Transcripts 84.9%
- CNN-RNN Transcripts + POS 91.1%
![Page 14: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/14.jpg)
But what did the neural model look at?
![Page 15: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/15.jpg)
Saliency Heat MapsIn
put T
ext
![Page 16: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/16.jpg)
Saliency Heat MapsIn
put T
ext
![Page 17: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/17.jpg)
Saliency Heat MapsIn
put T
ext
![Page 18: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/18.jpg)
Activation Clustering
Image from: https://towardsdatascience.com/applied-deep-learning-part-1-artificial-neural-networks-d7834f67a4f6
![Page 19: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/19.jpg)
Activation Clustering• Short answers and bursts of speech
• “Okay”, “yes”, “oh!”, “yes”, “fine”
• Repeated requests for clarification
• “Did I say facts?”, “Did I get any?”, “Did I say elephant?”
• Starting with interjections
• “Well I gotta see it”, “Oh I just a lot of uh…”, “So all the words that you can”
![Page 20: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/20.jpg)
Activation Clustering
![Page 21: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/21.jpg)
Conclusion
• Applied 3 different neural models to AD classification
• Achieved a new benchmark accuracy
• Utilized two visualization techniques
![Page 22: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/22.jpg)
Future Work• Multi-class classification to differentiate among stages
• Apply to other neurological diseases:
• Huntington’s
• Diffuse Lewy Body
• How early can we catch AD in language?
• Agatha Christie and Iris Murdoch novels
![Page 23: Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -](https://reader034.vdocuments.net/reader034/viewer/2022051916/6007d80aefdf2c43a0397381/html5/thumbnails/23.jpg)
Questions?