![Page 1: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/1.jpg)
Human Auditory Cognition
Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,
Mehmet Ozdas, Lakshmi Krishnan, Michael Crosse, Nima Mesgarani, Jose L Pepe Contreras-Vidal, Thusitha Chandrapala
Organizers: Shihab Shamma (Univ. of Maryland), Malcolm Slaney (Microsoft),Barbara Shinn-Cunningham (Boston University),
Edmund Lalor (Trinity College, Dublin)
![Page 2: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/2.jpg)
Imagined Sounds
• We hear sounds
• Cocktail Party Effect
• We imagine and expect sounds
![Page 3: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/3.jpg)
EEG Research
• Others• Muscles…• Simple stimuli (beeps, etc)• Simple responses (ERPs)
• Us• Natural speech• Not imagined movement• Imagined speech!!!
![Page 4: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/4.jpg)
One basket, one big egg
![Page 5: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/5.jpg)
Decoding
• Record EEG signals
• Three Approaches:• Mick/Nima
• Covariance of channels/SVM• Malcolm
• DCT of EEG/Nearest Neighbor• James
• Decode audio envelope/Correlate
![Page 6: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/6.jpg)
Scientific Results
Perc
enta
ge C
orre
ct
![Page 7: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/7.jpg)
Demo
• Ask question• Imagine answer• Decode/Classify• Display result
![Page 8: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/8.jpg)
What would the Queen not like to see?
![Page 9: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/9.jpg)
Quiet please
• Applause
![Page 10: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/10.jpg)
What's up in the sky?
![Page 11: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/11.jpg)
Quiet please
• Applause
![Page 12: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/12.jpg)
Andromeda galaxy is formed by billions of ...
![Page 13: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/13.jpg)
Quiet please
• Applause
![Page 14: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/14.jpg)
Over the Thames river you can find...
![Page 15: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/15.jpg)
Quiet please
• Applause
![Page 16: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/16.jpg)
What is the Capital of England?
![Page 17: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/17.jpg)
Quiet please
• Applause
![Page 18: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/18.jpg)
Why should you use sunscreen?
![Page 19: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/19.jpg)
Quiet please
• Applause
![Page 20: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/20.jpg)
What was Hitler's dream?
![Page 21: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/21.jpg)
Quiet please
• Applause
![Page 22: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/22.jpg)
What can be seen in the night sky?
![Page 23: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/23.jpg)
Quiet please
• Applause
![Page 24: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/24.jpg)
What would cause gridlock in a major city?
![Page 25: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/25.jpg)
Quiet please
• Applause
![Page 26: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/26.jpg)
What melody did Mozart compose as a child?
![Page 27: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/27.jpg)
Quiet please
• Applause
![Page 28: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/28.jpg)
What is the airspeed velocity of an unladen swallow?
![Page 29: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/29.jpg)
Quiet please
• Applause
![Page 30: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/30.jpg)
Thank you
![Page 31: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/31.jpg)
![Page 32: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/32.jpg)
![Page 33: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/33.jpg)
![Page 34: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/34.jpg)
What is the most shining thing you've ever seen?
![Page 35: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/35.jpg)
What is the main source of energy for our planet?
![Page 36: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/36.jpg)
When I look up into the sky I see...
![Page 37: Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,](https://reader036.vdocuments.net/reader036/viewer/2022070400/56649e685503460f94b64e38/html5/thumbnails/37.jpg)
What would an Irishman like to see?