the cognitive neural basis of object recognition and object …valser.org/2017/ppt/invited/valse...

Yanchao BiState Key Laboratory of Cognitive Neuroscience and Learning

IDG/McGovern Institute for Brain Research Beijing Normal University, China

The Cognitive Neural Basis of Object Recognition 

and Object Knowledge

1

2017.04  Xiamen, VALSE

2

Object Recognition

For a purpose beyond labeling – retrieve knowledge/response

Try to think of –Tying a snake, hugging a hammer, hugging a snake…

Lin, N., Guo, Q., Han, Z., Bi, Y., Brain Lang, 2011

Losing object knowledge…

3

Russian Literature professor in Shanghai 63 year old, left temporal lobe atrophy

“A kind of animal? Often seen in my neighbourhood. Long nose, can accompany me to grocery shopping…

”Knowledge impairment lead to difficulties in language, object recognition, object use…

Case  XRK

“moo~”

牛 /niu/, 可靠 忠厚 强壮

Object knowledge/semantic memory/concepts

4

Reductionist: to sensory/motor experience

Patterson et al., 2007; Lambon‐Ralph et al., 2017 Nat Rev Neurosci

John Locke1632‐1704

Whence comes [the mind] by that vast store, which the busy and boundless fancy of man has painted on it with an almost endless variety? ... To this I answer, in one word, From experience.

cattle

Knowledge = perceptual experiences?

brown

walk

Martin et al., 1995, Science Simmons et al., 2007, Cereb CortexSimmons et al., 2013 Nat Neurosci

6

Questions:

Relationship between perception and knowledge?

Shape on ventral visual pathway (VOTC): Visual format?

How are different types of knowledge bound together?

Anatomical and functional networks for object understanding7

Visual pathway: Object recognition areas – Organized by Domains

8Kanwisher 2010 PNAS; Martin 2007 Annu Rev Psyc

Kanwisher, 2010, PNAS (see also Martin, 2007, Ann Rev Psych.)

Center‐PeripheryOrganization Principle

Levy et al., 2001

Nasr et al., 2014

Srihasam et al., 2014

face

buildings

L

Objects > faces

8 2 ‐2 ‐8

Helvetica TetrisCartoon faces Monkey faces

Reflect visual properties?

Rectilinear vs. Round

Low‐level Visual Feature Principle

Scene‐Rectilinear Sensitiveness

Animate‐Curve Sensitiveness

9

Reflect visual properties?? -- Congenitally blind vs. Sighted

• Blind =/= sighted Reflect visual

• Blind = sighted Not fully dependent on visual

Measure 2:Resting-state functional connectivity pattern

Vector 1-180

Measure 1:Task induced activation level to 16 object categories: human face parts, human body parts, daily scenes, tools, mammals, reptiles, birds, fishes, bugs, fruits and vegetables, flowers, preserved food, clothes, musical instruments, vehicles, furniture

Wang, Peelen, Han, He, Caramazza, & Bi, 2015, J Neurosci 10

Function and connection: Congenitally blind vs. Sighted

11

Task induced activation level to 16 object categories: 

Resting‐state functional connectivity pattern

Wang, Peelen, Han, He, Caramazza, & Bi, 2015, J Neurosci

“Multi-modal” region1: anterior medial temporal

12Wang, Peelen, Han, He, Caramazza, & Bi, 2015, J Neurosci

“Multi-modal” region1: anterior medial temporal

13Wang, Peelen, Han, He, Caramazza, & Bi, 2015, J Neurosci

“Visual” regions

14Wang, Peelen, Han, He, Caramazza, & Bi, 2015, J Neurosci

VOTC

Animate Artifact

More transparent mappingwith action/function representationeasily accessed by diff modalities

More “multimodal”

Bi Y., Wang X. & Caramazza A., 2016, Trends Cog Sci

More “isolated” from nonvisual properties

More “visual”

Grill‐Spector, K., & Weiner, K.S., 2014, Nat Rev Neurosci

Linking anatomical features to large‐scale functional maps in the ventral temporal cortex

17

Plasticity – different representation in blind and sighted? Testing the “shape” knowledge explicitly…

Shape similarity ratings

• R between shape and tactile: .00

0.88**

0.84**

0.84**

19

Independent rating

Peelen, He, Han, Caramazza, Bi, 2014, J Neurosci

Artifact “multimodal” shape representation

IT: Shape knowledge in blind and sighted

IT

20Peelen, He, Han, Caramazza, Bi, 2014, J Neurosci

Relationship between perception and knowledge?

Shape on ventral visual pathway (VOTC): Visual format?

How are different types of knowledge bound together?21

YES for animals; NO for artifacts

Structural connections supporting object concept

?

Han, Z., Ma, Y., Gong, G., He, Y., Caramazza, A., Bi, Y., 2013. Brain

White matter tract integrity in 87 patients with brain damage

Picture associative matchingSound naming Picture naming

Structural connections for object conceptLesion volume Mean FA

Thalamas-IFG

IFG, pSTG/MTG, SPL/IPL/FFG/OECT/DLPFC…

Temporal lobe-IFG

Han, Z., Ma, Y., Gong, G., He, Y., Caramazza, A., Bi, Y., 2013. Brain

Functional connections for object processing efficiencyHealthy Ss

Resting-state fMRI

Picture associative matchingSound naming Picture naming

pMTG

Wei, T., Liang, X., He, Y., Zang, Y., Han, Z., Caramazza, A., Bi, Y., 2012. J. Neurosci.

0.01‐0.1 HzPositive FC All FCs(N = 4296) (N = 36110)

Category pairwise classificationFace‐Scene 95% 100%Face‐Animal 85% 85%Face‐Tool 93% 98%Scene‐Animal 100% 95%

Scene‐Tool 95% 90%Animal‐Tool 93% 93%Multi‐category classification

85% 83%

Wang et al., in prep25

Functional connectivity patterns predict object domain task states

Wang et al., 2016, Human Brain Mapp

How are different types of knowledge bound together?

26

By rich structural and functional connections, which MATTER for object recognition: • Lesions lead to deficits; • Strength correlates with efficiency; • Patterns predict states

Conclusions

27

“carving the nature at its joints” ‐‐ Plato

“Human brain is a ‘nature carver’ ‐‐ Evolution ‘develops’ different brain templates for processing different object domains of evolutionary signifiance, through local and connections properties”

Acknowledgment

Xiaoying WangXiaosha Wang Yuxing FangTao WeiChenxi He

CollaboratorsAlfonso Caramazza, Harvard/CiMec ItalyMarius Peelen, CiMec, ItalyYong He, BNUGaolang Gong, BNUZaizhu Han, BNU

Funding agenciesNational Natural Science Foundation China973 program, Ministry of Science and Techonology, ChinaBeijing Natural Science Foundation

28

Concept Lab at BNU