의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅...
TRANSCRIPT
![Page 1: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/1.jpg)
의미정보 해석- 지식기반 시스템 응용 -
2006.11.21최보윤
소프트컴퓨팅 연구실연세대학교
![Page 2: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/2.jpg)
Collaborative capturing and interpretation of interactions
Y. Sumi, I. Sadanori, T. Matsuguchi, S. Fels, and K. Mase Pervasive 2004 Workshop on Memory and Sharing of Experiences, pp. 1-7, 20
04.
![Page 3: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/3.jpg)
Overview
• Introduction• Capturing interactions by multiple sensors• Related works• Implementation• Interpreting interactions• Video summary• Corpus viewer: Tool for analyzing interaction patterns• conclusions
![Page 4: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/4.jpg)
Introduction
• Interaction corpus– Action highlights
• Generate diary
– Social protocols of human interactions
• Sensors– Video cameras, microphone and physiological sensors
• ID tags– LED tag: infrared LED– IR tracker:
• Infrared signal tracking device• Position and identity
![Page 5: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/5.jpg)
Capturing interactions by multiple sensors
• Recording natural interactions
– Multiple presenters and visitors in an exhibition room
• Sensors & Humanoid robots– Wearable sensors,
stationary sensors• Monitoring humans• Video camera, microphone,
IR tracker
– Recording robots’ behavior logs and the reactions of the humans which connect the robots
• Central data server– Getting the data from the
sensors and humanoid robots
![Page 6: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/6.jpg)
Related works
• Smart environment– Supporting humans in a room– The Smart rooms, Intelligent room, AwareHome, Kidsroom and EasyLiving– Recognition of human behavior and understanding of the human’s intention
• Wearable systems– Collecting personal daily activities– Intelligent recording system
• Video summary systems– The physical quantity of video data captured by fixed cameras
![Page 7: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/7.jpg)
Exhibition roomImplementation
![Page 8: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/8.jpg)
IR tracker & LED tagImplementation
![Page 9: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/9.jpg)
Interpreting interactions
• Define interaction primitives– Events– Significant intervals or moments of activites
• IR tracker and LED tag• minInterval and maxInterval
– minInterval: 5 sec– maxInterval
• Ubiquitous sensors: 10 sec• Wearable sensors: 20 sec
![Page 10: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/10.jpg)
Video summary
• Assumptions– User , Booth
• Co-occurences
• Video summarization
![Page 11: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/11.jpg)
Corpus viewer: Tool for analyzing interaction patterns
![Page 12: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/12.jpg)
conclusions
• Method to build an interaction corpus using multiple sensors• Segment and interpret interactions from huge data• Provide a video summary• Help social scientists
![Page 13: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/13.jpg)
Using context and similarity for face and location identification
M. Davis, M. Smith, F. Stentiford, A. Bambidele, J. Canny, N. Good, S. King and R. Janakiraman
Proceedings of the IS&T/SPIE 18th Annual Symposium on Electronic Imaging Science and Technology Internet Imaging VII, 2006.
![Page 14: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/14.jpg)
Overview
• Introduction• System Overview• Content Analysis• Experimental Data• Experimental Design• Evaluation• Discussion and Results• Conclusions & Future Work
![Page 15: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/15.jpg)
Introduction
• New way for the unsolved image content recognition– Mobile media capture, context-sensing, programmable computation and networking i
n the form of the nearly ubiquitous cameraphone• Cameraphone
– Platform for multimedia computing– Combination with the analysis of automatically gathered contextual metadata and m
edia content analysis• Contextual metadata
– Temporal– Spatial– Social– Face recognition and place recognition
• Precision of face recognition– PCA 40%, SFA 50%
• Precision of location recognition– Color histogram 30%, CVA 50%, contextual metadata and CVA 67%
![Page 16: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/16.jpg)
System Overview
• MMM2– Gathering data and metadata– Server application: store photo metadata and user profile information– Client application: run the client handset– MMM2 Context Logger
• University of Helsinki• Location information, Bluetooth radio• Detect new photos, display interface or web browser, upload MMM2 server
– MMM2 website• Select a region of a photo and associate a person’s name with this region
• Creation of Ground-Truth Dataset
![Page 17: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/17.jpg)
Location Recognition
• Similarity measures– Pattern recognition problem
• Cognitive Visual Attention– Comparison of two image– Drawn the parts in common– No memory of data
• Training and Classification– A nearest neighbor classifier– Location classification
• Visual Sub-cluster Extraction– Many different photos at each location– Location class by several sub-clusters– Adding more exemplars
• Not guarantee improvements
• Color Histogram Techniques– Pixel color distributions– Simplest
visual sub cluster example corresponding to an exemplar
Content Analysis
![Page 18: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/18.jpg)
Face Recognition & GPS
• PCA– Eigenface principle– Short training time– Best accuracy
• LDA+PCA– LDA: Multiple images training
• Bayesian MAP & ML– Maximum a posteriori (MAP), maximum likelihood (ML)– Difference or similarity between two photos
• SFA (Sparse Factor Analysis)– – Y: a vector of (partially) observed values, X: latent vector representing user preferenc
e, m: “model” predicting user behavior, N: noise function• GPS Clustering
– Suitable format– K-means and farthest first cluster
Content Analysis
![Page 19: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/19.jpg)
Experimental Data
• Face Recognition on Cameraphone Data– NIST FERET dataset
• Mugshot– Full frontal view– Head-and-shoulders
– 27,000 cameraphone potos• 66user, 10 months• Multiple people• Real world
• Photographic Location Data– 1209 images
• Nokia 7610 cameraphones• 12 location, 30 cell identities • Berkeley Campuss
![Page 20: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/20.jpg)
Experimental Design
• Training gallery– Hand-labeled with the names– Min of distances between all images in the photo and training gallery
image k
• SFA model– Training
• Contextual metadata and the face recognizer outputs• Contextual metadata only
– Evaluation• Precision-recall plots for each of the computer vision algorithms
– Time• Training time: 2 minutes• Training for the Bayesian classifiers: 7 hours• PCA and LDA classifiers: less than 10 minutes• Face recognition for 4 algorithms: less than 1 minute
![Page 21: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/21.jpg)
Evaluation
• Location by Contextual Metadata– Distribution of metadata: 579 items, 12 location
• Location by Metadata and Vision
![Page 22: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/22.jpg)
Face Identification Experimental ResultsDiscussion and Results
![Page 23: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/23.jpg)
Location Identification Experimental Results
• Histogram classifier, the CVA classifier and metadata classifier– Bad performance
• Metadata– Limit the errors with Cell ID– Specific place at certain times of the day and days of the week
Discussion and Results
Error Rate Increase Per Feature Removed
![Page 24: 의미정보 해석 - 지식기반 시스템 응용 - 2006.11.21 최보윤 소프트컴퓨팅 연구실 연세대학교](https://reader036.vdocuments.net/reader036/viewer/2022081506/56649dff5503460f94ae6e18/html5/thumbnails/24.jpg)
Conclusions & Future Work
• New approach to the automatic identification of human faces and location if mobile images
• Combination of attributes– Contextual metadata– Image processing
• Torso-matching
• Context-aware location recognition research