april morton gulcan can rui hu - unifr.ch morton gulcan can rui hu ... develop statistical shape...
TRANSCRIPT
![Page 1: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/1.jpg)
MAAYA: Multimedia Analysis and Access for Documentation and
Decipherment of Maya Epigraphy
April Morton
Gulcan Can
Rui Hu
February 27th, 2013
![Page 2: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/2.jpg)
The MAAYA Project
1
Main goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
Dr. Daniel Gatica-Perez
Gulcan Can
Dr. Jean-Marc Odobez
Dr Rui Hu
Prof. Stéphane Marchand-Maillet
Dr. Edgar Roman-Rangel
April Morton
Oscar Dabrowski
Prof. Nikolai Grube
Carlos Pallan
Guido Krempel
![Page 3: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/3.jpg)
Outline
• Mayan Overview
• Project Overview
• Data Preparation/Storage
• Computer Vision Algorithms
• Using Context to Improve Glyph Recognition
• Visualization
• Conclusion
2
![Page 4: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/4.jpg)
Mayan Overview
3
Ancient Mayan Culture
Flourished from BC 2000 - 1500 AD
![Page 5: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/5.jpg)
4
Mayan Writing System
Mayan Overview
Logographs
Syllabographs
Iconography
![Page 6: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/6.jpg)
Take Photographs
Obtain Digital Line Drawings
Identify Glyphs Annotate Glyphs
Documentation Process
5
Mayan Overview
Need better digital preservation and automatic analysis techniques
![Page 7: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/7.jpg)
6
The CODICES Project
Project Overview
Statistical Shape Descriptors
CBIR System
Main goal: Develop statistical shape descriptors and a CBIR system
![Page 8: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/8.jpg)
The MAAYA Project
Project Overview
Data Preparation/Storage
Content-Based Data Processing/Computer
Vision Algorithms High Level Analysis Visualization
Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
7
![Page 9: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/9.jpg)
8
Data Preparation/Storage Process
Segment glyphs and blocks
Upload segmented page into database
Annotate and store glyphs
![Page 10: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/10.jpg)
Computer Vision Algorithms
9
Glyph Representation
Identification
(Classification
- Retrieval)
Attribute Learning
Where is the
data?
How to recognize the data?
Localization
Segmentation
Decomposition
Sparse Representation
Shape Descriptors
Part-based Representation
![Page 11: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/11.jpg)
Computer Vision Algorithms
10
Localization
![Page 12: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/12.jpg)
Computer Vision Algorithms
Localization
11
![Page 13: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/13.jpg)
Computer Vision Algorithms
12
Segmentation
![Page 14: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/14.jpg)
Hierarchical Decomposition
Codices → Page → T'ol → Row of Blocks → Block → Glyph
Computer Vision Algorithms
13
![Page 15: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/15.jpg)
• Glyph representation + Identification • Can we utilize/capture diagnostic features?
Computer Vision Algorithms
14
Recognition
![Page 16: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/16.jpg)
Computer Vision Algorithms
15
Diagnostic Features
![Page 17: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/17.jpg)
Computer Vision Algorithms
16
Histogram-of-Orientations Shape Context
![Page 18: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/18.jpg)
17
Learning Features with Autoencoders
Computer Vision Algorithms
![Page 19: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/19.jpg)
Computer Vision Algorithms
18
Compute pivots and HOOSC descriptors
k-means clustering & bag-of-visual-words
representation
Ranking of “bov”s of each glyph
Identification – Glyph Retrieval
![Page 20: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/20.jpg)
Computer Vision Algorithms
16
Compute pivots and HOOSC descriptors
Classify each pivot
Apply voting for glyph-level prediction
Identification - Classification
19
![Page 21: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/21.jpg)
Using context to improve glyph recognition
20
![Page 22: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/22.jpg)
Using context to improve glyph recognition
21
![Page 23: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/23.jpg)
Using context to improve glyph recognition
22
![Page 24: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/24.jpg)
Category:T668.T102 Cha-ki Meaning: Rain god
Category:T227:T561.T23 xib?-ka’an-na Meaning: man, sky, ?
Using context to improve glyph recognition
23
![Page 25: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/25.jpg)
Data preparation
Using context to improve glyph recognition
24
![Page 26: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/26.jpg)
Glyph matching
Using context to improve glyph recognition
25
![Page 27: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/27.jpg)
Glyph matching
Using context to improve glyph recognition
26
![Page 28: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/28.jpg)
● Statistic language model
● Icon
● Translation
Category:T668.T102 Cha-ki Meaning: Rain god
Category:T227:T561.T23 xib?-ka’an-na Meaning: man, sky, ?
Using context to improve glyph recognition
27
![Page 29: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/29.jpg)
Context Information
Using context to improve glyph recognition
28
![Page 30: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/30.jpg)
● Improve recognition accuracy
● Help with noisy and partially missing examples
● High-level translation
Using context to improve glyph recognition
29
![Page 31: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/31.jpg)
A final goal is to advance state-of-the art in effective interactive visualization tools
Visualization
Input Data
Apply Data Mining
Technique
Display Visualization
Accept User Input
Update Parameters
30
![Page 32: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/32.jpg)
MDS projection based on weighted Euclidean distance function
Interactive Clustering
Visualization
Data from UCI wine data set
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
31
![Page 33: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/33.jpg)
Interactive Clustering
Visualization
Re-projection
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
32
![Page 34: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/34.jpg)
Interactive Clustering
Visualization
Updated Weights
Re-projection Could think of data points as glyphs
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
33
Distribution of values among attributes
![Page 35: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/35.jpg)
Visualization
Can filter and sort words based on probability of co-occurrence
Area encodes probabilities
Topic Model Visualization
Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
34
![Page 36: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/36.jpg)
Topic Model Visualization
Visualization
Area encodes probabilities
Can filter and sort words based on probability of co-occurrence
Can think of terms as glyphs
Can think of documents as codices (add) Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
35
![Page 37: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/37.jpg)
Conclusion
The MAAYA project aims to improve the entire process of digital glyph analysis
Data Preparation/Storage
Content-Based Data Processing/Computer
Vision Algorithms High Level Analysis Visualization
36
Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
![Page 38: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/38.jpg)
And a special thanks to the Swiss National Science Foundation: Project CR21I2L_144238
Acknowledgements
37
![Page 39: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/39.jpg)
Questions
38
![Page 40: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/40.jpg)
Backup Slides
![Page 41: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/41.jpg)
The webpage includes a user friendly upload wizard for segmented files
The Online Wizard
![Page 42: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/42.jpg)
As well as a standardized data annotation page
The Data Annotation Page
![Page 43: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/43.jpg)
5
Mayan Overview
Mayan Writing System
![Page 44: April Morton Gulcan Can Rui Hu - unifr.ch Morton Gulcan Can Rui Hu ... Develop statistical shape descriptors and a CBIR system . The MAAYA ... A final goal is to advance state-of-the](https://reader031.vdocuments.net/reader031/viewer/2022022009/5af5b3e17f8b9ae9488dd34d/html5/thumbnails/44.jpg)
Mayan Writing System
6
Mayan Overview