1 querying graphics through analysis and recognition inria lorraine

1

Querying Graphics through Analysis and Recognition

INRIA Lorraine

2

Research fields

•Image processing and segmentation

•Structural pattern recognition

•Statistical pattern recognition

•Information spotting and retrieval

In the context of the analysis and recognition of graphics-rich documents

3

Querying Graphics through Analysis and Recognition

4

Querying Graphics through Analysis and Recognition

5

Querying Graphics through Analysis and Recognition

6

Querying Graphics through Analysis and Recognition

7Scientific staff

Suzanne Collin, Assist. Prof. UHP

Philippe Dosch, Assist. Prof. Nancy 2

Bart Lamiroy, Assist. Prof. INPL/Mines

Gérald Masini, CR CNRS

Salvatore Tabbone, Assist. Prof. U. Nancy 2

Karl Tombre, Prof. INPL/Mines

Laurent Wendling, Assist. Prof. UHP

PhD students

Sabine Barrat, CIFRE contract (pending)

Thi Oanh Nguyen, joint supervision with IFI (Hanoi, Vietnam)

Oriol Ramos Terrades, joint supervision with UAB, Barcelona (Spain)

Jan Rendek, CIFRE France Télécom

Jean-Pierre Salmon, FRESH (European project)

Zhang Wan, joint supervision with City U. Hong Kong (pending)

Daniel Zuwala, MESR grant

Technical staff

Yamina Smail, Epeires project

X, Fresh project (pending)

Administrative staff

Isabelle Herlich (part time)

Françoise Laurent (part time)

8

Main results 2004-05Hierarchical binarization

9

Focus on symbol recognition – Symbol spotting combining Radon-based signature and structural approach

Main results 2004-05

10

Improvement of recognition rates through combination of shape descriptors

Main results 2004-05

The set of images I

Recognition rates by descriptors

010

2030

405060

7080

90100

1 2 3 4 5 6 7 8 9

Clusters

Re

co

gn

itio

n r

ate

s

Compactness

Ellipticity Degree

Angular Signature

Generic FourierDescriptors

R-signature

Weighted Sumwithout weighted map

11

Application : extraction of letters in heritage documents

Descriptors C E SA GFD TRf WS

Before 49 41 70 59 50 55

After 52 39 75 69 50 72Recognition rates

Descriptors C E SA GFD TRf WS

Before 85 78 90 89 80 96

After 93 81 97 94 87 100

Ranking

Improvement of recognition rates through combination of shape descriptors

Main results 2004-05

12

Main results 2004-05Raster-to-vector conversion method based on random sampling and parametric fitting

13

Segmenting the skeleton

RANVEC : Random sampling on pairs of vector pointsExtension of primitive as long as it fits arc or segment (linear regression)

14

Simplification and unification of primitives

15

GREC’01 GREC’03 GREC’05

Winner Dave Elliman JiQiang Song Xavier Hilaire

VRI 0,681 0,609 0,803

Arc segmentation contest

16

Application domains/transferElectrical wiring diagrams in aeronautics FRESH project (FP6 STREP Aeronautics program)

17

Application domains/transfer

Cultural heritage documents ACI Madonne, FP6 STREP proposal QUIMERA-Doc submitted 9/05

18QgarLib : library of C++ classes

QgarApps : applications

QgarGUI : user interface

qgar.org, APP

Refactoring to professional standards

Open architecture (XML)

80,000 lines of C++ code (comments not counted)

30 to 40 downloads of code per month

>10 documentation browses per day (robots excluded)

19

Positioning within INRIA

Fully within one of INRIA’s 7 challenges in strategic plan: Developing multimedia data and information processing

Regular partnership with Imadoc (research group at Irisa)

Joint contacts Texmex (Sym-C)/Qgar with industrial partner

Recent contacts with Lear on browsing of large image bases

20

Collaborations

National: informal consortium Nancy, Rennes, La Rochelle, Rouen, Tours, Lyon with several joint projects (ACI Madonne, RNTL past and submission, Techno-Vision Epeires, IST submission) and coordination of actions

CVC/UAB, Barcelona: long lasting relationship, associated team SymbolRec, joint PhD supervisions

City University Hong Kong: associated in Epeires, PAI submission accepted, joint PhD supervision

IFI, Vietnam: joint PhD supervision

University of Auckland (NZ), University of Bern, Carleton University (Canada)

21

Achievements, strengths, weaknesses

Leadership position at international level on graphics recognition

Announced in project and largely addressed:• Symbol recognition and spotting

• Performance evaluation

Strong and adequate applicative backing

Improvement in number of PhD students

Still low on permanent workforce

22

Future work

Scalability of symbol recognition methods• Large number of models

• Variations within the same shape class

– Combining structural and statistical methods

– Hierarchical approach

23

Future workComplex symbols

24

Future workDynamic, on-the fly recognition and spotting: from model-based recognition to freehand recognition

25

Future work

Multi-modal indexation (text / graphics / image / video) in multimedia and document databases (collaborations with Texmex, Lear, …)

Interactivity with user (relevance feedback)

26

Future work

Performance evaluation• International symbol recognition contests 2003 & 2005

• Epeires

– French Techno-Vision program

– 4 universities, FT R&D, 1 company + foreign partners UAB & CityU

– www.epeires.org

• Future research challenges

– Simple and non-biased metrics

– Ground-truth/recognition output matching methods

– Generation of large sets of training and benchmarking data using realistic image degradation models

27Epeires – ground-truthing

28

Future work

Software : increase number of applications