imageclef 2011 overvie · cross language image retrieval imageclef 2011 henning müller 1, theodora...
TRANSCRIPT
![Page 1: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/1.jpg)
Cross Language Image Retrieval ImageCLEF 2011
Henning Müller1, Theodora Tsikrika1, Steven Bedrick2, Hervé Goeau3, Alexis Joly3, Jayashree Kalpathy-Cramer2, Jana Kludas4, Judith
Liebetrau5, Stefanie Nowak5, Adrian Popescu6, Miguel Ruiz7 1 University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
2 Oregon Health and Science University (OHSU), Portland, OR, USA 3 IMEDIA, INRIA, France
4University of Geneva, Switzerland 5 Fraunhofer Institute for Digital Media Technology, Ilmenau, Germany
6 CEA LIST, France 7 University of North Texas, USA
![Page 2: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/2.jpg)
Support
![Page 3: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/3.jpg)
ImageCLEF History
![Page 4: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/4.jpg)
ImageCLEF 2011
• General overview o news, participation, management
• Tasks
o Medical Image Retrieval
o Wikipedia Image Retrieval
o Photo Annotation
o Plant Identification
• Conclusions
![Page 5: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/5.jpg)
News - ImageCLEF 2011
• Medical Image Retrieval o larger image collection, open access literature o challenges with many irrelevant images
• Wikipedia Image Retrieval o large image collection with multilingual annotations/topics o impoved image features, increased topic visual examples o crowdsourcing for image relevance assessment
• Photo Annotation o new sentiment concepts added o concept-based retrieval sub-task o crowdsourcing for image annotation
• Plant Identification
![Page 6: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/6.jpg)
Participation
![Page 7: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/7.jpg)
ImageCLEF Management
• Online management system for participants o registration, collection access, result submission
![Page 8: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/8.jpg)
ImageCLEF web site: http://www.imageclef.org
• Unique access point to all information on tasks & events
• Access to test collections from previous years
• Use of content-management system so that all 12 organisers can edit directly
• Very appreciated! • Very international access!
![Page 9: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/9.jpg)
Medical Image Retrieval Task
![Page 10: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/10.jpg)
Tasks proposed
• Modality detection task o purely visual task, training set with modalities given o one of 18 modalities had to be assigned to all images
• Image-based retrieval task o clear information need for a single image, three
languages, example images o topics are derived from a survey of clinicians
• Case-based retrieval task o full case description from teaching file as example but
without diagnosis, including several image examples o unit for retrieval is a complete case or article, closer to
clinical routine
![Page 11: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/11.jpg)
Setup
• New database for 2011! • 231,000 figures from PubMed Central articles
o Includes figures from BioMed Central journals o Annotations include figure captions o all in English
• Topics re-used from 2010 • Case-based topics used a teaching file as source, image-
based topics generated from survey of clinicians • Relevance judgements performed by clinicians in Portland
OR, USA o double judgements to control behavior and compare
ambiguity o several sets of qrels, but ranking remains stable
![Page 12: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/12.jpg)
Participation
• 55 registrations, 17 groups submitting results (*=new groups) o BUAA AUDR (China)* o CEB, NLM (USA) o DAEDALUS UPM (Spain) o DEMIR (Turkey) o HITEC (Belgium)* o IPL (Greece) o IRIT (France) o LABERINTO (Spain)* o SFSU (USA)* o medGIFT (Switzerland) o MRIM (France) o Recod (Brazil) o SINAI (Spain) o UESTC (China)* o UNED (Spain) o UNT (USA) o XRCE (France)
![Page 13: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/13.jpg)
Example of a case-based topic
Immunocompromised female patient who received an allogeneic bone marrow transplantation for acute myeloid leukemia. The chest X-ray shows a left retroclavicular opacity. On CT images, a ground glass infiltrate surrounds the round opacity. CT1 shows a substantial nodular alveolar infiltrate with a peripheral anterior air crescent. CT2, taken after 6 months of antifungal treatment, shows a residual pulmonary cavity with thickened walls.
![Page 14: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/14.jpg)
Results
• Modality detection task: o Runs using purely visual methods were much more
common than runs using purely textual methods o Following lessons from past years' campaigns, "mixed"
runs were nearly as common as visual runs (15 mixed submissions vs. 16 visual)
o The best mixed and visual runs were equivalent in terms of classification accuracy (mixed: 0.86, visual: 0.85).
o Participants used a wide range of features and software packages
![Page 15: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/15.jpg)
Modality Detection Results
![Page 16: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/16.jpg)
Results
• Image-based retrieval: o Text-based runs were more common- and performed
better- than purely-visual o Fusion of visual and textual retrieval is tricky, but does
sometimes improve performance • The three best-performing textual runs all used query
expansion, often a hit-or-miss technique • Lucene was a popular tool in both the visual and textual
categories • As in past years, interactive or "feedback" runs were rare
![Page 17: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/17.jpg)
Results
• Case-based retrieval: o Only one team submitted a visual case-based run; the
majority of the runs were purely textual o o The three best-performing textual runs all used query
expansion, often a hit-or-miss technique • Lucene was a popular tool in both the visual and textual
categories o In fact, simply indexing the text of the articles using
Lucene proved to be an effective method • As in past years, interactive or "feedback" runs were rare
![Page 18: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/18.jpg)
Results
![Page 19: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/19.jpg)
Judging
• Nine of the topics were judged by at least two judges • Kappa scores were generally good, and sometimes very
good… • Worst was topic #14 (“angiograms containing the aorta”) with ≈0.43 • Best was topic #3 (“Doppler ultrasound images (colored)”) with ≈0.92
• Kappas varied from topic to topic and judge-pair to judge-pair. • For example, on topic #2:
– judges 6 and 5 had a kappa of ≈0.79… – … while judges 6 and 8 had a kappa of ≈0.56
![Page 20: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/20.jpg)
Wikipedia Image Retrieval Task
![Page 21: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/21.jpg)
Wikipedia Image Retrieval: Task Description
• History: o 2008-2011: Wikipedia image retrieval task @ ImageCLEF o 2006-2007: MM track @ INEX
• Description: o ad-hoc image retrieval o collection of Wikipedia images
large-scale heterogeneous user-generated multilingual annotations
o diverse multimedia information needs
• Aim: o investigate multimodal and multilingual image retrieval approaches
focus: combination of evidence from different media types and from different multilingual textual resources
o attract researchers from both text and visual retrieval communities o support participation through provision of appropriate resources
![Page 22: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/22.jpg)
Wikipedia Image Collection • Image collection created in 2010, used for the second time in 2011
o 237,434 Wikipedia images o wide variety, global scope
• Annotations o user-generated
highly heterogeneous, varying length, noisy o semi-structured o multi-lingual (English, German, French )
10% images with annotations in 3 languages 24% images with annotations in 2 languages 62% images with annotations in 1 language 4% images with annotations in unidentified language or no annotations
• Wikipedia articles containing the images in the collection • Low-level features for
o CEA-LIST, France provided cime : border/interior classification algorithm tlep: texture + colour SURF: bag of visual words
o Democritus University of Thrace, Greece provided CEDD descriptors
![Page 23: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/23.jpg)
Wikipedia Image Collection
![Page 24: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/24.jpg)
Wikipedia Image Collection
![Page 25: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/25.jpg)
Wikipedia Image Collection
![Page 26: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/26.jpg)
Wikipedia Image Collection
![Page 27: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/27.jpg)
![Page 28: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/28.jpg)
Wikipedia Image Retrieval: Relevance Assessments
• crowdsourcing o CrowdFlower o Amazon MTurk workers
• pooling (depth = 100) • on average 1,500 images to assess
• HIT: assess relevance • 5 images per HIT • 1 image gold standard • 3 turkers per HIT • 0.04$ • majority vote
![Page 29: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/29.jpg)
Wikipedia Image Retrieval: Participation
• 45 groups registered • 11 groups submitted a total of 110 runs
51 textual 2 visual 57 mixed
42 monolingual 66 multilingual
15 relevance feedback 16 query expansion 12 QE + RF
![Page 30: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/30.jpg)
Wikipedia Image Retrieval: Results
![Page 31: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/31.jpg)
Wikipedia Image Retrieval: Conclusions
• best performing run: a multimodal, multilingual approach • 9 out of the 11 groups submitted both mono-media and multimodal runs
o for 8 of these 9 groups: multimodal runs outperform mono-media runs o combination of modalities shows improvements
increased number of visual examples improved visual features more appropriate fusion techniques
• many (successful) query/document expansion submissions • topics with named entities are easier and benefit from textual approaches • topics with semantic interpretation and visual variation are more difficult
![Page 32: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/32.jpg)
Photo Annotation Task
![Page 33: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/33.jpg)
Task Description
1) Annotation subtask: • Automated annotation of 99 visual concepts in photos • 9 new sentiment concepts
o Trainingset: 8,000 photos, Flickr User Tags, EXIF data o Testset: 10,000 photos, Flickr User Tags, EXIF data
• Performance Measures: o AP, example-based F-Measure (F-Ex), Semantic R-Precision (SR-Prec)
2) Concept-based retrieval subtask: • 40 topics: Boolean connection of visual concepts
o Trainingset: 8,000 photos, Flickr User Tags, EXIF data o Testset: 200,000 photos, Flickr User Tags, EXIF data
• Performance Measures: o AP, P@10, P@20, P@100, R-Precision
Both tasks differentiate 3 configurations: • Textual information (EXIF tags, Flickr User Tags) (T) • Visual information (photos) (V) • Multi-modal information (all) (M)
![Page 34: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/34.jpg)
GT Assessment: MTurk
Annotation subtask: • 90 concepts from 2010 • 9 sentiment concepts • Russel´s affect circle • automated verification • gold standard insertion • deviation: at most 90° • 10 images per HIT • 5 turkers per HIT • 0.07$ • GT: majority
![Page 35: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/35.jpg)
![Page 36: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/36.jpg)
![Page 37: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/37.jpg)
![Page 38: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/38.jpg)
![Page 39: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/39.jpg)
Results: Retrieval Task • Total: 31 runs
o 10 multi-modal o 14 visual o 7 textual
• Best automated run • Best manual run
• great variability of performance for different topics
Conclusions: -> great interest in annotation task (18 teams, countries) -> increase of textual runs with competitve performance to visual ones -> challenging NEW retrieval task
![Page 40: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/40.jpg)
Plant Identification Task
![Page 41: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/41.jpg)
Plant Identification: Task Description
• Objective: Automatic plant species identification based on images of their leaves
• Context = Taxonomic gap o Less and less people can identify plants o Collecting global information about plants is therefore very hard o Bridging this gap is essential for ecology management
• Task organized in collaboration with botany scientists (acquisition protocols, data collection, task objectives)
• First year pilot task focused on o Leaves (most studied and easiest organ) o Visual content + few metadata o Morphological & acquisition diversity
![Page 42: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/42.jpg)
Pl@ntLeaves dataset
• 70 Mediterranean species • 5436 images of 3 types
o Scans o Photos with uniform background o Unconstrained photos
• Built in a collaborative manner o 17 contributors from Telabotanica social network o ≠ locations, ≠ seasons, ≠ climates, ≠ ecosystems
• Metadata (XML) o Type (scans, photos,...) o GPS o Plant id o Author o Content (e.g. one or several leaves)
![Page 43: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/43.jpg)
Plant Identification: Participation
• 44 groups registered • 8 groups submitted a total of 20 runs
Group Nb of runs Methods/focus DAEDALUS 1 SIFT visual features + NN classifier IFSC 3 Boundary shape features KMIMMIS 4 SIFT visual features + NN classifier INRIA 2 Large scale matching, boundary
shape LIRIS 4 Model-driven boundary shape features RMIT 2 GIFT visual features, 2 ML methods
SABANCI-OKAN 1 Global visual features + SVM UAIC 3 Metadata & visual features
![Page 44: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/44.jpg)
Plant Identification: Results overview
Generalist CBIR & machine learning methods
Local features + rigid objects matching
Leaf boundary features
Metadata only
![Page 45: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/45.jpg)
ImageCLEF 2011 Conclusions
• Increasing interest, but stable participation... • Larger scale collections
o image retrieval (collections of 200,000 - 240,000 images) o image classification (test sets of 1,000 - 10,000 images)
• More and more realistic tasks • Fusion approaches becoming more effective, but remain difficult • Crowdsourcing for image annotation and relevance assessment • Several ideas for next year!
o What do you expect? o What are our ideas? o What data are available?
Fill in the survey www.imageclef.org/survey
![Page 46: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/46.jpg)
![Page 47: imageclef 2011 overvie · Cross Language Image Retrieval ImageCLEF 2011 Henning Müller 1, Theodora Tsikrika 1, Steven Bedrick 2, Hervé Goeau 3, Alexis Joly 3, Jayashree Kalpathy-Cramer](https://reader034.vdocuments.net/reader034/viewer/2022042404/5f1a8e538fb12808273590ec/html5/thumbnails/47.jpg)