recent advances in viper
DESCRIPTION
David Mihalcik David Doermann Charles Lin. Recent Advances in ViPER. What is ViPER?. A tool for evaluating video understanding algorithms. Includes: An Annotation Tool For labeling ground truth and browsing results. A Comparison Tool - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/1.jpg)
Recent Advances in ViPER
David Mihalcik
David Doermann
Charles Lin
![Page 2: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/2.jpg)
What is ViPER?
A tool for evaluating video understanding algorithms.
Includes:– An Annotation Tool
For labeling ground truth and browsing results.
– A Comparison Tool For evaluating result data with respect to ground truth.
![Page 3: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/3.jpg)
What is the Problem?
Lots of people, here and elsewhere, are working on video processing algorithms for information extraction, etc.
Evaluating performance of the algorithms requires a lot of work, usually with tools developed by the algorithm designer.
Is your solution any good? Prove it.
![Page 4: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/4.jpg)
How to evaluate the algorithm?
What is the problem?– ViPER focuses on evaluating solutions to detection
and tracking problems; these determine if or where in the video some entity or event appears.
Evaluation– Comparison of the result data set against a truth
data set.– Truth, metrics, and rules for comparison are task
dependent.
![Page 5: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/5.jpg)
Goal of the ViPER Project
To make evaluation of video algorithms simple, repeatable, and ubiquitous.
As ground truth is required for evaluation, annotation must be made simple, as well.– Avoid tedium.– Avoid frustration.– Support expert usage.
![Page 6: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/6.jpg)
Ground Truth Annotation
ViPER-GT supports annotation of temporally qualified spatial and nominal data on video files and still images.
Go from the simple per-frame or shot annotations to detailed spatial markup.– You can quickly indicate which frames contain people.– Then, you add how many people per frame.– With a lot of time and money, you can put boxes around
them.
![Page 7: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/7.jpg)
ViPER-GT: Video Ground Truth Annotation Tool
![Page 8: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/8.jpg)
For Example: Person Tracking
How well does an algorithm find and track humans moving through a video?
To evaluate detection:– Truth must indicate which frames contain the
person.
To evaluate tracking:– Truth must contain spatial information, indicating
where a person may be found.
![Page 9: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/9.jpg)
Example of Annotation: Person Detection
![Page 10: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/10.jpg)
Example of Annotation: Person Tracking
![Page 11: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/11.jpg)
The ViPER Data Model
Similar to a relational database:– Tables are Descriptor Definitions.– Columns are Attributes.– Rows are Descriptor Instances.
Most descriptors are OBJECT descriptors:– Attributes are temporally qualified.– Static OBJECTS have a frame range, but their
attributes are not temporally qualified. Useful for events, etc.
![Page 12: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/12.jpg)
The ViPER File Format and API
Uses XML. I won’t go into it here. There is a Java API.
![Page 13: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/13.jpg)
Related Work
VideoAnnEx OntoLog PhotoStuff Informedia
![Page 14: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/14.jpg)
VideoAnnEx
IBM’s MPEG-7 annotation tool. Cool Features:
– Cut detection makes it easy to add per-shot markup.– Supports MPEG-7.
Annoyances:– Not very good for spatial attributes.– Commercial software. Not as extensible as ViPER.
![Page 15: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/15.jpg)
OntoLog
Jon Heggland’s Tool for Temporal Markup with Ontologies
Advantages:– Good support for key bindings and playback.– Data model supports inheritance.
Annoyances:– No spatial data support.
![Page 16: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/16.jpg)
PhotoStuff
MINDSWAP’s Tool for Adding Semantic Web Markup to Images
Cool Features:– Semantic Markup!– Spatial Data!
Annoyances:– Buggy and beta.– No support for video.
![Page 17: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/17.jpg)
Informedia
CMU’s tool for browsing video libraries Cool Features:
– Advanced browsing functionality. But…
– Focus on video library, not annotation.– Not available for download, from what I can tell.– Not terrifically extensible.
See also: Silver and Malach
![Page 18: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/18.jpg)
Extending the Interface
ViPER provides a lot of functionality, but is very general.
It may be appropriate to extend viper-gt to better support marking up a different type of annotation.
![Page 19: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/19.jpg)
Example Extension: Adding Text Zones
Adds a toolbar that allows typed bounding boxes.
Instead of having to click create, auto-creates a new box.
![Page 20: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/20.jpg)
Architecture of ViPER-GT
Application Launcher– Loads a set of javabeans from an RDF model.– Allows modifying menus, i18n, etc.– Is a bit of a pain to handle 'menu change' events,
like for most recently used menu. Viper View Mediator
– Javabean container for ViPER API.– Adds 'user interaction' methods to keep track of
things not in API (focus, filters).
![Page 21: Recent Advances in ViPER](https://reader031.vdocuments.net/reader031/viewer/2022013108/5681547d550346895dc2921a/html5/thumbnails/21.jpg)
Continual Improvement
SourceForge web site:– http://viper-toolkit.sf.net/
Mail suggestions/comments to:– [email protected]