Analysing Creative Image Queries To Determine Important Facets

Andrew MacFarlaneCity University London

(On Behalf of the PhotoBrief Project)

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Motivation

Motivation

Motivation

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Data Collection

• Contextual Interviews– 13 (10 consumers, 3 producers) – Piclet project– Understand Search Behaviour; identify systems

• Commercial Search System Analysis– 5 most popular: SE, stock image, photo-sharing– Search facets used

• Brief Analysis– Expresses need for image in creative industries

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Analysis of Search Engine Facets

Image Search System Type

ISS1 Image search engineISS2 Stock Image LibraryISS3 Stock Image LibraryISS4 Photo-sharing social network

ISS5 Photo-sharing social network

Analysis of Search Engine Facets

• Identified facet areas• Image– Bibliographic: details of the image– Descriptive: details about image ‘content’

• Business: – context for buying scenario– Business model

Search Engine FacetsImage: Bibliographic

Facet #Systems Examples

Media/File type 5 JPEG, TIFF

Source [image provider name, domain address]

5 Photographer name, collection

Timestamp 5 Uploaded 1,2 days, 1 week

Format 4 17in x 13in @ 300ppi

ID, title, description 4 ‘Cute baby eating biscuit’

Geolocation 3 Lat/Long, France

Size 2 59mb, 29.5kb

Search Engine FacetsImage: Descriptive

Facet #Systems Examples

Category 5 News, Sport, Documentary

Keyword/Tag 4 Agility, risk

Presence of people* 3 1,2 or more peopleAdult material filtering* 3 Exclude nudity

Colour* 2 Colour, B&W

Composition (copyspace)* 1 Position of objects

Reference (Query-by-example)*

1 Uploaded image

* CBIR algorithms associated with facet

Search Engine Facets: Business

Facet #Systems Examples

Rights 5 Royalty free, rights managed

Price range 2 Per image, range slide bar

Usage (use, circulation, size of placement)

1 Presentation, website, business package

Target market 1 Creative, editorial

Territory of use 1 Worldwide…

Duration of use 1 …for 5 years

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Briefs: Stakeholders

Image Providers Image Consumers

Individual Photographers, Photo Agencies, Image Libraries

Advertising or marketing professionals (individuals, agencies, companies’ departments), book publishers, designers (individuals, agencies, companies’ departments), news professionals (companies’ and individual bloggers, news agencies),private and corporate individuals

Brief ExampleYoung girls 6-12 yrs old with

intellectual abilityGirls are more than just pretty This is a broad search on Intellectual prowess over superficial looks. I need a selection of images which can help demonstrate that young girls should have more emphasis on their intellectual ability then being just pretty.Photos need to be of young girls in the 6-12 yo age range. Images can be off a girl looking her body critically, having a stack of books, looking studious, could be a post it note on her forhead, anything really...please approach this brief as broadly as possible.INTENDED USAGEEditorialWHERE?AustraliaHOW LONG?1 month, starting from March 21, 2012EXCLUSIVE USE?Exclusive use in specified regions

Analysis of Facets in Briefs

• 85 briefs from ImageBrief, Photographic Assignment Meetup, Freelancer etc.

• Facets drawn from:– Commercial search systems– Image classification Scheme

• Iterative process of coding: eliminating/merging to create comprehensive set of facets

• Outcome: 1508 phrases split into 21 facets

Briefs Statistics

23%

24%26%

9%

1%

8%

5% 5% Advertisement

Editorial illustrationPromotional productsBooks covers, CD & DVD coversItems for resaleItems for personal or corporate useFilm or motion picturesUnknown

Analysis of Briefs: Facets Classes

29%

20%

51%

Facet Classes

Business

Context

Image

Business: business decisions about time/budget, rights etc.

Image: image description (Westman, 2009).

Context: Background information for context of use.

Analysis of Briefs: Business facetsFacet Examples

Deadline 4 days

Budget Payment per image

Rights Rights management, royalty free

Territory of Use Where?UK

Duration of Use How Long <Duration> from <Date>

Model Release Required?

Quantity No of Photos.

Analysis of Briefs: Image facetsFacet Examples

Colours &Textures Colour, B&WComposition BackgroundLight Daylight Post-processing Retouching Size Vast shotGeneric objects/ scenes Summery scene

Specific objects/ scenes Utrecht

Concepts RomanceSubjective evaluation Spooky

Ref similarity Similar image attached

Analysis of Briefs: Context Facets

Facet Examples

Usage Beauty page

Target audience Advertising

Purpose Brand Products

Additional Info Here is the article

Analysis of Briefs: Coded items

Generi

cUsag

e

Composition

Subjec

tive ev

aluati

onRigh

ts

Budget

Duration of U

se

Terri

tory of U

se

Colours &Te

xtures

Deadlin

e

Speci

fic

Targe

t audien

ce

Purpose

Light

Concept

Additional Inform

ation

Ref sim

ilarit

y

Quantity Siz

e

Postpro

cessin

g

Model Rele

ase0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.016.4

10.3

8.4

6.7 6.25.4 4.8 4.6 4.5 4.5

3.8 3.3 3.2 2.9 2.9 2.9 2.7 2.2 1.7 1.4 1.4

Analysis of Briefs: Facet Presence

Usage

Rights

Generi

c

Budget

Deadlin

e

Duration of U

se

Terri

tory of U

se

Composition

Subjec

tive ev

aluati

on

Targe

t audien

ce

Purpose

Ref sim

ilarit

y

Colours &Te

xtures Lig

ht

Quantity

Additional Info

Concept

Speci

fic

Postpro

cessin

gSiz

e

Model Rele

ase0

10

20

30

40

50

60

70

80

9084

73 7370 68 68 67

5248

3935 34 33

29 27 25 24 2219 18

14

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Image Search vs Brief FacetsMatch Type List of facet matches: Brief=Image search engine [brief facet class]

Full Match[Total=17]

Budget=Price Range [B]. Rights=Rights [B]. Territory of use=Territory of use [B]. Duration of use= Duration of use [B]. Colour/Texture=Colour [I]. Composition=Composition [I]. Size=Size/Format [I]. Generic Object; Specific Object; Concept; Subjective Evaluation; Light; Purpose, Additional Information=Keyword/Tag [I,C]. Ref Similarity=Reference [I]. Usage=Usage [C]. Target Market=Target Market [C].

Partial Match[Total=1]

Model Release=Presence of people [B,I].

Brief Only[Total=3]

Assignment Photography: Deadline, Quantity.Non-Search criteria: Post-Processing.

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Conclusion

• Facet match is high, and users can find what they need with current systems, however…

• … a more detailed facet system would help users, to reflect their need expressed in briefs

• Need coverage of more facets e.g. usage• Use of Keyword/Tag facet is key issue• Transcripts: use of generic descriptions to clarify

queries • Conceptually difficult queries might require context

for disambiguation

Agenda

• Motivation• Data Collection• Analysis of Search Engine Facets• Analysis of Facets in Briefs• Image Search vs Brief Facets• Conclusion• Acknowledgements

Acknowledgements

• City University London: Elena Konkova• Robert Gordon University: Ayşe Gӧker• AmbieSense: Hans Myrhaug• MediaReach: Saad Saraf• Technology Strategy Board: Funding for

PhotoBrief project TP12144-76203

References

• Image Brief: http://www.imagebrief.com/• Photographic Assignment: http://

www.meetup.com/Photographic-Assignments • Freelancer: https://www.freelancer.co.uk/• Westman, S. (2009) Image Users' Needs and

Searching Behaviour, in Information Retrieval: Searching in the 21st Century (eds A. Göker and J. Davies), John Wiley & Sons, Ltd, Chichester, UK. doi: 10.1002/9780470033647.ch4