© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Search Patterns

Jon Handler, Amazon CloudSearch Solution Architect

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Agenda

!   Amazon CloudSearch Basics !   Searching in the Cloud !   Ranking !   Location-Based Search !   Faceting !   Mixed Data Sources !   Performance

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Patterns !   Title-Body Search !   Social Search Patterns !   Mobile Search Patterns ! eCommerce Patterns

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

AMAZON CLOUDSEARCH

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Search, In The Cloud

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

The Cloud is Elastic

SEARCH INSTANCE Index Partition n

Copy 1

SEARCH INSTANCE Index Partition 2

Copy 2

SEARCH INSTANCE Index Partition n

Copy 2

SEARCH INSTANCE Index Partition 2

Copy n

SEARCH INSTANCE

DATA Document Quantity and Size

TRAFFIC Search Request Volume and Complexity

Index Partition n Copy n

SEARCH INSTANCE Index Partition 1

Copy 1

SEARCH INSTANCE Index Partition 2

Copy 1

SEARCH INSTANCE Index Partition 1

Copy 2

SEARCH INSTANCE Index Partition 1

Copy n

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

SEARCHING IN THE CLOUD

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

CloudSearch Batches { "type": "add",! "id": "tt0076759",! "fields": { ! "title": "Star Wars",!

! "director": "Lucas, George",!! "year": 1977,!! "genre": ["Action","Adventure","Fantasy","Sci-Fi"],!! "actor": ["Ford, Harrison","Fisher, Carrie","Hamill,!! Mark","Jones, James Earl","Guinness, !! ! ! ! Alec",...] } },!

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Bootstrapping Data

Source System

Processing Script

Queuing Batching

Amazon EC2

Amazon EC2

Amazon CloudSearch

Amazon SQS

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Configuring for Search

!   Text fields for individual word search •  User-generated and external text – titles, descriptions

!   Literal fields for exact matches •  Application-generated text like facets

!   Integer fields for range searching and ranking

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Sending Queries http(s)://<endpoint>/2011-02-01/search? !   Simple searches

•  q=<text> !   Filtering

•  bq= (and title:'iron man' genre:'Action') !   Filtering with integer ranges

•  bq=(and 'iron man' year:..2010) !   Geo filtering

•  bq=(and 'iron man' latitude:12700..12900 longitude:5700..5800)

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Search Results { "rank": "-text_relevance",! "match-expr": "(label 'star wars')",! "hits": { "found": 7, "start": 0,! "hit": [{"id": "tt1185834"},! {"id": "tt0076759"},! {"id": "tt0086190"},! {"id": "tt0120915"},! {"id": "tt0121765"},! {"id": "tt0080684"},! {"id": "tt0121766"} ]! } ...!}!!

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Updating CloudSearch

Amazon EC2 Amazon CloudSearch

Amazon SQS Amazon EC2

Amazon S3 DynamoDB Amazon RDS

Web Server

Users

Update Processor

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

BASIC RANKING

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Customizing Ranking

! text_relevance and cs.text_relevance !   Rank expressions

•  Compute a score for each document •  &rank=<function>

!   Defined in the console !   Defined at query-time

•  &q='iron-man'&rank-recency=text_relevance + year &rank=recency

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Document Structure

Movie

title

description

user_rating

likes

release_date

latitude

longitude

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Field Weighting

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Field Weighting

!   Adjust relative importance of fields !   &rank-title_boost=

cs.text_relevance({"weights":{"title":4.0}, "default_weight":1})

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Popularity

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Popularity

!   Convert floating point to integer !   Weight by the number of ranks !   rank-pop=

(user-rating - 2) * log10(number-user-ranks) * 10 + metascore * 3

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Freshness

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Freshness

!   Exponential decay function

!   &rank-decay= 200*Math.exp(-0.1*days_ago)

r = ce−λt

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Rank Expressions: Combined

!   &rank-combined=1.0 * title + 0.5 * popularity + 0.3 * freshness

!   &rank=combined

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

LOCATION-BASED SEARCH

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil. !

Iron Man (2008)!

Tony Stark has declared himself Iron Man and installed world peace... or so he thinks. He soon realizes that not only is there a mad man...!

Iron Man 2 (2010)!

When Tony Stark's world is torn apart by a formidable terrorist called the Mandarin, he starts an odyssey of rebuilding and retribution. !

Iron Man 3 (2013)!

On the hunt for a fabled treasure of gold, a band of warriors, assassins, and a rogue British soldier descend upon a village in feudal China, where a humble blacksmith...!

The Man With The Iron Fists (2012) !

Cancel Iron Man!

Movies Search Social Account Nearby

Done Iron Man

!

Movies Search Social Account Nearby

Mobile Experience

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Encoding Location !   Latitude and longitude expressed as

integers Movie

title

description

user_rating

likes

release_date

latitude

longitude

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Bounding Box Search

!  Latitude min/max !  Longitude min/max bq=(and 'theater' latitude:12700..12900 longitude:5700..5800)

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Location Sort

!   Cartesian distance function

!   &rank-geo=sqrt(pow(latitude - lat, 2) + pow(longitude - lon, 2)

!   &rank=-geo

(lat − latuser )2 + (lon− lonuser )

2

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

FACETING

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Facets

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Facets

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Simple Faceting: Document

Movie

title

description

genre

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Simple Faceting: Configuration

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Simple Faceting: Query q=iron+man&facet=genre

{"rank":  "-­‐text_relevance",  "match-­‐expr":  "(label  'star  wars')",  "hits":  {"found":  7,  "start":  0,  "hit":  []  },  "facets":  {      "genre":  {          "constraints":  [              {"value":  "Family",  "count":  62},              {"value":  "Action/Adventure",  "count":  21},              {"value":  "Drama",  "count":  5  },  

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Simple Faceting: UI <div  class='facet'>          <ul  class='facet_list'>                  <?php                          $genres  =  $resultsObj-­‐>facets-­‐>genre-­‐>constraints;                          for  ($i  =  0;  $i  <  count($genres);  $i++)  {                                  $curGenre  =  $genres[$i];  $curCount  =  $thisGenre-­‐>count;                    ?>                  <li  class='facet_item'>                          <div  class='facet_name'><?=$curGenre?></div>                          <div  class='facet_count'><?=$curCount?></div>                  </li>                  <?php  }  ?>          </ul>  </div>  

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Facets

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Document !   title: Lincoln !   description: ... !   oscar1: Awards !   oscar2: Awards/Best Actor !   oscar3: Awards/Best Actor/Daniel Day

Lewis

Movie title description oscar1 oscar2 oscar3

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Query &q=lincoln&facet=oscar1,oscar2,oscar3 {"rank":  "-­‐text_relevance",  "hits":{...},  "facets":  {      "oscar1":  {          "constraints":  [              {"value":  "Awards",  "count":  23},              {"value":  "Nominations",  "count":  124}]},      "oscar2":  {          "constraints":  [              {"value":  "Awards/Best  Actor",  "count":  6},              {"value":  "Awards/Best  Actress",  "count":  3}...]},            "oscar3":  {          "constraints":  [              {"value":  "Awards/Best  Actor/Daniel  Day  Lewis",  "count":  1},              {"value":  "Awards/Best  Actor/Denzel  Washington",  "count":  2}...]},        

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Drilldown

! bq=oscar1:'Awards' ! bq=oscar2:'Awards/Best Actor' ! bq=oscar3:'Awards/Best Actor/Daniel Day Lewis' ! bq=(and 'star' oscar2:'Awards/Best Actor')

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

MIXED DATA SOURCES

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Document

Showtime

type

title

theater_name

city

latitude

longitude

Movie

type

title

description

user_rating

likes

release_date

Review

type

title

movie_name

author

url

body

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Heterogeneous Data

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Multi Domain

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Trade-offs

!   Multiple domain •  Independent configuration •  Independent scale

!   Single domain •  Simpler •  Lower cost •  bq=(and 'iron man' type:'movie')

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

TUNING

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

What to Track

!   User queries !   Responses !   Response times !   Click positions

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Tuning Relevance

!   Return relevance values !   Check no-result queries !   Check most common results

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Tuning Performance

!   Identify consistent slow queries !   Tend towards text matching !   Cache slow queries when possible !   Benchmark with JMeter or Siege

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Q&A

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Resources

!   Amazon CloudSearch Overview Page http://aws.amazon.com/cloudsearch/ •  Developer Guide •  FAQs, Articles •  Community Forum •  Tutorial

!   Free 30-day trial !   Contact: handler@amazon.com