svc101 building search into your app - aws re: invent 2012
DESCRIPTION
Amazon CloudSearch is a fully-managed search service in the cloud that allows customers to easily integrate fast and highly scalable search functionality into their applications. In this session, we cover the basics of search and search engines. We take an introductory look at CloudSearch along with a deep dive showing how to build a CloudSearch-based web application.TRANSCRIPT
![Page 1: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/1.jpg)
![Page 2: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/2.jpg)
![Page 3: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/3.jpg)
Search experience = user retention and revenue
![Page 4: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/4.jpg)
![Page 5: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/5.jpg)
![Page 6: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/6.jpg)
![Page 7: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/7.jpg)
DNS / Load Balancing AWS Query
Search API Console Config
API
Command
Line Tools Console Doc
Svc API
Command
Line Tools Console
SEARCH SERVICE Search Documents
DOCUMENT SERVICE Add Documents
Update Documents
Delete Documents
Create Domains
Configure Domains
Delete Domains
CONFIG SERVICE
ACCESS CONTROL ACCESS CONTROL ACCESS CONTROL
Search Domain
![Page 8: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/8.jpg)
SEARCH INSTANCE Index Partition n
Copy 1
SEARCH INSTANCE Index Partition 2
Copy 2
SEARCH INSTANCE Index Partition n
Copy 2
SEARCH INSTANCE Index Partition 2
Copy n
SEARCH INSTANCE
DATA Document Quantity and Size
TRAFFIC Search Request Volume and Complexity
Index Partition n Copy n
SEARCH INSTANCE Index Partition 1
Copy 1
SEARCH INSTANCE Index Partition 2
Copy 1
SEARCH INSTANCE Index Partition 1
Copy 2
SEARCH INSTANCE Index Partition 1
Copy n
![Page 9: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/9.jpg)
![Page 10: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/10.jpg)
• The Challenge
• The Data: The Million Song Data Set
http://labrosa.ee.columbia.edu/millionsong/
• The Application
![Page 11: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/11.jpg)
Field name Description
artist_mbid The musicbrainz.org ID
artist_name Name of the artist
audio_md5 Hash code of the audio
danceability According to The Echo Nest
duration In seconds
loudness General loudness of the track
song_hottnesss According to Echo Nest
title Song title
year Song year
![Page 12: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/12.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 13: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/13.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 14: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/14.jpg)
![Page 15: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/15.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 16: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/16.jpg)
![Page 17: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/17.jpg)
Artist Name
Song Title
Familiarity
Year
![Page 18: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/18.jpg)
Genre
Artist
Year
![Page 19: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/19.jpg)
Title
Artist Name
Genre
Artist Familiarity
Year
![Page 20: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/20.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Prepare and upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 21: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/21.jpg)
![Page 22: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/22.jpg)
Million Song
DataSet
SDF
Batches
Amazon
CloudSearch
![Page 23: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/23.jpg)
SDF Batches [
{"type":"add",
"id": "soaczam12ab0181559",
"version":5,
"lang":"en",
"fields": {
"title":"Ruby Tuesday",
"artist_name":"The Rolling Stones",
"year":"1967",
"artist_familiarity":864830,
"genre":["alternative", "ambient", "dance",
"electronic", "pop", "r&b", "reggae"]
}
},
… ]
![Page 24: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/24.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Prepare and upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 25: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/25.jpg)
Text fields for
matching user terms
Result enabled to
retrieve source data
![Page 26: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/26.jpg)
Literal fields for
Faceting
Facet enabled to
retrieve facet counts
Search enabled for
narrowing
![Page 27: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/27.jpg)
Integer fields for
ranking, narrowing
![Page 28: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/28.jpg)
![Page 29: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/29.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Prepare and upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 30: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/30.jpg)
![Page 31: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/31.jpg)
![Page 32: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/32.jpg)
![Page 33: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/33.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Prepare and upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 34: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/34.jpg)
PHP Integration $results =
file_get_contents(
http://search-mn-songs-5bbplyghbb5tk257rsb7iamlsy." .
"us-east-1.cloudsearch.amazonaws.com" .
"/2011-02-01/search?q=" . $keyword .
"&return-fields=title,artist_name,year&" .
"facet=artist_name,year_facet,genre&" .
"rank=-" . $rank);
$resultsObj = json_decode($results);
![Page 35: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/35.jpg)
Simple Search Result
{"rank": "-text_relevance",
"match-expr": "(label 'rolling stone')",
"hits": { "found": 204, "start": 0,
"hit": [ { "id": "sontsst12cf5f88b42" },
{ "id": "sopvopr12ab017f082" },
{ "id": "sorzrpw12ac468a13b" },
] },
...
}
![Page 36: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/36.jpg)
Search Results With Return Values
"hit":
[ { "id": "sontsst12cf5f88b42",
"data": {
"artist_familiarity": [ "925048" ],
"artist_name": [ "The Rolling Stones" ],
"text_relevance": [ "326" ],
"title": [ "Heart Of Stone" ],
"year": [ "1964" ]
}
},
![Page 37: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/37.jpg)
Facets In Search Results
{…"hits": { … },
"facets": {
"genre": {
"constraints": [
{ "value": "pop", "count": 126 },
{ "value": "rock", "count": 125 },
{ "value": "alternative", "count": 109 },
{ "value": "electronic", "count": 106 },
{ "value": "jazz", "count": 58 }, ...
] } }
![Page 38: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/38.jpg)
X
X
![Page 39: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/39.jpg)
• Create an Amazon CloudSearch domain
• Identify use case and supporting data
• Prepare and upload data
• Configure the domain
• Improve document ranking
• Integrate with the front end
• Keep documents up-to-date
![Page 40: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/40.jpg)
![Page 41: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/41.jpg)
![Page 42: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/42.jpg)
![Page 43: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/43.jpg)
![Page 44: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/44.jpg)
![Page 45: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/45.jpg)
![Page 46: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/46.jpg)
![Page 47: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/47.jpg)
26ms
![Page 48: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/48.jpg)
![Page 49: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/49.jpg)
![Page 50: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/50.jpg)
https://github.com/pbs/haystack-cloudsearch
![Page 51: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/51.jpg)
![Page 52: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/52.jpg)
![Page 53: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/53.jpg)
Get Started Now, Free Trial
![Page 54: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/54.jpg)
We are sincerely eager to
hear your feedback on this
presentation and on re:Invent.
Please fill out an evaluation
form when you have a
chance.
![Page 55: SVC101 Building Search into Your App - AWS re: Invent 2012](https://reader034.vdocuments.net/reader034/viewer/2022051411/5453fd0eb1af9f8c228b48b6/html5/thumbnails/55.jpg)