Integrated Search for BetterUser Experience

Google Search Appliance Google Search AJAX/REST API

GCS Disadvantages

• Crawling frequency• Limited number of search requests• Vulnerability (XSS)• Result set always smaller than delivered by global Google• Multiple front-ends (UNL, IANR, CASNR)• Poor handling of password-protected documents• Unknown life-span• Meta tags (Cannot sort by date, prod. ID, etc.)

GSA Setup• Define your collection(s)• Decide what pages to crawl/exclude (+robots.txt, meta)• Decide on front-end (XSLT or otherwise)• Arrange underlying stylesheet transform code (global styles, page components, sort by date/relevance, etc.)• Build the wrapper• Define new global CSS classes

Adding Search to Your Web Site• As a text box or HTML anchor• As a form/anchor in your pages/templates• Create and name targeted search page• Add wrapper call to the search page <script> a = location.hostname + location.pathname; document.write("<scr" + "ipt type=\"text/javascript\" + src=\"http://citsearch.unl.edu/cgi-bin/gsa_search.cgi" + location.search + "&domacin=" + a + "\"></scr" + "ipt>"); </script>

Wrapper in Action• Intercepts requests/passes them to GSA• Parses the fetched result set page

- Strips off heading part (primarily: GSA CSS class definitions)- Adds info pages links- Translates cursor/sorting /submit button links- Adds ‘domain’ links (images), preserving parameters- Adds multimedia indications; improved search suggestions- Adds customized CSS global definitions

• Delivers results to the calling page (document.write())

CSS Class Names<style type="text/css"> .gs_title { font-family: … } .gs_info { color: … } .gs_snippet { font-family: … } .gs_cache { color: … } .gs_sort { font-family: … } .gs_cursorHit, .gs_i { color: … } .gs_cursorLive { text-decoration: none; … } </style>

• Prefix ‘gs_’ guarantees unique CSS class names

CSS Examples• Check the source code of the following web sites with

characteristic solutions:

marketjournal.unl.edu (resetting undesirable styles)byf.unl.edu (using UNL templates)cropwatch.unl.edu (search w/o text box; site using frames)beef.unl.edu (minimal CSS classes modifications)

AJAX• Asynchronous JavaScript + XML• A collection of web dev. technologies:

– HTML and CSS– DOM accessed with JavaScript– Browser/server asynchronous comm. method: XMLHttpRequest, Iframe, or dynamically added <script> tag– Data formats: XML, plain text, preformatted HTML, JSON data structures

(JSON, computer data interchange format)

• A framework used to build dynamic pages on the client side• Google AJAX search API is a JavaScript API

REST API• Representational State Transfer (REST) – a method for building web services• Used to access AJAX interface from non-JavaScript environment (Perl in our

case)• Based on HTTP and REST principles:

– The URI for the web service– MIME type supported by the web service– Operations supported by the web service (GET, POST,PUT,DELETE)

• Request’s final form, example:http://ajax.googleapis.com/ajax/services/search/web?hl=fr&q=Harry+Potter+site:amazon.com&v=1.0&start=0

• Result set returned in JSON format

JASON Object{"responseData": {"results":[{"GsearchResultClass":"GwebSearch","unescapedUrl":"http://www.amazon.com/Harry-Potter

Paperback-Boxed-Books/dp/0439682584","url":"http://www.amazon.com/Harry-Potter-Paperback-Boxed-Books/dp/0439682584","visibleUrl":"www.amazon.com","cacheUrl":"http://www.google.com/search?q\u003dcache:qOjqIJIQsoAJ:www.amazon.com","title":"Amazon.com: \u003cb\u003eHarry Potter\u003c/b\u003e Paperback Boxed Set (Books 1-5): J. K. \u003cb\u003e...\u003c/b\u003e","titleNoFormatting":"Amazon.com: Harry Potter Paperback Boxed Set (Books 1-5): J. K. and the Deathly Hallows (Book 7): J. K. \u003cb\u003e...\u003c/b\u003e","titleNoFormatting":"Amazon.com: Harry Potter and the Deathly Hallows (Book 7): J. K. ...","content":"\u003cb\u003eHarry Potter\u003c/b\u003e and the Half-Blood Prince (Book 6) by J.K. Rowling

: : content":"As his fifth year at Hogwarts School of Witchcraft and Wizardry approaches, 15- year-old \u003cb\u003eHarry

Potter\u003c/b\u003e is in full-blown adolescence, complete with regular \u003cb\u003e...\u003c/b\u003e"}],"cursor":{"pages":[{"start":"0","label":1},{"start":"4","label":2},{"start":"8","label":3},{"start":"12","label":4},{"start":"16","label":5},{"start":"20","label":6},{"start":"24","label":7},{"start":"28","label":8}],"estimatedResultCount":"3580000","currentPageIndex":0,"moreResultsUrl":"http://www.google.com/search?oe\u003dutf8\u0026ie\u003dutf8\u0026source\u003duds\u0026start\u003d0\u0026hl\u003dfr\u0026q\u003dHarry+Potter+site%3Aamazon.com"}}, "responseDetails": null, "responseStatus": 200}

Back to the Wrapper• Connects to AJAX API, using Google::Search::Rest perl module• Passes parameters and API key• Fetches and traverses the result set• Composes the page from scratch:

– Adds search box and domain logos– Forms document descriptions– Constructs the cursor– Adds CSS classes– Delivers the page as a JavaScript document.write() method