opencms days 2015 advanced solr searching

Daniel Seidel – Alkacon Software GmbH

Advanced searching

29.09.2015

2

What can you expect?

Presentation of new

OpenCms features

A tutorial on advanced Solr

features

3

A short history on searching

OpenCms 6.0

● Simple server-side search

● Many uni-lingual indexes

● Many proprietary configuration options

OpenCms 8.5

● Many more features ● Facets

● Highlighting

● Did you mean?

● …

● Multilingual indexes

● Support for client-side searches ?

4

Building a search page - Lucene

How to

● Configure your index (one per language)

● Write JSP using CmsSearch as bean

Pros and Cons

Easily produce a search page Easy query options, incl. simple filter and sort options

Pagination support

Scriptlet code necessary

All server-side code

Index configuration necessary (for each language)

No support for facets etc.

5

Building a search page - Solr

How to

● Write a custom client side search (using GWT)

● Adjust the demo search (may include altering GWT code)

Pros and Cons

Great search experience possible

All Ajax based

Many great features: facets, auto-completion, “Did you mean?”

“unlimited” sorting options

Relies heavily on JavaScript

Very complex to build and adjust

State keeping

Html rendering with JavaScript

Nearly always a per-customer solution

6

Combining the Advantages

Simplicity

since 6.0

Many features

since 8.5

● Stay on the server side (JSP)

● Class like CmsSearch, but

● Expose more Solr features

● Usable without scriptlet code

Design questions 7

What is the value of CmsSearch?

Simple search API tailored for OpenCms

Managing state

Hiding search details

Mapping from request parameters to

search options

Design questions 8

What can we support for Solr?

Simple search API tailored for OpenCms

Managing state

Mapping from request parameters to

search options

Hiding all search details

Design questions 9

How can we ease usage?

Reduce configuration to a minimal

Avoid explicit bean initialization

Allow to add raw Solr query parts

Easy access to results, state, configuration

(no scriptlet code necessary)

● Via search you obtain

● Configuration

● State (current query, checked facet entries, ...)

● Results (search results, facet items, ...)

● Via configFile a special XML content is given

● You drag search form configurations on the page

to render a fully functional search form

● A “feature-complete” default formatter is provided

10

The new tag: <cms:search>

<cms:search var="search"

configFile="${content.filename}"/>

● Live Demo

Drag & Drop your search form

Demo

Demo Demo Demo

デモ

Search form creation

● Request parameters

● In cases where the default causes collisions

● In cases where additional parameters should be

handled

● General search options

● Core / Index

● Search for empty query?

● Escape query string?

● Query string modifier

● Add additional query parameters

12

What can I configure? (I)

● Special search options

● Pagination

● Sorting

● Facets (field and query facets)

● Highlighting

● Did you mean?

13

What can I configure? (II)

Important

You can, but need not necessarily

configure everything!

I_CmsSearchResultWrapper

I_CmsSearchControllerMain

Single controllers with

configuration and state

Result information

Results list

Facets result

Highlighting result

Page info

...

I_CmsSearchStateParameters

14

Structure of the result object

Configure the UI

elements

(input field names

and current

state/value)

Print results,

facet items,

etc.

Create a suitable link

that contains the

complete state of the

search form

<c:set var="controllers" value="${search.controller}"/>



<c:set var="sort" value="${controllers.sorting}"/>



<select name="${sort.config.sortParam}">

<c:forEach var="option"

items="${sort.config.sortOptions}">

<option value="${option.paramValue}"

${sort.state.checkSelected[option]?"selected":""}>

${option.label}

</option>

</c:forEach>

</select>



15

Rendering example: Sort options

${option.label}

</option>

</c:forEach>

</select>



16


Use abbreviations

${option.label}

</option>

</c:forEach>

</select>



17


Use the configuration

${option.label}

</option>

</c:forEach>

</select>



18


Use the state

<c:forEach var="searchResult" items="${search.searchResults}">

<a href='<cms:link>${searchResult.fields["path"]}</cms:link>'>

${searchResult.fields["Title_prop"]}

</a>

<p>

${cms:trimToSize(

fn:escapeXml(searchResult.fields["content_en"])

,250)}

</p>

<hr />

</c:forEach>



19

Rendering example: Results

</a>

<p>

${cms:trimToSize(


,250)}

</p>

<hr />

</c:forEach>



20


Loop over the result collection

(get I_CmsSearchResourceBean objects)

</a>

<p>

${cms:trimToSize(


,250)}

</p>

<hr />

</c:forEach>



21


Access index fields

<c:set var="fControllers"

value="${controllers.fieldFacets}"/>

<c:forEach var="facet" items="${search.fieldFacets}">

<c:set var="facetController"

value="${fControllers.fieldFacetController[facet.name]}"/>

<c:if test="${cms:getListSize(facet.values) > 0}">



</c:if>

</c:forEach>



22

Rendering example: Facets (I)

</c:if>

</c:forEach>



23


Loop over facets from the

search result

</c:if>

</c:forEach>



24


Get the facet’s controller

</c:if>

</c:forEach>



25


Render the facet if necessary

(see next slide)

<div>${facetController.config.label}</div>

<c:forEach var="facetItem" items="${facet.values}">

<div class="checkbox">

<label>

<input type="checkbox"

name="${facetController.config.paramKey}"

value="${facetItem.name}"

${facetController.state.isChecked[facetItem.name]

?"checked":""} />

${facetItem.name} (${facetItem.count})

</label>

</div>

</c:forEach>



26

Rendering example: Facets (II)

<label>





?"checked":""} />


</label>

</div>

</c:forEach>



27


Use the configuration

<label>





?"checked":""} />


</label>

</div>

</c:forEach>



28


Use the result

<label>





?"checked":""} />


</label>

</div>

</c:forEach>



29


Use the state

<c:set var="pagination" value="${controllers.pagination}"/>

<c:forEach var="i" begin="${search.pageNavFirst}"

end="${search.pageNavLast}">

<c:set var="is">${i}</c:set>

<li ${pagination.state.currentPage eq i ? "class='active'" : ""}>

<a href="<cms:link>${cms.requestContext.uri}

?${search.stateParameters.setPage[is]}

</cms:link>">${is}</a>

</li>

</c:forEach>

<li>

<c:set var="pages">${search.numPages}</c:set>


?${search.stateParameters.setPage[pages]}

</cms:link>">Last</a>

</li>

30

Rendering example: Pagination









</li>

</c:forEach>

<li>





</li>

31


Use pagination support









</li>

</c:forEach>

<li>





</li>

32


Use current state









</li>

</c:forEach>

<li>





</li>

33


Use state parameters

● Two options to send the current state:

● Submit a form

● Use state parameters

● With state parameters

● Everything is manipulated explicitly

● You do not need a form at all

● With a form

● You must make sure to send all the state you want

(use hidden parameters, e.g., for the last query)

● You can intentionally prevent state transmission

(e.g., for the current page)

34

Form vs. state parameters

● We did

● Introduce the tag <cms:search>

● See its configuration via an XML content

● Explore how to write a formatter for a search form

● If you need more information on formatting

● Explore the default formatter

● Explore the classes under org.opencms.jsp.search.result

● We did not

● Cover all configuration options

● See all use cases for <cms:search>

35

Short summary

● Intention

● Show blogs, news, ...

● Link to detail pages

● Realization

● <cms:contentload>

● Manifold collectors

● Question

● Where’s the

difference between

lists and a search

result page?

36

Lists in OpenCms

● Intention

● Show blogs, news, ...

● Link to detail pages

● Realization

● <cms:contentload>

● Manifold collectors

● Question

● Where’s the

difference between

lists and a search

result page?

37

Lists in OpenCms

Essentially, there is no

difference!

38

A short history of lists

● Simple way to render lists

● Many specific collectors

● Facility to edit list entries directly

OpenCms 6.0 <cms:contentload> is introduced

OpenCms 8.5

● Collecting “byQuery” or “byContext”

● First contact of Search and Lists

● No facets etc. supported

CmsSolrCollector is added

OpenCms 10 <cms:search> and <cms:edit> are introduced

● Lists and search become one

● Facets etc. available for lists

● Editing becomes decoupled from lists ?

● What is specific for a list?

● XML content for configuration is unsuitable

● There is no query string provided by the user

● Usually there is no form

● XML contents, not the index is accessed

● How does <cms:search> handle the pecularities

● Configuration via JSON (String or file)

● An option to ignore the query parameter

● The already seen state parameters

(no form is necessary)

● I_CmsSearchResourceBean allows to access XML

contents via the CmsJspContentAccessBean

39

Lists with <cms:search>

● Live Demo

Watch out for the Solr features

Demo

Demo Demo Demo

デモ

A list with <cms:search>

<c:set var="searchConfig">

{ "ignorequery" : true,

"extrasolrparams" :

"fq=type:u-blog&fq=parent-folders:

\"/sites/default/.content/blogentries/\"",

"pagesize" : ${content.value.pageSize},

"pagenavlength" : ${cms.element.settings["navlength"]}



}

</c:set>

<cms:search configString="${searchConfig}" var="search"/>



41

The JSON configuration for a list

"extrasolrparams" :






}

</c:set>




42


Most important options

"extrasolrparams" :






}

</c:set>




43


Optional extra features

"extrasolrparams" :






}

</c:set>




44


The content adjusts the configuration

<c:choose>

<c:when test="${search.numFound > 0}">

<c:forEach var="result" items="${search.searchResults}">

<c:set var="content" value="${result.xmlContent}"/>



</c:forEach>

</c:when>

<c:otherwise>

No results found.

</c:otherwise>

</c:choose>

45

Rendering the list entries

Easier than before

Getting results and iterating over

them is separated

No nested <cms:contentload>

anymore

● <cms:contentload> supported

● Editing, adding or deleting list entries

● Adding entries to an empty list

● Can we provide the same feature?

● No, because iteration is separated from <cms:search>

● Do we need the same feature?

● No, there’s no reason why edit options and content

loading has to be coupled

● Can we get similar edit options differently?

● Yes, a special <cms:edit> tag

46

Editing list entries

● Places edit points wherever you like

● Allows to edit or delete existing XML contents

● Allows to add new contents

● Highlights the enclosed HTML when hovering

over the edit point

● New contents can be placed

● At the place configured in the sitemap configuration

● In a provided folder

● Where the content that is edited is located

47

The tag <cms:edit>

<c:choose>

<c:when test="${search.numFound > 0}">

<c:forEach var="result" items="${search.searchResults}">

<c:set var="content" value="${result.xmlContent}"/>

<cms:edit uuid='${result.fields["id"]}'

create="true" delete="true">



</cms:edit>

</c:forEach>

</c:when>

<c:otherwise>

<cms:edit createType="u-blog">

<div>No results found.</div>

</cms:edit>

</c:otherwise>

</c:choose>

48

Edit options for the list

Easier than before

Harmonized usage for empty

and non-empty list

● How about the “Publish this page” option?

● <cms:contentload> with attribute “editable” causes list entries to be related to the page

● Do we have a similar mechanism?

● Using <cms:edit> the edited content becomes related to the page

● But does this suffice?

● Not if we use pagination.

● So can we do better?

● Yes: <cms:search ... addContentInfo="true">

● All list entries become related to the page

49

Do we still miss a feature?

50

What should you remember?

Server-sided Solr search becomes easy

Search pages and lists become the same

Edit point needed: Use <cms:edit>

By, by <cms:contentload> and friends

● Containers in lists

● Container tag takes the element(s)

<cms:container ... elements="${uuid}"/>

● Rendering is done by a suitable formatter

51

What’s next? (Maybe)

Easily render contents of different types

Inline editing will work out of the box

Harmonized rendering of XML contents

● Containers in lists

● Container tag takes the element(s)

<cms:container ... elements="${uuid}"/>

● Rendering is done by a suitable formatter

52

What’s next? (Maybe)

Easily render contents of different types

Inline editing will work out of the box

Harmonized rendering of XML contents

53

Last comments

Solr updated to version 5.1

(further updates follow)

Gallery index replaced by “Solr offline”

(watch out for bugs)

This is OpenCms 10 Alpha 1

(don’t expect everything in the final state)

Daniel Seidel

Alkacon Software GmbH

http://www.alkacon.com

http://www.opencms.org

Thank you very much for your attention!