Managing Search Relevance in SharePoint 2013 and O365Mikael SvensonPrincipal ConsultantPuzzlepart

SPC382

David HollembaekSenior ConsultantMicrosoft

Agenda

The Problem

The Plan

The Tools

The Problem

Is this a good search result?

The Problem

Sure, if you are Alex

Marketing AssistantSales & Marketing

The Problem

Not so much, if you are Fabrice

AttorneyLegal

The Problem

Match Content to Query Term to Term Easy!

Reflect intent in the process My Term to the Content Owner’s Term Hard!

The Relevancy Problem

The Problem

What is your role in the organization? What URL are you at when searching? What language are you searching in? What time of day? What’s your location? Are you on a PC, laptop or mobile device?

Intent

The Problem

Conceptual Query with Context“las vegas”

(<100>las vegas) OR (<50>"sin city" OR "entertainment capitol" OR "marriage capitol") OR (<30> gambling)IN CONTEXT(<30>location=@Home AND <50>device=smartphone AND <20>freshness=30days AND <50>hobby=SHAREPOINT)

The Problem

The Plan

Query Content Ranking User Interface

What can we Influence?

The Plan

XRANK Poll

The Plan

What is Ideal? Create the Gold Set

Set Goals

The Plan

Relevancy Cycle Frequent at first Infrequent after

The Plan

Legal Repository Active Litigation Research Library

Fabrice in Legal - Authoritative Content

The Plan

The Tools

Fixing the Problem: New Tools

Query Rules: Reformulate the query

Dynamic Ranking Rules: Reorder the Results

Custom Rank Model: Build relevance from “scratch”

Rank Model Tuning App: Incrementally improve rank models

Con

trol

Sim

plic

ity

The Tools

Fabrice and the XT1000…

AttorneyLegal

The Tools

Fabrice in Legal Legal Repository Active Litigation Research Library

Quick “No‘s“ No Content Control No Thesaurus

The Tools

Capturing Intent – Query ModifiersWhat can change a query?

User.<property> Site/SiteCollection Today

path:{Site}author:{User.Name}webid:{Site.ID}property:{Page.Owner}

The Tools

What can trigger a rule based on intent?

Action terms “More likely” queries

Capturing Intent - Triggers

The Tools

Result BlocksThe Tools in Practice

Trigger on Intent Demo 1

• Show result block for blog posts• Add a custom “show more” link with

action term• Use action term “blog” to show only

blog posts

Create Result Source for blog posts

Create catch-all rule to show blog post block

Use action term to capture intent and only show blog posts

Demo 1 – recap

The Tools

Query RulesThe Tools in Practice

Query Rules: conditions and actionsDICTIONARY-BASED MATCHES• Exact match, beginning or end• Ad-hoc or term store dictionary• Match a regex (advanced)

HISTORICAL (LOG-BASED) MATCHES• Is this query more likely aimed at

the following source…?• Do people mostly click on result

of the following type…?

Query Conditions

CURATED (BEST BET)• Show a promoted result

REFORMULATE THE USER QUERY• Show a block of results• Replace the core results

with a different query

Actions

The Tools

What can trigger a rule based on context?

User Segment Categories (cross site publishing and

catalogs)

Capturing Context - Triggers

The Tools

Trigger on Context Demo 2

• Federate against legal repository if you work in Legal

• Use Department property to match legal intent

• Create result block with legal data• Add more link which goes to the federated

source

Create federated result source against legal repository

Create rule to capture intent based on department

Show legal data block based on Department

“More link” take you to the data repository

Demo 2 – recap

The Tools

VerticalsThe Tools in Practice

Might be the physical ones Often they are mixed

Find the mental information silos

The Tools

Verticals are separate pages

The Tools

“More Common” triggers Demo 3

• Find search term used on multiple verticals• Create a “Query more common in source”

rule• Show a video result block for people vertical• Show blogs for a common term

Find terms in log which is more common in one vertical

Show video results on people vertical on term more common in source

Show blog posts when posts are commonly clicked for a term

Demo 3 – recap

The Tools

Tuning – XRANKThe Tools in Practice

XRANK formula

Normalized boost, nb Promote or demote a

particular item Promotion with

builder uses stdb and cb.

The Tools

Promotion and Demotion

SharePoint 2013 Query ToolCodeplexCame out of MCSUses REST API

SharePoint 2013 Query Tool v2Rank DetailShow All Properties*Live MonitoringInline PreviewAnonymous LoginPick Result SourceFreshness Boost

Re-order with query ruleDemo 4• Use Query Builder v2 to see why an

item is ranked above another• Add dynamic ordering to move the item

higher with XRANK freshness boost

Hit 1 (pptx) Hit 5 (docx)

The Tools

Examine rank values using Search Query Tool v2

Create rule to re-rank results based on freshness

Non-trivial to see why a document is ranked higher

Demo 4 – recap

The Tools

Tuning – Rank modelThe Tools in Practice

You can Tune a Custom Ranking Model After…

You’ve considered or tried simpler optionsAuthorities, Thesaurus, Query Rules, Dynamic Rules

You’ve seen ranking could be improved for most queries

A meta-data property should be incorporated into ranking schema, eg: user rating

You’ve configured new managed properties and recrawled:

• Add a new managed property (integers: sortable/refinable, text: searchable)

• Recrawl

Rank Algorithm BM25F Ranks items

based on the appearance of query terms in the full-text index.

Input any managed properties in your full-text index. The Tools

Title Author Clicks File Type Authoritative sites - level

Default rank model features

The Tools

Tuning – AuthoritiesThe Tools in Practice

Authorities: SSA-level configuration

Sites that are important

Sites with low intrinsic relevance

Takes ~24hrs to propagate

Authorities: Connected

1

0

1

1

2

4

3

2

∞

4

Setting an authority affects all sites connected through hyperlinks

Sites are weighted

by distance to the authority

Tuning tool Demo 5

• In the Office Store• Rank Word documents

higher using the Rank Model Tuning

Create a new rank model based on existing

Add judgments for auto tune and feature selection

Change weight for docx/pptx – managed property boost

Demo 5 – recap

The Tools

Identify the Problem Plan your cycles with the appropriate tools

Match for Intent Capture Context Keep each cycle simple and manageable

Use the right tools Query Rules Result Blocks Verticals Dynamic Ranking Rank Model Tuning

Summary

Turn business rules into Query Rules!

See you at the Search booths & Search tables at Ask the Experts WED @6:15pm!

Session Session Room Time

Develop Advanced Search-Driven SharePoint 2013 Apps SPC402 Palazzo I, J Tue 1:45pm

Best practices for Hybrid Search deployments SPC306 Veronese 2401 Tue 5:00pm

SharePoint 2013 Search Analytics SPC340 Palazzo M, N Wed 9:00am

How to manage and troubleshoot Search: A practical guide SPC375 Veronese 2401Wed 10:45am

6 Proven Steps to Get the Best Out of Search in SharePoint 2013 SPC265 Delphino 4001 Wed 1:45pm

Best practices for Information Architecture and Enterprise Search SPC207 Veronese 2401 Wed 1:45pm

Search content enrichment and extensibility in SharePoint 2013 SCP414 Palazzo K, L Wed 1:45pm

Customizing Search experiences with Azure Hosted Data and Bing Maps SPC321 Veronese 2401 Wed 3:15pm

Futuristic Search applications using Kinect and Yammer! SPC405 Palazzo M, N Wed 3:15pm

Search architecture and sizing in SharePoint 2013 SPC336 Titian 2201 Wed 5:00pm

Effective Search deployment and operations in SharePoint 2013 SPC360 Veronese 2401 Thu 9:00am

SharePoint 2013 Search display templates and query rules SPC322 Palazzo M, N Thu 9:00am

Managing Search Relevance in SharePoint 2013 and O365 SPC382 Veronese 2401 Thu 12:00pm

Searc

h R

ela

ted S

ess

ion

s

MySPCSponsored by

connect. reimagine. transform.

Evaluate sessionson MySPC using yourlaptop or mobile device:myspc.sharepointconference.com

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.