content recommendation with sharepoint search

Content Recommendation with SharePoint SearchAndré Vala

SHAREPOINT SOLUTIONS ARCHITECT

/472

André ValaABOUT ME

BLOG

LINKED IN

TWITTER

EMAIL

MEMBER OF

MEMBER OF

WORKS FOR

andre.vala@create.pt

@atomicvee

http://blogit.create.pt/andrevala

http://www.linkedin.com/in/andrevala

SharePoint Solutions Architect

JOB TITLE

Office365 PT

SPUG PT

/473

Agenda• Content Recommendation• How does it work?• Reporting• Customization• Anonymous Users

/474

Content Recommendation

/475

Content RecommendationWhat?• Providing relevant contextual content to users• “People who viewed this also viewed that”• “These are the most popular items”

Why?• Enhanced user experience• Improved conversion (on e-commerce sites)

How?SharePoint 2013 includes two out-of-the-box webparts• Recommended Items web part• Popular Items web part

/476

Recommended Items Web Part• Out-of-the-box Web Part• Similar to a Content Search Web Part• Shows list of recommended items

related to the current content (“users who viewed this item also viewed these items”)

CONTENT RECOMMENDATION

/477

Popular Items Web Part• Out-of-the-box Web Part• Similar to a Content Search Web Part• Shows most viewed items• Can be filtered (by category, for example)

CONTENT RECOMMENDATION

/478

DEMOContent Recommendation

/479

Demo Summary• Recommended Items Web Part• Popular Items Web Part

CONTENT RECOMMENDATION

How does it work?

/4711

ArchitectureHOW DOES IT WORK?

Page withweb parts

UsageReports

SearchIndex

AnalyticsProcessing

ComponentEventStore

ExternalEvents

AnalyticsReporting DB

User

Admin

1

2

3

4

5

6

7

8

1 Usage events are recorded by the webparts and saved in the event store

Usage events from the event store or from external events are periodically processed by the Analytics Processing Component

23

Recommendation and popularity data about each item is added to the search index

4

Processed usage events are also stored in the Analytics Reporting database

5

Recommendation and popularity data is used in the respective webparts and also to influence relevance in search queries

6

Popularity data from the search index and analytics reporting database is used in usage reports

78

/4712

Usage Events• Default usage event types• View• Recommendation displayed• Recommendation clicked

• Recorded by the web parts and saved to the Event Store• You can create up to 12 custom usage event types

HOW DOES IT WORK?

/4713

Event StoreFolder in the file system that stores usage event log files

HOW DOES IT WORK?

1_2015110618_AV-SP-VM01.log

Event Type ID

Year Month Day Hour

Server

/4714

Usage Event Log1 http://dev.crt.pt/lists/games/dispform.aspx?

id=19368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:37 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/action 368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:43 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=1

368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:11:02 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/adventure CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:08 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=29

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:12 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/role-playing CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:23 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=19

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:33 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

HOW DOES IT WORK?

Event Type ID

1 http://dev.crt.pt/lists/games/dispform.aspx?id=19

Item URL

/4715

Usage Event Log1 http://dev.crt.pt/lists/games/dispform.aspx?

id=19368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:37 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/action 368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:43 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=1

368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:11:02 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/adventure CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:08 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=29

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:12 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/role-playing CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:23 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=19

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:33 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

HOW DOES IT WORK?

Obfuscated User ID

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1 11/6/2015 10:11:33 PM

Event Date and Time

/4716

Usage Event Log1 http://dev.crt.pt/lists/games/dispform.aspx?

id=19368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:37 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/action 368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:10:43 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=1

368FF3A1C93F4A0E93FCA31050E074E72EFD0E372E1D7A0226BE9A9154AC51B8

11/6/2015 10:11:02 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/adventure CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:08 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=29

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:12 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/role-playing CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:23 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

1 http://dev.crt.pt/lists/games/dispform.aspx?id=19

CAEE05C7DB68BD3E356026EEF05D4AC9C0618D8B698DC315A95BC085D31CF5F1

11/6/2015 10:11:33 PM

00000000-0000-0000-0000-000000000000

debd4ab4-1f06-48bb-aa62-86d4f227799e

23455e00-024d-47f0-8566-0b0817445c4f

1

HOW DOES IT WORK?

Subscription ID

00000000-0000-0000-0000-000000000000 debd4ab4-1f06-48bb-aa62-86d4f227799e 23455e00-024d-47f0-8566-0b0817445c4f

Site Collection ID Scope ID

/4717

Usage Analytics Processing• Usage Analytics processing runs once every 24 hours• How to manually jump start it

1. Start Search Analytics2. Push usage events to the Event Store3. Prepare usage event files4. Start Usage Analytics

HOW DOES IT WORK?

/4718

Start Search AnalyticsStart the Search Analytics timer job$a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition $sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob") $sa.StartAnalysis()

Check if the Search Analytics timer job is done$sa.GetAnalysisInfo()

HOW DOES IT WORK? > ANALYTICS PROCESSING

/4719

Push events to the Event StoreEvents are pushed every 10 minutes through a Timer Job$tj = Get-SPTimerJob -Identity ("job-usage-log-file-import") $tj.RunNow()

Check the event storeC:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Analytics_<GUID>\EventStore

HOW DOES IT WORK? > ANALYTICS PROCESSING

/4720

Prepare usage event files• Usage Analytics processes usage events from the previous day• Workaround to process current day• Copy usage log files to a custom folder• Make sure the folder is read-only• Run the job specifying the folder path

HOW DOES IT WORK? > ANALYTICS PROCESSING

/4721

Start Usage AnalyticsRun the following PowerShell script$tj = get-sptimerjob -type microsoft.office.server.search.analytics.usageanalyticsjobdefinition $tj.DisableTimerJobSchedule() $tj.StartAnalysis("\\<hostname>\Analytics_<guid>\EventStore\myevents") $tj.EnableTimerJobSchedule()

Check if the usage analytics job is done$tj.GetAnalysisInfo()

HOW DOES IT WORK? > ANALYTICS PROCESSING

/4722

Recommended Items Web Part• Shows items whose recommendedfor property includes a token from the

current URL• Display Template records usage events:• Recommendation Displayed• Recommendation Clicked

• To generate recommendations, associated items must have been clicked by three or more different users

HOW DOES IT WORK?

/4723

Popular Items Web Part• Shows items sorted by the ViewsRecent managed property (number of

views of the item in the last 14 days)• Uses owstaxIdMetadataAllTagsInfo managed property for automatic

filtering

HOW DOES IT WORK?

/4724

DEMOHow does it work?

/4725

Demo Summary• How events are recorded to the event store• Manually starting usage analytics processing• Configuring Recommended Items Webpart• Configuring Popular Items Webpart

HOW DOES IT WORK?

/4726

Reporting

/4727

Popular Items Report• Shows usage event statistics for all items in a list/library• Can be sorted by recent (last 14 days) or ever • Accessible from the list/library ribbon (Most Popular Items button)

/4728

Popularity Trends Report• Excel report• Shows usage event statistics for a: • Site collection (from site collection administration settings)• Web site (from site administration settings)• List/library item (from list/library files ribbon)

• Daily count for the last 14 days• Monthly count for the last 3 years

/4729

Using PowerShellGet Usage Reporthttps://msdn.microsoft.com/EN-US/library/microsoft.office.server.search.administration.searchserviceapplication.getrollupanalyticsitemdata.aspx

Retrieves usage data (hits and users) for a site collection or a specific web site, for a single day or a whole month.

$searchApp.GetRollupAnalyticsItemData

Get Search Reporthttps://msdn.microsoft.com/EN-US/library/microsoft.office.server.search.administration.searchserviceapplication.getsearchreport.aspx

Retrieves the search report (top queries) for one or all site collection, for a single day or a whole month.

$searchApp.GetSearchReport

/4730

DEMOReporting

/4731

Demo Summary• Popular Items Report• Popularity Trends Report• How to retrieve reporting data using PowerShell

REPORTS

/4732

Customization

/4733

Custom Event TypesGet the site collection$Site = Get-SPSite "http://localhost"

Create a custom usage event type$SSP = Get-SPEnterpriseSearchServiceApplicationProxy $EventGuid = [Guid]::NewGuid() $EventName = "MyEventType" $tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty) $newEventType = $tenantConfig.RegisterEventType($EventGuid, $EventName, "") $tenantConfig.Update($SSP)

/4734

Record a Usage Eventwindow.LogMyEventTypeToEventStore = function(url) {

ExecuteOrDelayUntilScriptLoaded(function() { var spClientContext = SP.ClientContext.get_current();

SP.Analytics.AnalyticsUsageEntry.logAnalyticsEvent(spClientContext, 256, url);

spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert("Failed to log event for item: " + document.URL + " due to: " + e.get_message()) })); }, "SP.js");

}

/4735

Event Recommendation Weight$SSP = Get-SPEnterpriseSearchServiceApplicationProxy

$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty) $event = $tenantConfig.EventTypeDefinitions | where-object {

$_.EventTypeId -eq 256}

$event.RecommendationWeight = 2 $tenantConfig.Update($SSP)

/4736

Recent Popularity Time Frame$SSP = Get-SPEnterpriseSearchServiceApplicationProxy

$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty) $event = $tenantConfig.EventTypeDefinitions | where-object {

$_.EventTypeId -eq 256 }

$event.RecentPopularityTimeFrame = 7 $tenantConfig.Update($SSP)

/4737

DEMOCustomization

/4738

Demo Summary• Create custom usage event type• Recording custom usage event• Changing event recommendation weight• Changing the recent popularity time frame

CUSTOMIZATION

/4739

Anonymous Users

/4740

Recommendations for Anonymous Users• Requires usage cookies• Cookies that identify a user session using a GUID• Lifetime of 14 days• By default usage cookies are disabled for SharePoint web applications

• By default, only the View event type can be recorded for anonymous users

/4741

Allow Event Recording for Anonymous Users

Get Search Service Application$SSP = Get-SPEnterpriseSearchServiceApplicationProxy

Get a usage event type$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty) $event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }

Enable the recording of a usage event for anonymous users $event.Options = [Microsoft.Office.Server.Search.Analytics.EventOptions]::AllowAnonymousWrite $tenantConfig.Update($SSP)

/4742

DEMOAnonymous Users

/4743

Demo Summary• Enabling the usage cookies in a web application• Enabling anonymous recording of events other than “view”

ANONYMOUS USERS

/4744

Conclusion

/4745

Key Takeaways• SharePoint records and analyzes usage data to calculate• Item popularity (view count)• Item recommendations (item relationships)

• SharePoint includes two out-of-the-box web parts• Recommendation features are based on Search• Recommendation features are customizable• Create custom usage event types• Change recommendation weight• Change recent popularity time frame

/4746

ReferencesHow to display recommendations and popular items in SharePoint Server 2013https://technet.microsoft.com/en-us/library/dn794244.aspx

Overview of analytics processing in SharePoint Server 2013https://technet.microsoft.com/en-us/library/jj219554

Configure recommendations and usage event types in SharePoint Server 2013https://technet.microsoft.com/en-us/library/jj715889.aspx#BKMK_CreateCustomUsageEventType

View usage reports in SharePoint Server 2013https://technet.microsoft.com/en-us/library/jj715890.aspx

Bella Engen’s bloghttp://blogs.technet.com/b/tothesharepoint

Waldek Mastykarz’s bloghttp://blog.mastykarz.nl

/4747

Thank YouObrigado

Tack

http://blogit.create.pt/andrevala

andre.vala@create.pt@atomicvee

http://www.linkedin.com/in/andrevala