estudio sobre el engagement online. - whitepaper
DESCRIPTION
Nedstat Publica Los Resultados Del Nuevo Estudio Sobre El Engagement Online. El whitepaper presenta un enfoque práctico para medir el engagement online y muestra las mejores prácticas de los principales clientes de Nedstat en el sector de retail, medios de comunicación, viajes y gobierno. El marco razonable y las acciones concretas descritas en el estudio dan a los responsables de marketing online las herramientas para empezar a medir el Engagement, especialmente cuando las métricas estándar de rendimiento no son aplicables.Los resultados demuestran que cualquier información sobre el comportamiento y las preferencias de los visitantes comprometidos (engaged) mejora significativamente la experiencia online, dando el valor adicional tanto a la empresa, como al cliente.TRANSCRIPT
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World Eric T. Peterson, Web Analytics Demystified, Inc.
Sponsored by Nedstat, now a part of comScore
September 2010
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 2
Executive Summary
According to a 2010 report from the British consultancy cScape, as many as 55 percent of
companies regard customer engagement as essential to the success of their businesses.
The goals behind an engagement strategy are noble indeed - creating long-term business
value while simultaneously delivering tangible value back to one’s customers. The fact that
cScape isn’t reporting that 100% of surveyed companies are focused on customer
engagement is perhaps more surprising given that few business owners are willing to
publicly declare that profits are more important than the good of their customers.
Considered macroscopically this makes perfect sense, but problems arise when any
business starts to look more closely at what creating a customer engagement strategy
really involves, especially when they examine how their efforts will be measured. “Long-
term business value” is moderately complex to measure but certainly possible.
Measurements like “tangible value back to customers” and more short-term measures of
engagement are not only difficult to measure but difficult to even define.
Still, a widespread fascination with the topic persists so we can be said to be “engaged with
engagement.”
The Internet and the rapid adoption of mobile technologies and social networks add a twist
into the conversation about engagement. Given the perception that the digital world is
“infinitely measurable,” there is a reasonable belief that engagement online should be
relatively easy to measure. Unfortunately there are about as many opinions about how to
measure engagement online as there are people having opinions. Despite the clear interest
and likely value associated with having a common definition for this term, few firms have
committed the necessary resources to define an online measure of engagement and show
how it can be applied to generate value back to both the business and their customers.
This paper attempts to fill that gap.
Written by Web Analytics Demystified, globally recognized for their thought-leadership in
the realm of digital measurement, and sponsored by Nedstat, Europe’s premier digital
measurement service, a part of comScore since September 2010. This paper provides a
reasonable framework, examples, and actionable guidance for companies looking to deploy
their own measure of engagement online. Our hope is that readers will be inspired to stop
treating engagement as a theoretical goal and instead create their own measures and then
diligently work to improve whatever baseline values they discover.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 3
Executive Summary .................................................................................................................... 2 1. Introduction ............................................................................................................................ 4
1.1 Why Measure Engagement? .............................................................................................. 5
1.2 Our Definition of Engagement ............................................................................................ 5
1.3 What Engagement is Not ................................................................................................... 6
2. How to Measure Engagement Online ................................................................................... 8
2.1 Simple Measures ................................................................................................................ 9
2.2 Page-Centric Measures .................................................................................................... 10
2.3 More Robust Measures .................................................................................................... 11
2.4 Very, Very Complex Measures ......................................................................................... 13
3. Sample Applications of Engagement ................................................................................. 14
3.1 In Online Retail ................................................................................................................. 14
3.2 In Media and Multimedia .................................................................................................. 17
3.3 In Government, Education, and Nonprofits ...................................................................... 21
3.4 In Hospitality and Travel ................................................................................................... 24
4. Conclusions .......................................................................................................................... 28
About the Author ..................................................................................................................... 28
About Web Analytics Demystified .......................................................................................... 28
About Nedstat ......................................................................................................................... 29
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 4
1. Introduction
Few topics have grabbed sales and marketing’s collective attention in the past few years
like the ability to measure and improve customer engagement. Be it with products, brands,
or entire companies, in traditional digital channels or emerging platforms like Facebook,
Twitter,YouTube, on large screens and small, engagement is incredibly hot in 2010 and will
likely continue for years to come.
At Web Analytics Demystified we have been thinking about engagement for years. In 2007
we set forth to craft a singular measure of online engagement that could be applied to any
and all digital channels1. Many others, including industry analysis powerhouse Forrester
Research2 and great minds like the Advertising Research Foundation3, Ron Shevlin4, and
Google’s own Avinash Kaushik5, all followed us, staking their claim and taking a position
regarding the measurement and utility of this seemingly simple term.
We say “seemingly simple” because even the most cursory examination of the topic reveals
that there is no single definition of “engagement” being used today. Marketers view
engagement one way, sales another, analysts yet another, not to mention the technologists,
consultants, and vendors. But leadership is looking for a single measure of engagement
that can be used to tease out opportunities and govern investments in an increasingly
complex business environment.
The reality is that measuring engagement is complex and tremendously dependent on your
businesses goals and the relationship you have with your customers. Even in our highly
structured mathematical model first published in 20076 we comment that thresholds need to
be individually set and may differ across lines of business, much less businesses
themselves. Companies wishing to leverage a measure of engagement need to do so with
great clarity regarding the outcomes they are trying to create. They also need to be
prepared to define and defend a metric that nearly everyone already believes they
understand, a difficult task even in the best of times.
With this in mind you may be asking, “why bother measuring engagement at all?” Why
indeed.
1 http://blog.webanalyticsdemystified.com/weblog/2006/12/how-do-you-calculate-engagement-part-i.html 2 http://www.forrester.com/rb/Research/marketings_new_key_metric_engagement/q/id/42124/t/2 3 http://en.wikipedia.org/wiki/Brand_engagement 4 http://en.wikipedia.org/wiki/Customer_engagement 5 http://www.kaushik.net/avinash/2007/10/engagement-is-not-a-metric-its-an-excuse.html 6 http://www.webanalyticsdemystified.com/content/white-papers.asp
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 5
1.1 Why Measure Engagement?
Fortunately the answer to this question is simple: because 97 percent of people don’t
purchase online. Companies are looking for a reasonable and useful measure of
engagement because we are at a critical transition point in the evolution of online channels,
one where consumers are well-trained to use technology as part of the research process
but still pick up the phone, hop in the car, or walk down the street to complete the
transaction.
Until marketers and business leaders are effectively able to optimize against a significant
number of online transactions, be they purchases, leads, views, or clicks - and by
“significant” we mean significant to overall corporate revenues - a need for proxies for these
actions will persist. Simple statistics necessitate a reasonable volume of outcomes to
differentiate efforts to improve against the status quo; since most businesses fail to
generate truly significant volumes of primary outcomes (e.g., purchases, qualified leads,
etc.) some other method for testing is required.
While there are possibly better proxies, the global marketing collective appears to have
decided that “engagement” was the answer, thus digital measurement practitioners and
their managers have been asked to provide reasonable measures of engagement and
show how these measures can be used to drive the business forward. Still, the question of
how to create these reasonable measures is open and worthy of discussion.
1.2 Our Definition of Engagement
As we noted, Web Analytics Demystified began studying how to measure engagement in
digital channels back in 2007 when we published the first of several blog posts on the
subject. Our work culminated in the publication of what is still the most widely cited
technical work on the subject, in which we assert that in digitally connected channels, any
robust measure of engagement needs to account for clicks, duration, recency, loyalty,
brand awareness, and direct outreach on the part of the consumer. This insight was
represented mathematically as follows:
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 6
Technically speaking this equation translates into the following description:
Engagement is the measurable demonstration of specific neural activity that serves to focus
an individual’s behavior.
Basically we are saying that based on specific, measurable activities, once we determine
an individual is paying attention, we have already determined that they are engaged.
Extending this into the online world and simultaneously encouraging users of the metrics to
take an outcomes-focused approach, our definition effectively becomes:
Engagement is an estimate of the depth of visitor interaction against a clearly defined set of
goals.
By way of example, the following screenshot shows one way this paper’s sponsor’s
technology can be used to define an engagement segment based on loyalty (number of
visits), brand awareness (search term), and a combination of click-depth and site goals
(“total engagement”, to be discussed at length later in the paper):
Figure 1: Example engagement segment from Nedstat focusing on loyalty, brand, and Nedstat’s page scoring
methodology
1.3 What Engagement is Not
Any debate about the measure of engagement, much less engagement’s utility in online
marketing, invariably results in questions about where engagement stops and where
conversion and satisfaction start. On this point we have always been delightfully clear:
engagement is not conversion, engagement is not satisfaction.
These points, while confounding to some, are surprisingly easy to understand when put into
context. Our first point, engagement is not conversion, is incredibly easy to understand
when you consider the web sites of great European brands including Porsche, Mercedes,
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 7
and Ferrari. Even from our own direct experience, a great number of individuals likely
spend inordinate amounts of time reviewing technical specifications for these cars, viewing
pictures, sharing links with friends, reviewing online forums, and generally demonstrate a
tremendous amount of “neural activity” … without ever successfully completing a
transaction with the brand, online or off.
Similarly regarding satisfaction, an individual can be
simultaneously highly engaged and highly unsatisfied,
highly engaged and highly satisfied, poorly engaged but
highly satisfied, and poorly engaged and poorly (or
neutrally) satisfied. Imagine the last time you had an issue
with your mobile service provider; perhaps spending an
inordinate amount of time on hold seeking an answer only to
eventually be told “no” - a common occurrence, one where
the individual demonstrated a high-level of attention but the
outcome was decidedly unsatisfying.
The reality is that engagement, conversion, and satisfaction are three interrelated
dimensions of every customer’s interaction with a product, brand, or company. At any point
during the lifetime of the relationship between consumer and company each of these three
dimensions will ebb and flow based on needs and availability. The trick is not recognizing
that these dimensions are different; the trick is quantifying each in a way that can be
leveraged effectively by the business.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 8
2. How to Measure Engagement Online
Because of the confusion surrounding the term there are, generally speaking, nearly as
many definitions and descriptions regarding how to measure engagement as there are
participants in the debate. Some people like to keep it simple, some prefer more complex
models, and some people have boxed themselves into an uncomfortable little corner
insisting that “engagement is an excuse.”
The best news is that everyone is correct.
Since we published our original report on the subject, we have come to a profound
realization that “engagement” is not a technical term with a strict, technical measurement
strategy attached to it:
Engagement is an idea, a business concept, that will vary appropriately from business to
business and audience to audience.
In this context, our original measure is just as correct when consistently applied as the most
simple measure possible. It is equally as valuable to a business that has invested hundreds
of thousands of Euros building an end-to-end customer data warehouse capable of
evaluating the relationship between consumer engagement and lifetime value online and
off.
Put another way, much like the underlying technology required to measure engagement
online, it’s not the tools you use but how you use them. Again, this is fantastic news,
because any business anywhere can immediately begin thinking of their audience in terms
of engagement and segmenting based on levels thereof. All they need to do to get started
is come to agreement on what “engagement” means for their business.
With this in mind, here are a handful of examples of how engagement can be measured in
and across connected channels. In all cases the fundamental technology required to really
take advantage of the approach is visitor-level segmentation, a feature that is thankfully
found in nearly all Enterprise-class web analytics solutions in the market today.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 9
2.1 Simple Measures
The simplest measures of engagement are typically derived from click-depth and time on
site. For example, if the “average” visitor to your web site views 4 pages then we can treat
this as a baseline for our definition of “attention” and reasonably decide that any visitor who
views more than twice that number of pages (8) is engaged.
Figure 2: Breakdown of depth of visits to a web site, as reported by Nedstat
Similarly we could work from average time on site or some measure of recency or loyalty,
the process works exactly the same way. You create a visitor segment of visitors who
exceed a reasonable “attention threshold” and treat that segment as “engaged” with your
site.
Invariably when discussing engagement in the context of click-depth and time on site
someone declares that “more clicks is not necessarily a good thing” and that “long time on
site can be an indicator of confusion, not intent to purchase.”
Exactly.
Recall that we have said that engagement, satisfaction, and conversion are different
dimensions of visitor behavior. In these examples the visitor or visitors are clearly
demonstrating attention without regard to their level of satisfaction or propensity to transact.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 10
More clicks might not be a good thing, but the very fact that those clicks are being
generated means that the users are engaged.
2.2 Page-Centric Measures
One of the areas we focused on in our original research into engagement was the idea that
specific interactions are good indicators of consumer attention. Examples include requests
for additional information, downloading documents or applications, watching video,
commenting on blog posts, reaching out via social media, and so on. In our model we
created a simple scoring mechanism, the Interaction Index that allowed users to score
these interactions as a component of visitor engagement.
This paper’s sponsor, Nedstat, and others have something simple yet slightly more
powerful, the ability to assign specific “scores” to individual pages and actions and then
sum those scores to create an “engagement profile” for each site visitor. These scores can
then be applied to any dimension in the analytics system, essentially creating a two- to
multi-dimensional analysis of engagement scores.
Type of page or action Value
Any landing page 10
Any main menu page 20
Any sub menu page 30
Any demo request form or event registration form 40
Any request or registration confirmation page 50
While the set-up of this approach is somewhat involved and requires making certain
assumptions, in our experience those assumptions are simply part of the process of
determining what engagement means to your site, products, brand, or company. As you will
see in the sample applications section below, this strategy can be particularly useful in non-
transactional systems and in complex sales situations.
Another valuable use of this type of engagement scoring, at least in systems having
reasonably robust attribution modeling functionality, is to create an additional model for
comparison. Rather than evaluate campaign revenue attribution against simple rules like
“First click 100%” and “Equal share to all”, an engagement model essentially allocates
campaign revenue based on the aggregate of visitor “scores” driven by each traffic driver.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 11
Figure 3: Example of Nedstat’s engagement attribution model, with revenue attribution by campaign based on
“equal share” (left) and “engagement” (right) with the percentage-wise difference between the two calculated for
comparison.
2.3 More Robust Measures
While the simple and page-centric measures are both useful and appropriate for some
types of businesses, most companies actively working to understand visitor and consumer
engagement are seeking slightly more complex measures. Typically these more
complicated approaches involve the creation of visitor segments to incorporate multiple
measures of attention, for example, click-depth, time spent, and recency.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 12
Figure 4: Example from Nedstat showing visitor segmentation module capable of creating moderately complex
measures of engagement by combining multiple dimensions, metrics, and thresholds using a very simple drag-
and-drop interface.
Web Analytics Demystified is particularly encouraged by this type of strategy because it
allows companies and brands flexibility in their definition of engagement. Especially when
the technology used allows for simple boolean logic in the combination of segment
descriptors (e.g., “more than five page views OR more than five minutes AND visited in the
last 14 days”), analysts can create reasonably robust models that vet well with their
business counterparts understanding of engagement.
This strategy, the combination of simple models and page-centric approaches with
segmentation tools, will be discussed in the context of specific businesses and business
types later in this paper.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 13
2.4 Very, Very Complex Measures
Inevitably there are more fantastic complex measures of engagement. Some companies
known to Web Analytics Demystified are using incredibly powerful segmentation engines
and data warehouses to create entirely new engagement metrics that can be applied to any
other dimension contained in the system. Other companies are currently combining data
from multiple online and offline sources to build statistical models designed to identify key
demonstrations of attention likely to lead to long-term customer value, decreased levels of
attrition, and lower customer support costs over time.
This latter point is important, at least to companies committed to measuring customer
engagement over multiple channels and touch-points. Ultimately,every company needs to
have a reason to measure engagement. Usually that reason is “revenue” or “profitability” so
while the strategies described elsewhere in this document are perfectly reasonable proxies,
the most powerful measure of engagement is one that has been mathematically shown to
be strongly correlated with long term customer value.
Put another way, if you really want to create a measure of engagement that has predictive
value to your business, you’re going to need to combine as many data points as you
reasonably can and test statistical models to determine which demonstrations of attention
are true precursors to long-term revenue. With this output you can then reasonably go back
and set page scores and determine which dimensions and metrics are worth incorporating
into your measure of engagement and which are worth leaving out. Otherwise what you are
left with is guesswork, hypotheses, and assumptions.
That said, given the relatively small number of companies able to conduct this type of
complex analysis juxtaposed against the hype surrounding the concept of engagement and
the fact that so few people ultimately complete transactions online, Web Analytics
Demystified strongly believes that creating a useful, believable, and well-vetted model for
measuring engagement on your site is time well spent. Doing so will help you better
understand if measuring engagement adds value to your site and analysis and creates the
mindset necessary to take full advantage of a more complex model and approach.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 14
3. Sample Applications of Engagement
With knowledge of the several ways through which engagement can be measured online,
the balance of this paper will demonstrate how those strategies can be applied on a variety
of sites, business models, and technologies. Our hope is that by putting these ideas into
more concrete terms we will encourage readers to start considering the application of
engagement measurement to their own sites.
Web Analytics Demystified is indebted to the companies willing to discuss their approach
towards measuring engagement in this section; it is early days for most companies in this
regard and these companies are thinking ahead of the curve, looking for competitive
advantages in data and analysis wherever they can. Furthermore, in the following section
we often present hypothetical uses of this technology in an effort to protect participating
companies’ efforts and intellectual property.
3.1 In Online Retail
In online retail, engagement is usually applied to help mine opportunities in the 97% who do
not transact and assign “additional value” to marketing efforts that don’t immediately
convert into sales and revenue. By way of example, Nedstat customer Ladbrokes (UK) is
focused on acquiring customers online likely to become loyal gamers. To facilitate this, the
company’s Web Analytics Manager, Jagjeet Mann, is using the engagement and
segmentation functionality in Nedstat’s flagship product Sitestat to mine their digitally
collected data. “Our Nedstat implementation creates tremendous visibility into digital
response and opportunity,” says Mr. Mann. “We consider engagement somewhat
classically, focusing on click depth, time spent, recency, and cross-session behavior to
determine where we can optimize the entire gaming experience.”
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 15
Figure 5: Nedstat customer Ladbrokes.com home page showing the types of engagement and interaction the
company is working to drive visitors and customer towards.
To this end, and by way of example only, Ladbrokes could use the following page scores to
incrementally measure visitor engagement with their site:
Type of page or action Value
Any landing page 10
Any game specific page 20
Any game registration form 30
Any game registration form completion 40
Any game download request 50
Any request to open an account 60
While online gaming is clearly a variation on traditional online commerce, hopefully the
reader can see how these page scores could be applied to the sale of goods and services
of any kind. All that is required is internal agreement about the types of pages business
leaders believe provide measurable evidence that the visitor is paying attention.
What’s more, using the “robust” model described previously in this document, Ladbrokes
could couple page scores with other measures of visitor attention to create an even more
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 16
robust engagement segmentation strategy. For example, Ladbrokes could create a visitor
segment of “highly engaged” visitors who:
> Have a cumulative engagement score greater than 500 points
> Have visited the site more than 5 times recently
> Have spent more than 30 minutes on the site during any visit
> Have arrived directly to the site
Figure 6: Example of a “highly engaged” segment as detailed in Nedstat’s Visitor Segmentation module.
With this “highly engaged” segment, Ladbrokes can then go back and look at marketing
acquisition efforts including campaigns and landing pages in the context of their propensity
to drive high engagement visitors to the site.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 17
Figure 7: Example of a “highly engaged” visitor segment applied to Ladbrokes online marketing efforts. This type of
report is especially valuable in comparison to both a “less engaged” segment and the site audience as a whole.
Depending on the exact implementation of the Sitestat technology, Ladbrokes may be able
to take this analysis one step further and attempt to market directly to highly engaged
visitors who have not yet completed the conversion process.
3.2 In Media and Multimedia
The measures described in this paper are incredibly valuable on sites lacking direct
transactions and where the primary measure of long-term success is loyalty and ultimately
engagement. Monetized via ad impressions and clicks, media and multimedia sites around
the world are lacking simple financial measures like “conversion” and “average order value.”
Having realized that page views alone are poor measures of financial success, smart
companies like the UK’s Cantos can depend on slightly more complex measures of
engagement to understand which content, acquisition marketing, layout, and channel are
most likely to drive long-term relationships.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 18
Figure 8: Nedstat customer Cantos featuring video front-and-center on their home page, evidence of the value the
company places on video-based engagement.
What’s more, media properties like Cantos are increasingly relying on multimedia
presentation (e.g., video) and mobile formatted content to ensure the most engaging, most
present relationship with their visitors possible. To this end, Cantos may choose to assign
the following scores to content on their site for use in the engagement model described in
this paper:
Type of page or action Value
Any landing page 10
Any company page 20
Any view of a video 30
Any mobile view of non-video content 40
Any mobile view of video-based content 50
Any sharing of content via social media 60
Any completed registration 100
The example of “sharing content via social media” is especially important to media and
multimedia companies because of the potential for viral sharing.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 19
Figure 9: Cantos sharing options, available with each video view, potentially driving viral engagement among
members of the sharer’s social network.
Just like in the retail model, media and multimedia properties could choose to use the more
complex engagement calculation described earlier in this paper to create a segment of
“highly engaged” site visitors who:
> Have a cumulative engagement score greater than 500 points
> Have visited the site more than 15 times recently
> Have spent more than 30 minutes on the site during any visit
> Watch at least three videos during each visit to the site
> Have arrived directly to the site
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 20
Figure 10: Example of Cantos “highly engaged” visitor segment as detailed in Nedstat’s Visitor Segmentation
module.
With this highly engaged segment defined, Cantos could then go back and examine the
different companies, topics, and individuals talked about in each of the company’s videos to
explore for highly engaging content.
Figure 11: Sample of Cantos video playlists for “highly engaged” site visitors as tracked using Nedstat’s video
tracking capabilities.
Also, given the importance of mobile content distribution and the increasing popularity of
smart phones, netbooks, and tablet computers, media properties like Cantos may take this
segmentation a step further and examine the popularity of content relative to the type of
device used for browsing.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 21
Figure 12: Example of “highly engaged” visitors to Cantos web site by mobile device used.
Armed with this information, Cantos can begin to make much more informed programming
decisions on a channel, device type, and even device basis depending on what their
infrastructure will allow. While the analysis would certainly not govern which news was
covered, insights gained from the exploration of engagement can certainly help editors and
publishers decide which stories to lead with, feature, and promote.
3.3 In Government, Education, and Nonprofits
One of the central challenges facing many types of sites, especially those in government,
education and the nonprofit sector, is ensuring that the most compelling and appealing
content is adequately promoted, structured and presented.
On the other hand, such organizations are, at the same time, often tasked with determining
which content is no longer being valued (or is of less value) by visitors and is therefore
worthy of removal, especially when content is expensive to produce and maintain. Often
over time, service-oriented websites have a tendency to accumulate content that has to be
reflected in site design but does little to add value to the user experience.
An example of how a measure of engagement can be used to alleviate this challenge
comes from Learning and Teaching Scotland (LTS), funded by the Scottish Government
and a Nedstat customer. Recently, its website (an online service for school teachers and
practitioners across Scotland) was considerably re-developed. This involved an extensive
review of all educational content, with the new website bringing together information and
resources from a range of former and separate LTS websites (including Curriculum for
Excellence, Glow Scotland, Enterprise in Education, Global Citizenship, Inclusion and
Equality and many more). It is now anticipated that it will be easier for users to find what
they are looking for and make links across all areas of learning.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 22
Figure 13: The new Learning and Teaching Scotland web site, much refined and well-focused on delivery of LTS
“Curriculum for Excellence” program.
One of several key content areas on the new website is ‘Learning, teaching and
assessment’. Assuming that Scottish school teachers will be particularly engaged with this
specific topic, a relatively simple measure of engagement could be defined on the basis of
these conditions: visitors who
> have a cumulative engagement score greater than 100 points
> have recently visited the LTS site more than twice (>2)
> have spent more than 15 minutes during any visit to the LTS site
> have viewed one or more pages belonging to ‘Learning, teaching and assessment’
> have downloaded one or more “documents” (.pdf, .doc, .xls, .ppt, .mp4 and so on)
from the site.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 23
Figure 14: Example of LTS “highly engaged” visitor segment for ‘Learning, teaching and assessment’.
In this model we essentially extend active use and engagement with Learning, teaching
and assessment back to the main site via segmentation to examine the content presented
on the LTS site (looking for information that “engaged Learning, teaching and assessment
users” consider to be valuable).
Going a step further, the above high engagement segment can then be applied to the LTS
site as a whole to gain additional behavioral insights into those visitors already known to be
engaged with ‘Learning, teaching and assessment’. For example, what other content do
they consider to be useful?
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 24
Figure 15: Example of Nedstat’s Direct View which makes it possible to display Sitestat metrics directly on the LTS
site. Notice that Direct View is also being used specifically for the high engagement segment shown in Figure 14.
The above approach and analysis will help LTS content managers and information
architects better understand which content is deemed most relevant and useful to the most
engaged audiences.
This information will prove invaluable when the LTS site is again refreshed in the future.
For example, LTS communications personnel, in close association with content managers,
information architects and subject matter experts, will be able to make informed decisions
about what online content “works” and how it should be structured and presented.
3.4 In Hospitality and Travel
Few online sectors are as competitive as online hospitality and travel where rates rise and
fall, margins are typically thin, and external factors have the potential to change the rules of
the game completely. Perhaps unsurprisingly then is the observation that travel and
hospitality sites have given great thought to how a measure of engagement might be
leveraged to gain a competitive edge.
“Engagement has long been a proxy for conversion within accommodations’ sites like
Hostelbookers.com,” says Mats Diedrichen, Nedstat customer Hostelbookers.com’s
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 25
Director of E-Commerce. “Philosophically we understand that we work in an incredibly
competitive sector and so we use our measures of engagement to best understand how
visitors are using our site.” By treating engagement as a proxy for conversion, sites like
Hostelbookers.com and TUI (www.tui-group.com) are able to make nuanced business
decisions about display advertising and search marketing efforts.
Figure 16: Hostelbookers.com website, their refund action gives an indication of just how competitive this sector is.
“Engagement is rebalancing the emphasis we put on search engine marketing,” says TUI’s
Nioklina Bernascone, Online Marketing Executive. By leveraging Nedstat and a model very
similar to that described in this paper, TUI is able to differentiate levels of engagement
across their acquisition marketing efforts. One key observation the company has already
made is that visitors acquired via organic search are measurably more engaged than
visitors acquired through more costly channels. What’s more, even if highly engaged
visitors don’t necessarily convert immediately, given basic contact information TUI is able to
appropriately re-market, further increasing conversion rates online.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 26
Figure 17: Nedstat customer TUI measures visitor engagement to allocate marketing budget most effectively
In either example it is quite easy to imagine a scoring model similar to those presented
elsewhere in this document, one where scores increase as visitors get increasingly close to
completing the booking process:
Type of page or action Value
Any landing page 10
Any search for a destination or route 20
Any view of accommodations or flight options 30
Any selection of accommodations or flight options 40
Any view in the booking process 50
Any view of the next-to-last step in the booking process 60
Any submission of contact information (e.g., email address) 100
Given that non-business travelers are somewhat less likely to return to a travel and
hospitality site, frequently the definition of “highly engaged” visitors may differ somewhat
from traditional retail or media sites:
> Have a cumulative engagement score greater than 500 points
> Have spent more than 20 minutes on the site during any visit
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 27
> Have arrived directly to the site from a “branded” search
Travel and hospitality sites also have a unique opportunity to explore visitor engagement in
the context of brands, locations, options, and destinations. For example, TUI may discover
that their most engaged audience prefers to book flights on a particular airline or
Hostelbookers.com may discover that a particular set of properties are most compelling.
Figure 18: Example of Hostelbookers.com “highly engaged non-converting” visitor segment definition and the
segment’s application to pages showing popular destinations offered by the company.
Especially in competitive markets where conversion is primarily a function of price and
availability, information about the levels of visitor engagement can be an incredibly valuable
asset.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 28
4. Conclusions
Both Web Analytics Demystified and Nedstat sincerely hope that you found this paper
practical and useful. While it is still “early days” for measuring engagement online, it is our
firm belief that companies willing to invest the time and energy into the effort will appreciate
value derived from a deeper understanding of their visitors that results in a refinement in
their ongoing efforts to create value for both the business and their customers. This paper
and the examples provided demonstrate that reasonable insights are possible based on a
relatively small amount of work and investment. All that is required is willingness to “think
different” about your digital measurement efforts, your customers, and your business goals.
The author welcomes comments and questions about this document. Please feel free to
write directly to Eric T. Peterson ([email protected])
About the Author
Eric T. Peterson, CEO and Principal Consultant at Web Analytics Demystified, has worked
in web analytics since the late 1990's in a variety of roles including practitioner, consultant,
and analyst for several market-leading companies. He is the author of three best-selling
books on the subject, Web Analytics Demystified, Web Site Measurement Hacks, and The
Big Book of Key Performance Indicators, as well as one of the most popular web analytics
bloggers at www.webanalyticsdemystified.com. Mr. Peterson has committed much of his
life to the betterment of the web analytics community, so much so that Jim Sterne,
President and co-founder of the Web Analytics Association says "Eric's leadership in the
industry in unparalleled, his devotion to the community is legendary, and his years of
experience translate immediately into strategic and tactical competitive advantage for
everybody who works with him."
About Web Analytics Demystified
Web Analytics Demystified, founded in 2007 by internationally known author and former
Jupiter Research analyst Eric T. Peterson, provides objective strategic guidance to
companies striving to realize the full potential of their investment in web analytics. By
bridging the gap between measurement technology and business strategy, Web Analytics
Demystified has provided guidance to hundreds of companies around the world, including
many of the best known retailers, financial services institutions, and media properties on the
Internet. For more information on Eric T. Peterson and Web Analytics Demystified, please
visit www.webanalyticsdemystified.com, email
[email protected], or call (503) 282-2601.
Visitor Engagement: A “No Excuses” Approach to Measure Engagement in the Digital World
© Nedstat BV - now a part of comScore 29
About Nedstat
Nedstat is a leading provider of web analytics and innovative solutions for online business
optimization. The Nedstat products and services help marketers improve the effectiveness
and profitability of online marketing programs and optimize customer experiences and
marketing spend. The September 2010 acquisition of Nedstat by comScore, Inc., a leader
in measuring the digital world, creates opportunity for new product development and market
expansion using the collective assets of both companies to enable even greater value for
clients.
The customer list includes many renowned and internationally operating companies such
as Electrabel, Europcar, London Stock Exchange, Panasonic, Renault, Repsol and Wolters
Kluwer. Stream Sense is used by many public and commercial television organizations
throughout Europe such as Eurosport, FiveTV, France Televisions, NOS, NRK, RTL, SVT
and VRT.
Key accreditations by Europe’s leading independent web standards organizations such as
ABCe, OJD, KIA and Audiweb ensure that metrics are in full compliance with leading
industry standards.
Nedstat was founded in 1996 and employs 125 people with offices in Amsterdam (HQ),
Antwerp, Frankfurt, London, Madrid, Paris and Stockholm. Local people staff all offices and
the Nedstat products, documentation, consultancy and training are offered in local
languages.