copyright is owned by the author of the thesis. permission is ......(educause, may 2005). all social...
TRANSCRIPT
Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere without the permission of the Author.
Interface and Feedback Factors in Social Bookmark Usage
Jingning Li
Submitted in partial fulfilment of the requirements for a
Master of Information Sciences at Massey University
Auckland, New Zealand
September 2010
i
Statement of academic integrity
I declare that this research study is entirely the product of my own work and
that is has not been taken from the work of others. When the work and ideas of
others have been used in the study, the work has been properly cited in the
text.
Signature
ii
Acknowledgements
I would like to express my gratitude to all those who helped me to complete this
research.
I am deeply indebted to my supervisor, Dr. Brian Whitworth, whose stimulating
suggestions and encouragement constantly helped me throughout the time I
was researching and writing this paper.
I would like to thank my family for supporting me through this academic study.
I want to thank the students who participated in the evaluation survey and
provided valuable feedback.
iii
Table of contents
Abstract ................................................................................................................ 1
Chapter 1: Introduction ............................................................................................ 2
1.1 Background .................................................................................................................... 2 1.2 Social bookmarking ....................................................................................................... 2 1.3 Social bookmark buttons ................................................................................................ 3 1.4 Purpose of the study ....................................................................................................... 4
Chapter 2: Literature Review .................................................................................... 5
2.1 Introduction .................................................................................................................... 5 2.2 A brief literature survey ................................................................................................. 5
2.2.1 The tag feature .................................................................................................. 6 2.2.2 The filter feature ............................................................................................... 8 2.2.3 The ranking feature ......................................................................................... 14 2.2.4 Usability and functionality ............................................................................. 15 2.2.5 Three relevant groups ..................................................................................... 18 2.2.6 Culture ............................................................................................................ 28
2.3 Variables....................................................................................................................... 29 2.4 Theoretical framework ................................................................................................. 36 2.5 Research question ........................................................................................................ 38 2.6 Hypotheses ................................................................................................................... 39 2.7 Research type ............................................................................................................... 39
Chapter 3: First Phase Study Method ..................................................................... 40
3.1 A new type of social bookmark button – Bligg ........................................................... 40 3.1.1 Processing order ............................................................................................. 40 3.1.2 Each processing step’s versions ...................................................................... 41 3.1.3 Interface design ............................................................................................... 44 3.1.4 Database.......................................................................................................... 46
3.2 Research design ........................................................................................................... 49 3.3 Subject requirement ..................................................................................................... 53 3.4 Questionnaire design ..................................................................................................... 55 3.5 Procedure plan ............................................................................................................. 56
3.5.1 Unknown phase .............................................................................................. 57 3.5.2 Explaining and seeing phase ........................................................................... 57 3.5.3 Using phase ..................................................................................................... 57
3.6 Measurement method ................................................................................................... 58 3.6.1 Getting data ..................................................................................................... 59 3.6.2 Analysing data ................................................................................................ 59
Chapter 4: First Phase Study Results ....................................................................... 60
4.1 Getting data ready for analysis..................................................................................... 60 4.1.1 Editing data ..................................................................................................... 60 4.1.2 Handling blank responses ............................................................................... 60 4.1.3 Coding data ..................................................................................................... 60 4.1.4 Categorising data ............................................................................................ 61 4.1.5 Entering data ................................................................................................... 61
4.2 Data analysis ................................................................................................................ 62 4.2.1 Feel for the data .............................................................................................. 62 4.2.2 Hypothesis testing ........................................................................................... 66
4.3 Interpretation of results ................................................................................................ 71 Chapter 5: Second Phase Study Method ................................................................. 73
iv
5.1 Pilot studies ................................................................................................................. 73 5.1.1 First pilot study ............................................................................................... 73 5.1.2 Second pilot study........................................................................................... 74
5.2 Last social bookmark button – Bligg ........................................................................... 75 5.2.1 Processing order ............................................................................................. 75 5.2.2 Feedback versions ........................................................................................... 75 5.2.3 Interface design ............................................................................................... 76 5.2.4 Database.......................................................................................................... 76
5.3 Research design ........................................................................................................... 77 5.4 Subject requirement ..................................................................................................... 78 5.5 Questionnaire design ................................................................................................... 78 5.6 Procedure plan and measurement method ................................................................... 79
Chapter 6: Second Phase Study Results .................................................................. 80
6.1 Getting data ready for analysis .................................................................................... 80 6.1.1 Editing data ..................................................................................................... 80 6.1.2 Handling blank responses ............................................................................... 80 6.1.3 Coding data ..................................................................................................... 80 6.1.4 Categorising data ............................................................................................ 80 6.1.5 Entering data ................................................................................................... 81
6.2 Data analysis ................................................................................................................ 81 6.2.1 Feel for the data .............................................................................................. 81 6.2.2 Hypothesis testing ........................................................................................... 85
6.3 Interpretation of results ................................................................................................ 86 Chapter 7: Discussion ............................................................................................. 88
7.1 Conclusions ................................................................................................................. 88 7.2 Future potential ............................................................................................................ 88
References ............................................................................................................. 89
Appendix A: Comparing Social Bookmark Buttons .................................................. 98
Appendix B: Different Versions of Cognitive Effort and Social Feedback ................ 101
Appendix C: Bligg Buttons Interface Screenshots – First Study .............................. 103
Appendix D: The Beatles Mini Website Interface Screenshots ............................... 106
Appendix E: Survey Letter .................................................................................... 108
Appendix F: Timetable ......................................................................................... 109
Appendix G: Consent Form ................................................................................... 110
Appendix H: First Study Questionnaire ................................................................. 111
Appendix I: Procedure Navigation Page ................................................................ 115
Appendix J: Open-Ended Questions Response – First Study .................................. 116
Appendix K: Updated Bligg Button Interface Screenshots – First Pilot Study .......... 119
Appendix L: Protocol Analysis Results– First Pilot Study ........................................ 120
Appendix M: Protocol Analysis Results– Second Pilot Study .................................. 121
Appendix N: Cognitive Walkthrough– Second Pilot Study ..................................... 122
Appendix O: Bligg Button Interface Screenshots – Second Study ........................... 131
v
Appendix P: Second Study Questionnaire ............................................................. 132
Appendix Q: Open-Ended Questions Response – Second Study ............................. 135
vi
List of tables
Table 2-1: Summary of social bookmarking’s main features ........................................ 15
Table 2-2: Possible factors by usage stages .................................................................. 35
Table 2-3: Cognitive effort and social feedback by social bookmark buttons (collected
by July 16th, 2009 from bbc.co.uk) .............................................................................. 37
Table 3-1: Bligg buttons by cognitive effort and social feedback ................................. 44
Table 3-2: Three phases following usage stages ........................................................... 50
Table 3-3: Two optional research methods .................................................................. 50
Table 3-4: Questionnaire questions’ details ................................................................. 55
Table 4-1: Mean task time taken in milliseconds by cognitive effort (N=60) ............... 64
Table 4-2: Likelihood of using social bookmarking by usage stages ............................. 67
Table 4-3: Likelihood of using the Bligg social bookmark button by Bligg buttons ...... 68
Table 4-4: Bligg buttons' ease of use and usefulness by Bligg buttons ........................ 68
Table 4-5: Mean easy to use social bookmark buttons by cognitive effort and social
feedback (N=60) ........................................................................................................... 68
Table 4-6: Mean usefulness of social bookmark buttons by cognitive effort and social
feedback (N=60) ........................................................................................................... 68
Table 4-7: Mean like to use and will use social bookmarking by cognitive effort and
social feedback (N=60) ................................................................................................. 69
Table 4-8: Mean will use and would recommend social bookmark button by cognitive
effort and social feedback (N=60) ................................................................................ 69
Table 4-9: Likelihood of using social bookmarking by social bookmark button's
cognitive effort ............................................................................................................. 70
Table 4-10: Likelihood of using social bookmarking by social bookmark button's social
feedback ....................................................................................................................... 71
Table 5-1: Questionnaire questions' details ................................................................. 78
Table 6-1: Mean willing and effort to click the links by gender (N=24) ........................ 83
Table 6-2: Mean time taken in milliseconds by social feedback (N=24) ...................... 84
Table 6-3: Mean like to use and will use social bookmarking by social feedback (N=24)
...................................................................................................................................... 85
vii
Table 6-4: Mean will use and would recommend social bookmark button by social
feedback (N=24) ........................................................................................................... 85
Table 6-5: Likelihood of using social bookmarking by social bookmark button's social
feedback ....................................................................................................................... 86
Table 6-6: Social bookmark button details ................................................................... 87
viii
List of figures
Figure 1-1: delicious.com tag cloud................................................................................ 3
Figure 1-2: bbc.com social bookmark buttons list (BBC, 2008) ...................................... 3
Figure 2-1: Information discovery in social bookmarking service (Klaisubun,
Kajondecha and Ishikawa, 2007) .................................................................................. 10
Figure 2-2: Pivot browsing’s working theory ................................................................ 12
Figure 2-3: Conceptual model of web interfaces (Ivory and Megraw, 2005, p.468) .... 19
Figure 2-4: “diggs” Bullets (Digg, 2010) ........................................................................ 21
Figure 2-5: The triadic colour scheme (Kuler, 2010) .................................................... 22
Figure 2-6: Compare digg.com and delicious.com’s main colours ............................... 23
Figure 2-7: Compare digg.com and delicious.com’s unique visitors (Compete, 2010) 24
Figure 2-8: Variables diagrams ..................................................................................... 32
Figure 2-9: Usage stages ............................................................................................... 35
Figure 2-10: Research question variables diagram....................................................... 38
Figure 3-1: The Beatles official website screenshot (Thebeatles, 2010) ...................... 47
Figure 3-2: Data transfer details ................................................................................... 49
Figure 3-3: Research method design ............................................................................ 52
Figure 4-1: Mean preference for social bookmark buttons by cognitive effort (N=60) 63
Figure 4-2: Mean helpfulness by details entered already (N=60) ................................ 63
Figure 4-3: Mean willing and effort to enter the details (N=60) .................................. 64
Figure 4-4: Mean usefulness of social feedback (N=60) .............................................. 66
Figure 6-1: Mean usefulness, liking to use and preference for social bookmark buttons
by social feedback (N=24)............................................................................................. 82
Figure 6-2: Mean like to use social bookmark buttons by social feedback (N=24) ...... 82
Figure 6-3: Mean ease of use Bligg buttons by social feedback (N=24) ....................... 84
1
Abstract
Individual bookmarks are a fundamental feature of Internet web browsers,
letting users save and collect their favourite web page locations, but users
cannot use their bookmarks on other computers and cannot share their
bookmarks with others. Social bookmarking aims to improve this situation by
letting people share bookmarks on the Internet. The term was first used by
Delicious in late 2003. They not only let users store, organise and access their
bookmarks online, but also let them share them with other users. Social
bookmarking lets people see what sites other people bookmark under the
common tags that users commonly organise their bookmarks by. This research
investigates the personal and social factors affecting social bookmark usage
and suggests how they work together to influence usage. The two factors
investigated were: cognitive effort and social feedback. To study them, a social
bookmark simulation called Bligg was created, which allowed various levels of
effort and feedback to be evaluated. In the first study, cognitive effort
significantly affected willingness to use social bookmarking, but social feedback
had no effect. However, in the second study that controlled for reading effort, it
was significant. It was concluded that cognitive effort is an enabling factor for
the effect of feedback on social bookmark usage.
Keywords: Social bookmarking; cognitive effort; social feedback; likelihood of
use.
2
Chapter 1: Introduction
1.1 Background
Bookmarks are a fundamental feature of almost all web browsers, and are
visible in web browser menus. Users use bookmarks to save and collect their
favourite web page locations (URLs) into their computers. Folders let users
classify and manage bookmarks and find their bookmarks easily on their own
computers, but users cannot find their saved bookmarks on other computers or
share their bookmarks with others.
In April 1996, the concept of shared online bookmarks was introduced on the
website itList.com (Nie, September 17, 1999). Companies such as Backflip,
Blink, Clip2, HotLinks and Quiver subsequently entered the online bookmark
services market over the next three years (Festa, 1999; Lawlor, 2000). Tagging
and the term social bookmarking were introduced by delicious.com (formerly
del.icio.us) in late 2003 (Mathes, 2004), and Furl (diigo.com), citeulike.org and
connotea.org were launched in 2004 (Nature Publishing Group, 2005). Since
2004, an increasing number of social bookmarking websites have been
launched and sites such as digg.com, reddit.com and newsvine.com all provide
an organisational system for “social news”. Meanwhile, IBM entered the social
software market. IBM Lotus Connections 1.0.2, which included a social
bookmarking service aimed at businesses and enterprises, was shipped on
November 16th, 2007 (Kelley, 2007). IBM Lotus Connections has several
integrated components, and a social bookmarking Dogear is included (IBM,
n.d.).
1.2 Social bookmarking
Social bookmarking is a way for Internet users to store, organise, search and
manage bookmarks of web pages on the Internet with the help of metadata
3
(Educause, May 2005). All social bookmarking services are free to use but
require users to register. Once users have registered, they can begin
bookmarking. Bookmarks can be public and shared with other users, or can be
saved privately and hidden from other users. Users can also subscribe to other
users’ bookmark lists (TVB, 2007), enabling them to find helpful bookmarks
from other users’ bookmark lists and save them into their own bookmark list.
Tags are used in social bookmarking services to organise bookmarks. The tag
cloud is a well known feature of social bookmarking services. Below is
delicious.com’s social bookmarking service tag cloud (Figure 1-1).
Figure 1-1: delicious.com tag cloud
If a tag’s font size is large and bold, then there are lots of social bookmarks
related to this tag. The smaller the font is, the fewer social bookmarks are
related to the tag. The tag cloud’s visual features communicate certain
information to users, such as which tags are popular and which tags are used
the least (Wann, September 4, 2008).
Some extra features have been added to social bookmarking services, such as
ratings and comments on bookmarks, emailing options, web annotation etc.
1.3 Social bookmark buttons
When browsing the Internet, one will find that social bookmark buttons appear
on many websites. For example, the bbc.com website shows several social
bookmark buttons under each news story (Figure 1-2).
Figure 1-2: bbc.com social bookmark buttons list (BBC, 2008)
4
Social bookmark buttons allow users to bookmark favourite web pages via an
automatic popup window. Users do not need to open their social bookmarking
service in a new window or tab to bookmark web pages.
1.4 Purpose of the study
Social bookmarking offers the user several advantages, such as user created
tag-based search resources and rank resources (Heymann, Koutrika and
Garcia-Molina, February 2008), and has become tremendously popular (Millen,
Feinberg and Kerr, 2006). In March 2008, nearly 20 million people visited the
website digg.com (Compete, 2008). In just two years (2007–2008), digg.com
had become the most popular social bookmarking website. In April 2009, nearly
38 million people had visited the website digg.com (Compete, 2009).
That being said, only 0.41% of all Internet users visited digg.com in May 2009,
while about 18% of all Internet users visited facebook.com in May 2009 (Alexa,
2009). So although the number of visitors to digg.com seems large, in term of
wider Internet usage relatively few people use social bookmarking.
Why are only a small number of people using social bookmarking? This
research investigates how socio-technical and traditional Human Computer
Interaction (HCI) factors combine to affect the usage of a socio-technical
system. While many studies have focused on social bookmarking’s functions
and features, few have addressed its most basic element – the social
bookmark button itself. This research compares and contrasts the roles
cognitive effort (an HCI factor) and social feedback (a socio-technical factor)
play in determining the likelihood of social bookmark usage. This research was
conducted by creating different versions of social bookmark buttons that
required different levels of cognitive effort and social feedback. This paper will
be of interest to social bookmark sponsors, analysts, designers and operators.
5
Chapter 2: Literature Review
2.1 Introduction
A large number of people use social bookmarking: 19,706,430 people visited
the website digg.com in March 2008 and the number of people visiting the
website stumbleupon.com increased by 579.4% from 2007 to 2008 (Compete,
2008). However, a social bookmark buttons’ usage survey was carried out on
the website doshdosh.com in 2007, and 221 participants took part in the survey.
Less than 40% of these participants used social bookmark buttons (DoshDosh,
2007).
Furthermore, another survey on the use of social web tools was carried out on
the netsquared.org website in 2006. The survey lasted for three weeks (from
April 10th to April 28th) and involved 949 participants. It was found that 41% of
the participants used photo sharing, 37% of the participants used social
calendaring, and just 13% of the participants used social bookmarking
(NetSquared, 2006). These results suggest that social bookmarking is the least
widely used of all the social web tools. Another survey on social bookmarking
usage found similar results in 2008. This survey was launched on the website
pcpitstop.com in 2008, and 1,073 people took part. Just 13.4% of these
participants had ever used an online social bookmarking service (PCPitstop,
August 01, 2008).
Some of the factors that affect social bookmarking usage are discussed in this
chapter. This chapter also reveals some of the reasons why only a small
number of people are using social bookmarking.
2.2 A brief literature survey
In order to study the factors that affect social bookmarking usage, social
bookmarking must first be defined.
6
Social bookmarking involves saving bookmarks to a public website and using
keywords to tag them, so that visitors can search and retrieve resources using
keywords, usernames and tags (Educause, May 2005). Users provide
metadata via social bookmarking, and social bookmarking services have added
extra features, such as rating and comment options, emailing options, web
annotation etc. (Educause, May 2005).
2.2.1 The tag feature
Each social bookmark can have several tags entered by users in social
bookmarking services. Multiple tags can describe the social bookmark in more
than one domain and can act as short, free-form labels (Sen et al., 2006; Millen,
Feinberg and Kerr, 2006). Moreover, tags can help users remember and
manage information, and can also be powerful tools for discovering and
sharing new information (Sen et al., 2006; Millen, Feinberg and Kerr, 2006).
Because current social bookmarking services have used tags, tags are a key
reason current social bookmarking services have had greater success than
their 1990s equivalents (Millen, Feinberg and Kerr, 2005; Dugan et al., 2007).
IBM used the sharing and tagging features of social bookmarking services to
design and develop an enterprise-scale social bookmarking service called
Dogear. Dogear’s potential includes improving “information sharing, expertise
location, and support of communities of interest within the enterprise” (Millen,
Feinberg and Kerr, 2005, p.35). An enterprise could use the Dogear social
bookmarking service to achieve individual, collaborative and organisational
goals by creating records in the form of {user, resource, tag} (Dugan et al.,
2007). The resource is a URL, and the tag is a word or phrase describing the
resource.
Furthermore, in order to help people understand a social bookmarking service’s
characteristics, six tag metrics have been proposed (Farooq, Kannampallil, et
al., 2007); two of these describe designs that could enhance social
7
bookmarking services (Farooq, Song, et al., 2007).
The six tag metrics are “tag growth, tag reuse, tag non-obviousness, tag
discrimination, tag frequency, and tag patterns” (Farooq, Kannampallil, et al.,
2007, p.351). These metrics are used to evaluate the tagging behaviour of
social bookmark users and could be used as design heuristics to implement a
social bookmarking service such as CiteSeer. Moreover, linking tag growth with
tag reuse from the six tag metrics can provide a direct explanation of how often
users rehash tags in a social bookmarking service. If a social bookmarking
service has low tag growth and high tag reuse, its users do not create new tags;
instead they just recycle previous tags (Farooq, Song, et al., 2007).
In order to make sure a social bookmarking service has high tag growth and
low tag reuse, tag quality needs to be improved. Descriptions of the resource’s
words or phrases need to be clear and correct, so that the right vocabulary is
put at an important position.
However, tag duplication does occur on social bookmarking sites. The same
idea described differently by users that makes high tag reuse, and users speak
different languages would affect the value of other users’ tags (Sen et al., 2006).
Moreover, the barriers to users adding social bookmarks are low, so any user
can add tags into a social bookmarking service, which can result in inconsistent
or otherwise poorly-used tags (Educause, May 2005). Because there is no
standard set of controlled vocabulary and no standard for the structure of tags
in a tagging system, tags with spelling errors, double meanings and unclear
synonym/antonym use can occur with social bookmarking services (Educause,
May 2005; Guy and Tonkin, January 2006). It is necessary to correct “sloppy
tags” to increase a tagging system’s effectiveness (Guy and Tonkin, January
2006).
“Sloppy tags” also could affect users’ ability to discover useful resources using
social bookmarking services.
8
2.2.2 The filter feature
Social bookmarking services operate using three main processes, which are
the storing, managing and retrieving of bookmarks (Educause, May 2005;
Bateman, Muller and Freyne, 2009). The storing and managing processes are
related to tagging, while the retrieving process is connected to filter and
metadata conceptions. Information filtering or information discovery can be
accomplished through search and pivot browsing (Bateman, Muller and Freyne,
2009; Millen, Feinberg and Kerr, 2006).
2.2.2.1 Search method
The search method is one way that filtering can be achieved. It uses search
boxes to find tags or usernames that match the user’s queries (Bateman,
Muller and Freyne, 2009). After users store a social bookmark on a social
bookmarking service, other users can find that social bookmark using the
search method to find the tag (Millen, Feinberg and Kerr, 2006).
There are many advantages to using social bookmarking data. The main
advantages are the “high dynamics, attached metadata, available temporal and
sentiment information”, so data from social bookmarking services can be
exploited to enhance web searches (Yanbe, Jatowt, Nakamura and Tanaka,
2007, p.115). Moreover, social bookmarking is a current phenomenon, which
possesses the potential to provide a mass of data about web pages that are
actively updated and prominent in search results, and tags are overwhelmingly
relevant and objective. For instance, the annotated bookmark’s tags can be a
useful data source that can be harnessed to improve web searches (Bao, Xue,
Wu, Yu, Fei and Su, 2007; Heymann, Koutrika and Garcia-Molina, February
2008; Yanbe, Jatowt, Nakamura and Tanaka, 2007) or web page classifications
(Golder and Huberman, 2006).
However, social bookmarking cannot improve web searches at this stage
(Heymann, Koutrika and Garcia-Molina, February 2008). One social
bookmarking service, such as delicious.com, can only create small amounts of
data when compared to the web’s scale, and tags cannot annotate resources
9
such as URLs correctly (Heymann, Koutrika and Garcia-Molina, February
2008). For these reasons, URLs created by social bookmarking services are
unlikely to be numerous enough to impact major search engines, and tags
created by social bookmarking services are unlikely to be much more useful
than a full text search accentuating web page titles (Heymann, Koutrika and
Garcia-Molina, February 2008).
As mentioned earlier, social bookmarking is new and having only been around
for a few years, it is still evolving. Over the next several years, social
bookmarking may rapidly reach the current web’s scale, and user interface
features may improve tag quality (Heymann, Koutrika and Garcia-Molina,
February 2008). At that time, social bookmarking may be able to improve web
searches.
Social bookmarking services’ small amounts could effectively enhance web
searches. Moreover, “sloppy tags” could affect users’ ability to discover
information using social bookmarking services.
An experimental site ReMarkables was used to evaluate social bookmarking
services’ effectiveness for information discovery; this site lasted for 10 days
from May 11th to 21st in 2007 (Klaisubun, Kajondecha and Ishikawa, 2007). A
user questionnaire survey asked participants about the effectiveness of the
social bookmarking service for information discovery. The results can be seen
in the below diagram.
10
Figure 2-1: Information discovery in social bookmarking service (Klaisubun, Kajondecha and Ishikawa, 2007)
The percentages represent how often each transition was used; the results
were that:
l Almost half of the participants (42.6%) copied a bookmark from a user
page (by people)
l 31.9% of the participants found a bookmark from a tag page (by content)
l Just 25.5% of the participants copied a bookmark from the search result
page
This experiment also found out that 60% of the participants think that being
able to search by tags is the main benefit of using social bookmarking services
(Klaisubun, Kajondecha and Ishikawa, 2007). This highlights that tags are an
important part of social bookmarking services, and providing effective tags can
improve social bookmarking services’ usefulness. “Sloppy tags” affect users’
ability to discover useful resources using social bookmarking services.
Moreover, there are two options to improve the efficiency of social bookmarking
services’ navigation function. These two options are that “the navigational
function should provide sufficient information about tags attached with each
bookmark”, and that it “should provide social presence cue of other users in
11
order to judge which other’s library is appropriate for the user” (Klaisubun,
Kajondecha and Ishikawa, 2007, p.787). Improving the navigation function
would enable more efficient usage of social bookmarking services for
information discovery (Klaisubun, Kajondecha and Ishikawa, 2007).
Other research has focused on social bookmarking service’s search results,
and it has been suggested definition of purpose tagging in social bookmarking
service’s search area.
A fundamental problem was found when users searched resources in tagging
based social bookmarking services. The search terms are usually different from
the terms entered as tags by users (Heymann, Koutrika and Garcia-Molina,
February 2008). For example, a user might search “top 10 social bookmarking
services”, but the keywords “top 10” and “social bookmark services” are not
included in tag lists. This problem is referred to as the “gulf of execution”, which
means the cognitive gap between a system’s functionality and a user’s goals
(Norman, 1990). Moreover, the most commonly used tags in social
bookmarking services focus on representing content (Golder and Huberman,
2006; Yanbe, Jatowt, Nakamura and Tanaka, 2007) rather than intent.
For this reason, purpose tagging was introduced. Purpose tagging focuses on
intent (“what it can be used for”) rather than content (“what it is”), and can
exchange a term of user intent with a term of content and tags that are provided
by social bookmarking services (Markus, 2008). Moreover, purpose tagging
facilitates goal-oriented social bookmarking searches (Markus, 2008).
2.2.2.2 Pivot browsing method
The other way to filter is pivot browsing (Bateman, Muller and Freyne, 2009;
Millen, Feinberg and Kerr, 2006). Pivot browsing is an interaction technique,
and can explore, discover, and refine bookmark lists easily after users click
filter terms. Pivot browsing is a simple and expedient way to find information
(Bateman, Muller and Freyne, 2009; Millen and Feinberg, 2006). For example,
if a user wants to find bookmarks with the tag “social”, he/she can click “social”
in a tag cloud, and all bookmarks with the tag “social” will be displayed. If the
12
user wants to find more bookmarks with the tag “social” from other users, the
user can click one username from the bookmark list. The below figure shows
pivot browsing’s working theory:
Figure 2-2: Pivot browsing’s working theory
However, if the user wants to find bookmarks with the tag “social” from another
user’s bookmark lists, there is another step. This step is a refinement step. A
refinement step is “the selection of each filtering criterion”, and bookmark lists
are displayed depending on the current criteria (usernames or tags) (Bateman,
Muller and Freyne, 2009, p.92; Millen and Feinberg, 2006; Millen, Feinberg and
Kerr, 2006). For this reason, users have to take multiple steps to reduce
bookmark lists to a tractable size, and then find the right one (Bateman, Muller
and Freyne, 2009).
Moreover, an overload of information also affects users’ ability to find the right
social bookmark (Bateman, Muller and Freyne, 2009). Social bookmarking
services suffer from issues created by enormous bookmark collections and a
high percentage of invalid bookmarks (Bateman, Muller and Freyne, 2009;
13
Cockburn and McKenzie, 2001). Because of the gigantic social bookmarks in
social bookmarking services, users have to take several steps to find their
desired bookmarks using pivot browsing; 70% of successful retrievals are
accomplished via multiple steps (Bateman, Muller and Freyne, 2009).
Social bookmarking services’ ordering method is another factor that affects
users’ ability to find information (Bateman, Muller and Freyne, 2009). Each
social bookmark has four components: a URL, a username, a tag(s), and a
timestamp. Tags and usernames can be used as filtering criteria, and URLs
and timestamps can be used in ordering metrics (Bateman, Muller and Freyne,
2009). Most social bookmarking services’ default ordering metrics display
social bookmarks in the most recent (date-based) order, the most frequent
(popularity) order (Bateman, Muller and Freyne, 2009) or using the hybrid
method (Dover, 2008). The hybrid method displays social bookmarks in order
of the most recent and the most frequent order. However, with the general
ordering methods it is not possible to list relevant bookmarks at the top of
bookmark lists for users (Bateman, Muller and Freyne, 2009), and users
typically choose bookmarks from the top of bookmark lists (Keane, O’Brien and
Smyth, 2008). One researcher found that users selected the top bookmark over
70% of the time, with the second bookmark being clicked only 10% of the time
(Keane, O’Brien and Smyth, 2008). This shows the importance of displaying a
relevant bookmark at the top of bookmark lists and the importance of display
bookmarks in an order that is suitable for users.
How can pivot browsing help users find relevant bookmarks easily and quickly?
Web search engines can check relevance based on web-scale measures, but
social bookmarking services only produce small amounts of data on the scale
of the web (Heymann, Koutrika and Garcia-Molina, February 2008), making it
difficult to find relevant bookmarks in social bookmarking services (Bateman,
Muller and Freyne, 2009). However, recent research has focused on providing
personalisation to improve search results’ relevance by monitoring user
communities (Freyne, Smyth, Coyle, Balfe and Briggs, 2004), querying
predefined groups (Liu, Yu and Meng, 2002), and monitoring search and
browsing patterns to recommend and provide relevant information (Farzan,
14
Coyle, Freyne, Brusilovsky and Smyth, 2007). To improve social bookmark lists’
relevance, one researcher focused on a personalised ordering algorithm. The
personalised ordering algorithm’s aim is to reduce the refinement steps needed
and improve relevant bookmarks’ display positions by paying attention to an
individual’s personal choices – this focuses on “an individual searcher’s
previous actions as an alternative to community-based reordering metrics”
(Bateman, Muller and Freyne, 2009, p.92).
2.2.3 The ranking feature
Social bookmarking has lots of benefits, one of which is that it allows users to
rank resources (Heymann, Koutrika and Garcia-Molina, February 2008). The
ranking feature uses the number of users that have bookmarked a web page to
measure the authoritativeness of the web page (Chen, Scripps and Tan, 2008).
Voting or rating web pages is useful feedback for this feature (Bian, Liu,
Agichtein and Zha, 2008a). Moreover, the ranking feature is the core of an
effective search (Bian, Liu, Agichtein and Zha, 2008a). The number of votes for
a web page can show the popularity of the web page, which can help with a
web search (Bao, Xue, Wu, Yu, Fei and Su, 2007).
However, the quality of users’ feedback needs to be considered. While social
bookmark services are popular and any users can vote or rate bookmarks
(Educause, May 2005), not all users’ votes or rates are reliable (Bian, Liu,
Agichtein and Zha, 2008a). A small portion of malevolent users try to “game the
system” by selectively promoting (thumb up) or demoting (thumb down)
bookmarks for profit or fun; these bad or fraudulent votes or rates are known as
vote spam (Bian, Liu, Agichtein and Zha, 2008a).
In order to provide an effective ranking of resources for users, there have been
some studies of vote spam. Social bookmarking services are one sort of online
social media (Bian, Liu, Agichtein and Zha, 2008b) and some of this research
has addressed social media’s vote spam problem. One study discussed how to
change a currently presented algorithm for ranking social media so that it would
15
recognise vote spam (Bian, Liu, Agichtein and Zha, 2008b); this study
introduced a machine learning based ranking framework for social media (Bian,
Liu, Agichtein and Zha, 2008a).
Furthermore, vote spam affects people’s ability to trust social bookmarking
services. There are bidirectional effects by trust and rating, and the bidirectional
interaction consists in social booking services (Matsuo and Yamamoto, 2009).
Other users’ ratings of a bookmark have an effect on users’ behaviour; whether
another user will rate the same bookmark or not depends on trust (Matsuo and
Yamamoto, 2009). How does one create trust between users? Friendships are
built on trust, so users might exchange information online with their friends
(Subramani and Rajagopalan, 2003). Moreover, users are influenced by people
they trust (Forman, Ghose and Wiesenfeld, 2008) and even just one trust
person (Leskovec, Adamic and Huberman, 2007), and trust is also transitive
(Guha, Kumar, Raghavan and Tomkins, 2004). For example, user A trusts user
B, and user B trusts user C, so user A can trust user C. Trust can affect ratings,
and vice versa. Similar ratings also can induce trust among users (Golbeck,
2009; Ziegler and Golbeck, 2007). If vote spam exists in social bookmarking
services, it will affect how much users trust these services and how much they
use them.
2.2.4 Usability and functionality
As mentioned above, social bookmarking’s main features have been analysed,
and each feature’s status quo has been summarised below:
Table 2-1: Summary of social bookmarking’s main features Features Advantages Disadvantages
Tag
l Describes the social bookmark in more than one domain
l Helps users to remember, manage, discover and share information
l Tag duplication (“Sloppy tags”)
l No standard set of controlled vocabulary
l Low barriers to users
Filter Search l Social bookmarking data are
high dynamics, attached metadata, available
l Small amounts of data on the scale of the web
l Tags cannot annotate
16
temporal and sentiment information
l The annotated bookmark’s tags can be a useful data source
resources such as URLs correctly
l “Sloppy tags” affect users’ ability to discover useful resources
l Search results are usually not relevant
Pivot
Browsing
l Can explore, discover and refine bookmark lists easily after users click filter terms
l Is a simple and expedient method to find information
l Default ordering metrics display social bookmarks in order of the most recent and the most frequent order
l Multiple refinement steps
l Ordering method affects users’ ability to find information
l General ordering methods do not list the relevance of bookmarks for users
Rank
l Uses the number of users who have bookmarked a web page to measure the authoritativeness of the web page
l Is core to an effective search in social bookmarking services
l Not all users’ votes or rates are reliable and the users’ feedback downgrades
l Vote spam problem l Affects users’ degree to
trust
It is clear from the summary table that social bookmarking services provide
some features that allow users to access bookmarks conveniently, but that they
also have some defects. Some of these defects are the multiple refinement
steps and the way the ordering method affects the pivot browsing feature, the
way the vote spam affects the ranking feature, and the lack of a set tagging
standard, all of which affect the search feature. All of these problems make the
features hard to use, and thus affect social bookmarking usage.
These findings can be analysed using the usability and functionality evaluation
criteria. Usability and functionality were added as a technology acceptance
criterion by TAM (Technology Acceptance Model) (Davis, 1989), which is from
the origin – the theory of reasoned action (Fishbein and Ajzen, 1975). Initially,
usability and functionality were named PEOU (perceived ease of use) and PU
(perceived usefulness) (Davis, 1989). TAM made PEOU and PU relatively
distinct criteria (Hix and Schulman, 1991) and used them to map usability and
functionality (Whitworth, Banuls, Sylla and Mahinda, 2008). Usability and
functionality are also used as evaluation criteria by the WOSP (Web of System
Performance) model. The WOSP model suggests eight evaluation criteria:
17
functionality, extendibility, connectivity, flexibility, usability, security, privacy, and
reliability. The WOSP evaluation is more accurate and complete than the TAM
evaluation for complex sociotechnical software (Whitworth, Banuls, Sylla and
Mahinda, 2008).
What are usability and functionality? What is the relationship between usability
and functionality? Usability is used to check whether a system is easy to use or
not – if it is easy to learn and can be used to accomplish tasks easily and
quickly (Whitworth, Banuls, Sylla and Mahinda, 2008). On the other hand,
functionality is used to check whether a system is useful or not – whether it can
improve productivity, performance and effectiveness (Whitworth, Banuls, Sylla
and Mahinda, 2008). Moreover, the relationship between usability and
functionality must be one of equivalence and tension (Whitworth, Banuls, Sylla
and Mahinda, 2008). For example, if usability is improved, functionality will be
reduced. If a system’s usability and functionality are equal, then the system is
mature.
Now, social bookmarking services’ problems can be explained using the
usability and functionality evaluation criteria. The functionality of the pivot
browsing feature, for example, is good because it offers an expedient way of
finding information. However, the multiple refinement steps and ordering
method affect the pivot browsing feature, and these defects make the task of
finding information difficult. Because usability and functionality are not equal
and the functionality value is more than the usability value, these services are
immature and this affects the services’ usage.
Some researchers believe that users pay attention to social bookmarking
service’s usability and functionality. When evaluating a social bookmarking
service, users decide whether the system is good or not based on eight criteria:
ease of use, group features, page annotation, page caching, support,
popularity and longevity, export options and multi-tool bookmarking (Stanford,
2007). Of these eight criteria, ease of use is usability, and group features, page
annotation, page caching, support, popularity and longevity, export options, and
multi-tool bookmarking are functionality. Another view is that users focus on
18
three major factors, namely features, interface and tools (Kirkpatrick, 2006). Of
these factors, interface is usability, and features and tools are functionality.
Some researchers also believe that social bookmarking services’ usability and
functionality affect the systems’ usage. On an idiographic social bookmarking
service such as reddit.com, users pay attention to the system’s interface,
functions, features and submitting speed when using it (Florczak, 2007). From
these, the interface and submitting speed are usability, and functions and
features are functionality. However, the reddit.com interface is ugly, the
functions and features are confusing, and the submitting speed is slow. For
these reasons, users consider reddit.com to be messy and boring and do not
want to use it.
2.2.5 Three relevant groups
The earlier analysis focused on social bookmarking services themselves. Now,
the focus is changing to people, and different people’s tasks on social
bookmarking services will be analysed. Some of the factors that affect social
bookmarking usage will be discussed.
There are three relevant groups, which face of Human-Computer Interaction
and relate with social bookmarking services evaluation criteria (Grudin, 2005).
These groups are the designers (who create social bookmarking services), the
managers (who fund social bookmarking services), and the users (who use
social bookmarking services) (Grudin, 2005).
2.2.5.1 The designers - interface factors
The designers need to know what users want from social bookmarking services
(Grudin, 2005). How can these services meet users’ demands? One survey
was launched on the website pcpitstop.com in 2008, and was completed by
1,073 participants. The survey found that 18.2% of participants did not
understand how social bookmarking services work (PCPitstop, August 01,
2008). In order to answer the aforementioned questions, the designers need to
19
know three things:
l What social bookmarking is, which features should be included (these
details were discussed in previous chapters), and what the end users
want (Whitworth, Banuls, Sylla and Mahinda, 2008).
l What sort of users will use the service. The target users and the users’
experience (such as the users’ skills/knowledge, cultural dimensions,
habits and tasks) should be considered (“User Centered Design” –
UCD). A design based on users’ experience will create a successful
and effective website (Chou, 2002; Ford and Gelderblom, 2003; Watrall
and Siarto, 2009).
l How to let users liking to use a social bookmarking service. Firstly, we
need to know how to measure users’ performance. Speed, accuracy,
training time and satisfaction can be used to measure users’
performance (Bailey, 1996). To please users, designers need to make
sure they can use features “efficiently, effectively, and satisfactorily”
(named as “usability techniques”) (Chou, 2002).
How to design an easy to use social bookmarking service? Firstly, web
interfaces are one of the main factors. Below is a conceptual model of web
interfaces:
Figure 2-3: Conceptual model of web interfaces (Ivory and Megraw, 2005, p.468)
Looking at Figure 2-3, it is not very hard to find that:
l Information, navigation, and graphic design are at the basic level of
web interface design and are comprised of text, links, and graphic
design.
20
l Experience design like site architecture design is on the top level of
web interface design.
l Web interface design is divided into three parts – the element level,
page level and the site level; the element level is basic and the site
level is complex.
Some of the details that are included in information, navigation and graphic
design are listed below:
l Text:
n Font Styles and Sizes:
Using different font styles can affect users’ feelings and emotions; it
can also make text more readable (Watrall and Siarto, 2009). Some
studies suggest using serif typefaces (e.g. Times New Roman),
sans serif typefaces (e.g. Arial), or both; serif typefaces can be
used for larger text (such as large headings) (Bernard and Mills,
2000; Bernard, Liao and Mills, 2001; Nielsen, 2000; Schriver, 1997;
Watrall and Siarto, 2009). Moreover, sans serif typefaces are the
most commonly used font style for good looking pages, and sans
serif typefaces are more legible than serif typefaces online. Sans
serif typefaces can be displayed in a smaller size more safely and
properly than serif typefaces (Ivory and Megraw, 2005; Watrall and
Siarto, 2009). The font sizes that are recommended are from 9pt to
14pt, and larger font sizes are suggested for older users (Ivory and
Megraw, 2005).
l Links:
n Length of Link Text:
Some studies suggest using 2-4 words for link text (Nielsen, 2000;
Ivory and Megraw, 2005) while others suggest using 7-12 useful
words (Sawyer and Schroeder, 2000).
n Number and Types of Links:
Some studies suggest using minimum links, avoiding graphic,
repeated or within-page links, and providing multiple links to the
same content in different forms (such as text, text with a graphic, or
graphics) (Sano, 1996; Flanders and Willis, 1998; Spool, Scanlon,
21
Schroeder, Snyder and DeAngelo, 1999; Spool, Klee and
Schroeder, 2000; Nielsen, 2000; Sawyer and Schroeder, 2000;
Scanlon and Schroeder, 2000). For example, the most popular
“diggs” bullets draw users’ attention, and link to the story on the
website digg.com (Watrall and Siarto, 2009) (see Figure 2-4).
Figure 2-4: “diggs” Bullets (Digg, 2010) l Graphics:
n Number and Types of Graphics:
In order to improve download speed, graphics should be minimised.
Although some people have broadband connections (Madden and
Rainie, 2003), most users still use low speed connections
(Harwood and Rainie, 2004). Moreover, some types of graphics
should be avoided, such as images containing text, navigation
images and animated images (Flanders and Willis, 1998; Nielsen,
2000; Scanlon and Schroeder, 2000).
l Colour:
Colour is “the unsung hero” of website design; a good colour palette
can pull users into the site. Colour is as important as text,
links/navigation and graphics (Watrall and Siarto, 2009, p.xii). Some
studies suggest using few colours, browser-safe colours, adequate
contrast colours and default link colours (Flanders and Willis, 1998;
Kyrnin, 2006; Murch, 1985; Nielsen, 2000; Spool, Scanlon, Schroeder,
Snyder and DeAngelo, 1999; Watrall and Siarto, 2009). A good web
page tends to use up to three colours for headings, one to four
high-contrast colours for body text and two to five colours for links
(Ivory and Megraw, 2005). Different colours impact users’ emotions
differently (Watrall and Siarto, 2009; McNeil, 2008). For example, red is
exciting, yellow is cheerful, purple is stately and blue is formal; the
blue-and-green combo is a trusty standby – it looks great, it is safe and
it is the most conservative and attractive colour combination (Watrall
and Siarto, 2009; McNeil, 2008). How to use different colours and let
22
them work together well? Using the triadic colour scheme often works
well, which is why it is one of the most commonly used colour schemes
(other, less common schemes are monochromatic, analogous,
complementary, and tetradic) (Watrall and Siarto, 2009). Firstly,
designers should choose a base colour, and then they should use the
triadic scheme to find two other colours (Watrall and Siarto, 2009). For
example, if the base colour FF2823 (red) is chosen, the other two
colours (yellow and blue) can be found using the scheme (see Figure
2-5).
Figure 2-5: The triadic colour scheme (Kuler, 2010)
Knowing what users want from social bookmarking services and how to design
an easy to use social bookmarking service helps to explain some of social
bookmarking services’ problems; it also helps reveal some of the factors that
affect social bookmarking usage.
l Colour:
The website digg.com’s main colours are blue, green and red, and the
website delicious.com’s main colours are gray, blue and black (see
Figure 2-6). As mentioned earlier, the blue-and-green combo is known
to be attractive and the colour blue is formal. More users used the
website digg.com (39,826,655) than the website delicious.com
(1,179,562) in January 2010 (Compete, 2010) (see Figure 2-7). Colour
might be a factor affects social bookmarking usage in this case.
l Redesign Web Interface:
The website delicious.com was redesigned after the website name was
changed from del.icio.us, and the new website was launched on July
31st, 2008 (Arrington, September 6, 2007; Delicious, July 31, 2008).
The main changes related to:
u Speed: a new infrastructure and platform makes every page
faster and ensures that the new site is more responsive and
23
reliable (Delicious, July 31, 2008).
u Search: an improved search engine ensures it is faster, more
powerful and more effective (Delicious, July 31, 2008).
u Design: a redesigned user interface ensures ease of use and
provides more features (such as navigation, tag bar,
bookmarks, sidebar, and action box) (Delicious, July 31, 2008;
Arrington, September 6, 2007). Moreover, the new interface is
easier to learn than the old version (Delicious, July 31, 2008).
After the website delicious.com was redesigned, the number of
individual visitors increased from 207,503 (July, 2008) to 1,595,342
(August, 2008) and continued to grow (Compete, March, 2009). This
finding suggests that the web interface affects social bookmarking
usage.
Figure 2-6: Compare digg.com and delicious.com’s main colours
24
Figure 2-7: Compare digg.com and delicious.com’s unique visitors (Compete, 2010)
Finally, the principles of interface design should be applied to design an easy to
use social bookmarking service. The interface design should be as a vinculum
between users and information (Beaird, 2007), and provide an easy way to let
users retrieve their desired information (Sklar, 2009). Users do not want to
spend a long time on scanning a page for some information they want. While
79% of online users usually or always scan for information on a page, only 16%
of online users read each word on a page (Underwood, 2001). If users cannot
find information they are looking for, they will close the page and open another
site. In order to make sure an interface easy to use and help users find their
desired information quickly, information should be made for easier on-screen
reading by the following: break information into smaller pieces, head difference
information, link difference information with hypertext, keep pages to one
screen, so using the scroll bar is avoided (Sklar, 2009; Kyrnin, n.d.; Underwood,
2001; Ivory and Megraw, 2005).
2.2.5.2 The managers - social factors
Managers should know what makes users accept a new social bookmarking
service (Grudin, 2005). What is a good way to make users accept a new social
bookmarking service? A survey was launched on website pcpitstop.com in
2008, and 1,073 people took part. The survey found that 35.1% of participants
did not see any value in social bookmarking services (PCPitstop, August 01,
2008). In order to put more value in a social bookmarking service, the social
bookmarking service should be a high quality one that might make users
accepts. However, it requires that users notice the new social bookmarking
25
service. If users have not noticed and used the new social bookmarking service,
how can this be encouraged? As a manager, the second question should be
considered. Sociology can help to answer this second question:
l The Hundredth-Monkey Effect:
The Hundredth-Monkey Effect was published as a foreword (written by
Lyall Watson) to the book “Rhythms of Vision: The Changing Patterns
of Belief” in 1975 (Blair, 1975), and it was further popularised in the
book “The Hundredth Monkey” (Jr., 1984). The Hundredth-Monkey
Effect is from the following story: scientists observed the Macaca
fuscata (the Japanese monkey) in the wild for over 30 years (from
1952), and observed one 18 month old female monkey (named Imo)
washing sandy sweet potatoes in a stream to make the sweet potatoes
more delicious; over a short period this skill spread over the whole
island of Koshima, as the other monkeys learned to wash the sandy
sweet potatoes; eventually even colonies of monkey on other islands
and the mainland troop of monkeys at Takasakiyama also washed the
sweet potatoes (Jr., n.d.). The first monkey created an ideological
breakthrough, and the new awareness was communicated from mind
to mind instantaneously once a certain critical number achieved the
awareness (Blair, 1975; Jr., 1984; Jr., n.d.).
l The Tipping Point:
The Tipping Point also mentions a certain critical number (or a critical
point), and a previously rare phenomenon becoming dramatically and
rapidly more common, and a little change in a system having a big
effect (Gladwell, 2000). The Tipping Point was created by Morton
Grodzins, and was popularised in daily life by Malcolm Gladwell’s book
“The Tipping Point: How Little Things Can Make a Big Difference”
(Gladwell, 2000). In this book, the three rules of epidemics are
introduced:
u The Law of the Few:
This is the 80/20 principle, which is the idea that 80% of the
work will be done by 20% of the participants; the participants
play a critical role in word of mouth (the most important form of
user communication) epidemics. The main communicators on
26
the web are Connector (named social glue, who know lots of
people in different area), Mavens (named data banks, who
accumulate knowledge from different area, and share and
trade the knowledge), and Salesmen (persuaders, who
persuade users to convince what the users heard) (Gladwell,
2000).
u The Stickiness Factor:
This is a specific method of making a contagious message
memorable, so the message sticks in users’ memories and
cannot be gotten rid of (Gladwell, 2000).
u The Power of Context:
This shows that users are much more sensitive to their
environment than they saw (Gladwell, 2000).
From these two terms (The Hundredth-Monkey Effect and The Tipping Point),
managers should know how to make users accept a new social bookmarking
service. Managers should find people to act as connectors, mavens and
salesmen so that more people notice the new social bookmarking service.
Meanwhile, managers should create a contagious message that sticks in users’
minds. The power of context also should be considered.
Knowing how to make users accept a new social bookmarking service and
what the managers need to do to popularise the service should help solve
some of the services’ problems and reveal some of the factors that affect social
bookmarking usage. A social bookmark buttons usage survey was carried out
on the website doshdosh.com in 2007 and 221 people took part. The survey
found that there are lots of social bookmark buttons and some users do not
know which is the best; also some users add a bookmark using several social
bookmark buttons (DoshDosh, 2007). The survey suggested users stick to the
best social bookmark button, but the question is which social bookmark button
is the best? Managers need to ask connectors, mavens and salesmen to
propagandise the new social bookmarking service, so that more people notice
and come to trust the new social bookmarking service. The managers also
need to find the new social bookmarking service’s point of difference, and make
27
it memorable. Sociology might be a factor that affects social bookmarking
usage.
2.2.5.3 The users - feedback factors
The users need to know how to get a social bookmarking service to do what
they need (Grudin, 2005). How can users get a social bookmarking service to
do what they need?
Several surveys have assessed whether users have problems using social
bookmarking services. A social bookmark buttons usage survey was carried
out on website doshdosh.com in 2007 and 221 people took part. The survey
found that 19% of participants did not know what a social bookmark button was
and about 45% of participants had never used one (DoshDosh, 2007). Another
survey was launched on the website pcpitstop.com in 2008, which attracted
1,073 participants. This survey found that 46.7% of participants did not know
social bookmarking services existed, and 86.6% of participants had not used
an online social bookmarking service (PCPitstop, August 01, 2008). These
results show that the problems of noticing a social bookmarking service exist
and using the social bookmarking service among users.
How to help users notice and use a social bookmarking service to do what they
need?
l Attractiveness:
Attractiveness will cause users to notice social bookmarking services
when they see them (Grudin, 2005). Attractive colours will attract users’
attention (McNeil, 2008).
l Sociality and Sharing
One of social bookmarking’s characteristics is that it is social and users
can share information online (Hines, 2009; Sontag, 2009; Chen,
Scripps and Tan, 2008). To find information, users prefer to browse
others’ libraries (user page) to find useful new resources (Klaisubun,
Kajondecha and Ishikawa, 2007). Moreover, the “Add Friends” and
“Send” features of social bookmarking services also help users to get a
social bookmarking service to do what they want. The “Add Friends”
28
feature of the website digg.com allows users to add friends from their
email contact list (such as Hotmail, Gmail, and so on), and the “Send”
feature of the website delicious.com allows users to send social
bookmarks to friends (Digg, 2010; Delicious, August 7, 2009).
2.2.6 Culture
Culture might be one of the factors affecting social bookmarking usage. The
website compete.com found that there were more users on the website
digg.com than the website delicious.com in 2009, and the unique visitor
numbers of the website digg.com were 36–37 times than that of delicious.com
from March to April in 2009 (Compete, 2010). Moreover, another web analytics
company (alexa.com) also found the same results. Over the past couple of
years (from 2009 to 2010), the daily number of visitors to digg.com has been
much higher than that of delicious.com (Alexa, 2010a).
However, the two websites’ usage numbers are different in the different
countries. An online survey started on March 4th, 2009 and finished on April 29th,
2009 on the website livlarge.co.nz, found that 46.2% of New Zealand
participants used the website delicious.com and 30.8% of New Zealand
participants used the website digg.com (LivLarge, 2009). Moreover, a
worldwide traffic rank can also show that different countries have different
usage statistics. In Spain, the website delicious.com’s worldwide traffic rank
was 452 and the website digg.com’s was 484 on February 20th, 2010 (Alexa,
2010b; Alexa, 2010a). The same usage phenomenon also happened in Japan.
The worldwide traffic rank of the website delicious.com was 1195 and that of
the website digg.com was 1268 on the same date (Alexa, 2010b; Alexa, 2010a).
Based on these findings, culture might be a factor that affects social
bookmarking usage.
29
2.3 Variables
After reviewing the literature on social bookmarking services’ main features and
analysing the relevant people related with social bookmarking services, some
of the potentially significant factors that affect social bookmarking were found.
Relationship 1: easy/hard to learn, easy/hard to use, complete task time taken,
and web interface design – these factors all affect the relationship between
usability and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: usability
Moderating Variable: easy/hard to learn, easy/hard to use, complete task
time taken, and web interface design
Relationship 2: useful/useless, productivity, performance, effectiveness, and
features/tools – these factors all affect the relationship between functionality
and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: functionality
Moderating Variable: useful/useless, productivity, performance,
effectiveness, and features/tools
Relationship 3: font style, font size, link text length, link number, link type,
graphic number, graphic type, and colour – these factors all affect the
relationship between web interface design and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: web interface design
Moderating Variable: font style, font size, link text length, link number, link
type, graphic number, graphic type, and colour
Relationship 4: colour, link type, social bookmark button, and share feature –
these factors all affect the relationship between attractiveness and use of social
bookmarking:
Dependent Variable: use of social bookmarking
30
Independent Variable: attractiveness
Moderating Variable: colour, link type, social bookmark button, and share
feature
Relationship 5: the word of mouth epidemic, the power of context, and the
stickiness message – these factors all affect the relationship between
application in sociology and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: application in sociology
Moderating Variable: the word of mouth epidemic, the power of context,
and the stickiness message
Relationship 6: culture, skills, knowledge, and habit – these factors all affect
the relationship between the target user experiences and use of social
bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: the target user experiences
Moderating Variable: culture, skills, knowledge, and habit
Relationship 7: share feature and share behaviour – both of these factors
affect the relationship between stickiness and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: stickiness
Moderating Variable: share feature and share behaviour
Relationship 8: tag quality, bookmark quality, scale of the web, barriers to
users, standard set of tags, search results quality, search method, and tags’
focus on intent/content – these factors all affect the relationship between
search by tags and use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: search by tags
Moderating Variable: tag quality, bookmark quality, scale of the web,
barriers to users, standard set of tags, search results
31
quality, search method, and tag’s focus on
intent/content
Relationship 9: rank quality, vote quality, trust, rank framework, and presented
algorithm – these factors all affect the relationship between search by rank and
use of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: search by rank
Moderating Variable: rank quality, vote quality, trust, rank framework, and
presented algorithm
Relationship 10: refinement steps, order method, and bookmark quality –
these factors all affect the relationship between view by pivot browsing and use
of social bookmarking:
Dependent Variable: use of social bookmarking
Independent Variable: view by pivot browsing
Moderating Variable: refinement steps, order method, and bookmark
quality
Relationship 11: tag quality, tag annotation, and vocabulary – these factors all
affect the relationship between social navigation by content tags and use of
social bookmarking
Dependent Variable: use of social bookmarking
Independent Variable: social navigation by content tags
Moderating Variable: tag quality, tag annotation, and vocabulary
Relationship 12: information about social others – this factor affects the
relationship between social navigation by user tags and use of social
bookmarking
Dependent Variable: use of social bookmarking
Independent Variable: social navigation by user tags
Moderating Variable: information about social others
32
33
34
Figure 2-8: Variables diagrams
The factors are also analysed in certain usage stages (see Figure 2-9). Initially,
users do not use a social bookmarking service, and then, after it, they begin
using it. After discovering some of the disadvantages of the social bookmarking
35
service, the users stop using it. If the social bookmarking service was improved,
these same users might use it again. If the improved social bookmarking
service has problems, the users will stop using it again.
Figure 2-9: Usage stages
Factors that potentially affect social bookmarking usage are listed below:
Table 2-2: Possible factors by usage stages Don’t
Use Use Stop Using
Restart Use
1. Usability ● ● ●
2. Functionality ● ● ●
3. Web Interface Design ● ● ● ●
4. Attractiveness ● ●
5. Application in Sociology ● ●
6. The Target User Experiences ● ●
7. Stickiness ● ● ●
8. Search by Tags ● ● ●
9. Search by Rank ● ● ●
10. View by Pivot Browsing ● ● ●
11. Social Navigation by Content Tags ● ● ●
12. Social Navigation by User Tags ● ● ●
The table reveals that web interface design, attractiveness, application in
sociology, and the target user experiences are all factors that potentially affect
social bookmarking usage and are also possible reasons why only a small
number of people are using social bookmarking.
However, the group of people that do not use social bookmarking can be
divided into three groups. The first is the group of people who do not know
36
about social bookmarking. The second is the people who know about social
bookmarking but do not use it. In order to find and analyse the reasons why
only a small number of people are using social bookmarking, both groups have
to be considered. Moreover, the third group of people – those who know about
social bookmarking, have used it and have stopped using it – also have to be
considered.
2.4 Theoretical framework
For more people to know about and begin using social bookmarking, they need
to hear about it from others – the word of mouth epidemic (application of
sociology). If people are unable to hear about social bookmarking from others,
what other ways might they find out about it? Social bookmarking itself could
draw their attention (attractiveness). There are two ways to achieve this: one is
through the service itself, the other is through the social bookmark buttons. If
people do not know about social bookmarking and have no other people telling
them, then there are fewer opportunities to use the service itself initially.
However, people can still come into contact with social bookmarking if they like
to browse the Internet and the social bookmark buttons are attractive. For
example, if people read news on the website bbc.com, they might notice the
social bookmark buttons under each news story. Social bookmark buttons
would be their first introduction to social bookmarking. Social bookmark buttons
are used as tool to investigate why only a small number of people are using
social bookmarking.
There are lots of social bookmarking buttons (see Figure 1-2). Are these
buttons the same or different? Social bookmarking services work slightly
differently (BBC, 2009). Do these slight differences exist among social
bookmark buttons also? How many of these differences exist? After the
different social bookmark buttons’ work processes were compared, some
differences were found and these differences were not slight (see Appendix A).
The best idea is to achieve the largest effect for the smallest effort (Clark and
Wilkes-Gibbs, 1986; Sperber and Wilson, 1986): more than one extra required
37
click can stop people using a social bookmark button (Burkard, 2009). Social
bookmark buttons’ differences are summarised into two main categories:
cognitive effort and social feedback:
Table 2-3: Cognitive effort and social feedback by social bookmark buttons (collected by July 16th, 2009 from bbc.co.uk) Cognitive Effort Social Feedback
Digg
l Processing Steps Bookmark à Login/Register à Feedback
l Login and Register Entries Many details need to be entered to register.
l Bookmark Page URL, Title, Description are entered automatically. Preview section is listed on the Bookmark page.
l Re-bookmark Page Easy to save a bookmark after one has clicked “Digg” number.
l Feedback Page More details on the feedback page. Users can rate others’ comments.
Delicious
l Processing Steps Login/Register à Bookmark à Feedback
l Login and Register Layout Login and Register are listed on separate pages.
l Bookmark Page URL and Title are entered automatically.
l Feedback Page No Feedback sometimes. Have to click the relative buttons to see more feedback details.
StumbleUpon
l Bookmark Page Have to enter bookmark details twice.
l Feedback Page More details on the feedback page.
l Login and Register Layout Login and Register are listed on the same page.
l Login and Register Entries Few details need to be entered to Register.
l Feedback Page More details on the feedback page.
For the purposes of this research, cognitive effort is defined as the number of
clicks or required text box entries, and social feedback is defined as the amount
of real-time information given to users. Cognitive effort and social feedback
might be the factors that affect use of social bookmarking and why only a small
number of people are using social bookmarking. If the best idea is to achieve
the largest effect for the smallest effort, users might stop using a social
38
bookmark button (social bookmarking usage) if it requires lots of text box
entries (cognitive effort) and no information is provided to users (social
feedback). For example, the Digg social bookmark button includes lots of
required text box entries and the Delicious social bookmark button does not
give users feedback sometimes (see Table 2-3). If users enter lots of details (as
for Digg) and get no feedback (as for Delicious), fewer people might continue to
use social bookmarking. Cognitive effort might affect social bookmarking usage,
or social feedback might affect social bookmarking usage, or both cognitive
effort and social feedback might affect social bookmarking usage.
Cognitive and learning theories (Jarvis, Holford and Griffin, 2003; Biggs, 1993)
were used in this research. People realise a new object going through four
phases, such as hearing, olfaction, vision, and feeling. In order to let people,
who have not used social bookmarking, to realise social bookmarking and use
social bookmarking, the theories should be used to analysis at which phase
users stop using social bookmarking.
2.5 Research question
Does the cognitive effort and social feedback of a social bookmark button affect
the likelihood of using social bookmarking on the Internet?
l Does the cognitive effort involved in using a social bookmark button
affect the likelihood of using social bookmarking on the Internet?
l Does the social feedback involved in using a social bookmark button
affect the likelihood of using social bookmarking on the Internet?
Figure 2-10: Research question variables diagram
39
2.6 Hypotheses
1. Social bookmark button’s cognitive effort affects the likelihood of using
social bookmarking on the Internet.
2. Social bookmark button’s social feedback affects the likelihood of using
social bookmarking on the Internet.
2.7 Research type
This research is a quantitative research (included qualitative responses to
open-ended questions in a questionnaire). This experimental study on the
effect of cognitive effort and social feedback on the likelihood of using social
bookmarking developed in two phases.
40
Chapter 3: First Phase Study Method
3.1 A new type of social bookmark button – Bligg
This research investigated whether the cognitive effort required and social
feedback offered by social bookmark buttons affect the likelihood of using
social bookmarking or not. Using current and ready-made social bookmark
buttons to compare cognitive effort and social feedback would be imprecise
because each social bookmark button might be updated and the differences
between them would be not very obvious. For this reason, an entirely new type
of social bookmark button called “Bligg” was created to simulate social
bookmark interaction after the differences among cognitive effort and social
feedback were found (see Appendix A and Table 2-3). For the subjects for this
research, they were recruits of social bookmarking. The recruits would not put
their previous using experience and opinions into this research.
3.1.1 Processing order
Social bookmark buttons have four main processing steps – register, login,
bookmark, and feedback – but the processing order is different for each. Digg’s
processing order for the first of these three is bookmark à login/register, but
the processing order for Delicious is login/register à bookmark. For the login
and register processing order, Digg and Delicious follow login à register, but
StumbleUpon’s processing order is register à login and reddit’s login and
register are listed on the same page. Which processing order is better and
should be used? In order to ensure it was logical and every subject to go
through login/register processing steps after bookmark processing step, the
login/register and bookmarking processing order login/register à bookmark
was used on Bligg. To ensure Bligg was convenient to use, the login and
register processing order register à login was selected, and the register and
login processing steps were separated.
41
Moreover, the register and login processing steps could be skipped if the
subjects had already registered as new users and logged in. If a subject is
using Bligg for the first time, the processing order is register à bookmark à
feedback. If the subject has already created an account and logged in, the
processing order is bookmark à feedback.
3.1.2 Each processing step’s versions
In order to ensure that differences in cognitive effort and social feedback were
obvious, Bligg’s cognitive effort had two levels (low and high) and social
feedback had three levels (none, low and high). The processing steps of
register, login and bookmark were focused on cognitive effort, and the
processing step feedback was focused on social feedback. Cognitive effort is
defined as the number of clicks or required text box entries, and social
feedback is defined as the amount of real-time information given to users. All
the versions created depended on current and ready-made social bookmark
buttons’ reality and simulated social bookmark interaction (see Appendix B).
3.1.2.1 Register
Some text boxes, drop down selections and tick boxes are used to register for
current and ready-made social bookmark buttons. Some social bookmark
buttons require only a couple of text boxes to be filled, but some require users
to fill several text boxes, select drop down selections, and check tick boxes. In
order to make cognitive effort obvious, the two versions included the different
number of clicks and required text box entries:
Low Cognitive Effort: Requiring Username, Email Address, Password, Re-type
Password. (This is the reddit version, although it did not include Captcha and
Remember Me, to keep it at low cognitive effort.)
High Cognitive Effort: Requiring Username, Email Address, Password, Re-type
Password, First and Last Name, Gender, Country, Zip or Postal Code, Birthday,
Captcha, and Term and Conditions. (This is the Digg version.)
42
In order to make Bligg seem real, the drop down selection’s items (such as
Gender, Country and Birthday) were the same as the current Digg social
bookmark button, and all of captchas were copied from the Digg social
bookmark button.
3.1.2.2 Login
Because current social bookmark buttons’ logins all require the same text
boxes to be filled, Bligg’s login was the same for both versions:
Required text box entries: Username and Password
Un-required tick box: Remember Me
3.1.2.3 Bookmark
The processing step bookmark asks the user to provide the bookmark’s details,
such as its URL, title, description and topic; most social bookmark buttons
include these details. However, some social bookmark buttons enter some
details automatically, and some social bookmark buttons do not. This requires
different amounts of cognitive effort from users. In order to make the
differences obvious, two versions that included similar required text box entries
were created, but the automatic conditions were different:
Low Cognitive Effort: Requiring URL, Title and Description. All were entered
automatically. (This is the Delicious version, without the Tags and Do Not Share
features. The Description text box was also changed, and was entered
automatically as the Digg version. All the changes were made to ensure
subjects expended less effort with this version.)
High Cognitive Effort: Requiring URL, Title, Description, Thumbnail, Topic and
Captcha. Only URL was entered automatically. (This is the Digg version, with
subjects having to enter and choose the bookmark’s details. It is also similar to
the reddit version. These changes ensured that the subjects spent more
physical and mental energy on this version.)
To re-bookmark, nothing was changed and subjects had to do the same tasks
as the first time, but the Bligg number was changed. For the first time bookmark,
the Bligg number is one. If it is re-bookmark, the Bligg number will be two or
more. This design ensured that every subject performed the same tasks and
43
expended the same amount of cognitive effort, whether it was their first
bookmark or not. (This is similar to the Delicious and StumbleUpon versions,
without the Popular Tags and Suggested Tags features.)
Bligg did not include a tag feature. In order to avoid “sloppy tags”, which would
affect usage of Bligg and research results’ precision, a tag feature was omitted.
In order to make Bligg seem real, the Topic drop down selection’s items were
all the same as the current Digg social bookmark button and all of captchas
were based on Digg. The Char Left feature was created to look like the Digg
and Delicious versions.
If the social feedback version is high, a Preview feature will be added into the
processing step for each of the two different cognitive effort versions. The
details of Preview were low social feedback (see details below).
3.1.2.4 Feedback
Social feedback is defined as the amount of information given to users. This
information can be put into two categories:
• The new social bookmark information provided by the user who submitted it.
• Other social bookmarks’ information provided by other users.
An overview of the current and ready-made social bookmark buttons reveals
that some buttons offer a lot of social feedback, some buttons offer only some
basic information about the new submitted social bookmark, and some buttons
offer no social feedback at all. After a user has submitted a new social
bookmark, the popup window is closed and no feedback information is
provided to users. The social feedback was divided into three levels, the details
of which are listed below:
No Social Feedback: No information was given to users. (This is similar to the
Delicious version, when Delicious users register or are already logged in.)
Low Social Feedback: Basic information about the newly submitted social
bookmark is provided to users. This information included Title, Description,
44
Bligg Number, Submitted User’s Username, Submitted Time, and Topic. (This
is similar to the Digg version, when Digg users are logged in and the new
bookmark is submitted for the first time. It is also similar to the reddit version.)
High Social Feedback: This version included all the low social feedback
information and added more details about the newly submitted social bookmark
(such as Thumbnail; Comment – Date, Comment, Username, Vote), and also
provided other bookmarks’ information to users (such as Related by Keyword –
Bligg Number, Title; Related by Source – Bligg Number, Title; Topic in it – Bligg
Number, Thumbnail, Topic; People Who Saved This Also Saved – Bligg
Number, Thumbnail, Topic). (This is similar to the Digg version, when Digg
users re-bookmark bookmarks.)
An add a Comment feature was included in the feedback processing step.
Subjects could use this feature to add a comment about the newly submitted
bookmark. This feature had two versions:
Low Cognitive Effort: Requiring Comment. (The reddit version.)
High Cognitive Effort: Requiring Comment and Captcha. (The StumbleUpon
and Digg versions. Captchas were copied from the Digg social bookmark
button.)
Six different Bligg buttons were created for low and high cognitive effort and
social feedback with three levels: none, low and high:
Table 3-1: Bligg buttons by cognitive effort and social feedback
High Cognitive Effort Low Cognitive Effort
No Social Feedback
Low Social Feedback
High Social Feedback
3.1.3 Interface design
Bligg’s interface design was based on previous research (see Chapter 2.2.5.1
The designers - interface factors), and followed the conceptual model of web
interfaces (see Figure 2-3). The details are listed below, and the interface
45
screenshots are displayed in Appendix C.
3.1.3.1 Text
The Bligg font style was sans serif typeface – Arial, because it is the most
common font style for good looking pages and is more legible and safe than
serif typefaces. The heading sizes were 20-25px, and text sizes were 12-15px.
3.1.3.2 Link
As mentioned earlier, there were four processing steps – register, login,
bookmark and feedback. Because subjects were recruits, the main processing
steps were register, bookmark, and feedback. One text link “Login”/“Register”
was added on the register and login page, so that subjects could navigate
between the two processing step pages using the links. The length of the link
text was one word, to make it simple and clear. On the register, login, and
bookmark processing step pages, there were two buttons. One was the Cancel
button, which could be used to cancel tasks and close the Bligg popup window.
The other was the Next button, which could be used to navigate to the next
processing step page. The Next button’s text changed depending on the
different processing steps. On the feedback page, the text link “Close” was
added. Subjects could use this to close the popup window after reading the
feedback details.
3.1.3.3 Graphic
In order to improve download speed, the graphics were small; the largest was
4KB for the captcha in Bligg. There were no images containing text, navigation
images, or animated images in Bligg. Most of the image formats were JPG and
BMP.
3.1.3.4 Colour
Although the blue-green combo is a trusty standby, is great looking, safe, and
the most conservative and attractive colour combination, the colour blue was
predominantly used in Bligg because it is a formal colour and it encouraged
people to take Bligg seriously. This aimed to improve the quality of the research
results.
46
3.1.4 Database
Social bookmark buttons are added on web pages and allow users to
bookmark their favourite web pages in an automatically opened popup window.
Users do not need to open social bookmarking services in a new window or tab
to bookmark web pages. Social bookmark buttons can transfer the web pages’
details to the popup window and then save the details into the social bookmark
buttons’ database. Users can access the database using social bookmark
buttons and social bookmarking services.
It was necessary to create a website for the Bligg social bookmark button. The
website could be as a carrier, which allows adding Bligg on each web page in
the website. The website’s details and how to access database details are
listed as below:
• The Beatles mini website
Given that the website was intended to be used for research, the content
needed to be interesting – options were news, images, videos and music. It
was thought that news might be boring, images might be hard to control
and videos might take a long time. For these reasons, music was selected.
What sort of music should be selected? What songs should be added to the
website? To ensure that every subject enjoyed the music and found the
songs familiar, pop music and the Beatles’ songs were selected. Ten songs
were selected from the album “The Beatles’ Most Favourite Songs”, all of
which were famous Beatles songs. Due to copyright, all of the songs were
samples only and were played for just 40 seconds.
How to design and create the website? Because all the songs were by the
Beatles, the website was designed based on the Beatles official website –
www.thebeatles.com (see Figure 3-1). The interface design (the layout, font
styles and sizes, length of link text, number and types of links, number and
types of graphics, and colour) was the same as the Beatles’ official website,
and pictures from the official website were used. Moreover, to ensure that
47
the website was simple and clear, the content was changed. Because the
website was about the Beatles songs, no menu bar was added. When the
website opened, the songs were listed. After the subject chose a song from
the list, that song’s web page was opened and the selected song was
played automatically. The Stop/Play button could be used to stop/play the
selected song. Meanwhile, the song’s name, an introduction, the album’s
thumbnail, and the Bligg social bookmark button were displayed on the web
page. The Beatles songs’ introductions were copied from the website
wikipedia.org, and album’s thumbnails were found on the Beatles official
website. The website’s interface screenshots are displayed in Appendix D.
Figure 3-1: The Beatles official website screenshot (Thebeatles, 2010)
• Bookmark data display
As mentioned earlier, there were four versions of the bookmark processing
step page (see Appendix C). The details transferred to this page were
different in the different versions:
n In the low cognitive effort version, the selected song’s URL, title,
and description would be entered automatically. In the high
cognitive effort version, the selected song’s URL would be entered
automatically and the selected song’s album thumbnail would be
displayed with the Beatles’ “Past Masters” album thumbnail (from
the Beatles official website).
n If social feedback was high and the cognitive effort was low, the
selected song’s title and description and the submitted subject’s
48
username, submitted time and topic would be listed in the preview
section. If social feedback was high and the cognitive effort was
high, just the submitted subject’s username, submitted time and
topic would be listed in the preview section. If the song’s details
were changed in the text boxes, the preview details would be
changed at the same time.
• Feedback data display
Although there were three levels of social feedback (none, low and high),
just two of these levels were displayed (low and high). The social feedback
was divided into submitted bookmark feedback and other bookmarks
feedback.
n Submitted Bookmark Feedback:
The details were the same as those from the bookmark processing
step page. If the submitted song’s details changed on the
bookmark page, the new details would be displayed on the
feedback page. In order to make comments to appear real, the
submitted song’s comment details (such as comment, username
and date) were taken from the website youtube.com in September,
2009. Each submitted song had 3 comments – a good comment, a
normal comment and a bad comment. The good comment’s voting
number was 1, the normal comment’s voting number was 0 and
bad comment’s voting number was -1. Subjects could vote on the
comments. The comments were controlled feedback for subjects in
this research.
n Other Bookmarks Feedback:
This feedback was Related by Keyword, Related by Source, Topic
in it and People Who Saved This Also Saved. Because the website
was about the Beatles songs, the “Topic in it” was changed into
“Topic in Music”. All the resources were from a Beatles song
bookmark’s feedback on the website digg.com and all the details
were real and collected by September 10th, 2009.
The feedback was controlled, real information and every subject was able
to get the same quantity and quality of information from the feedback
49
processing step page. This ensures the quality of the results.
• Database access
Subjects could bookmark the Beatles songs on an automatically opened
popup window, and the Bligg button would then transfer song’s details
using JavaScript programming language. The data transfer details are
listed in Figure 3-2.
After a song was bookmarked, the details were saved into a .txt file using
PHP programming language. Why use a .txt file and not a MySQL database
online? Saving data into a .txt file made it easy to collect data correctly
because the website was online and everyone could use it at anytime.
Which details were saved? The details included the song ID; the newly
submitted bookmark’s date and time, the title, description, thumbnail, topic,
submitted comment, register time taken, bookmark time taken, comment
time taken; the submitted subject’s username and user ID; Bligg’s version
number and each song’s Bligg number.
Figure 3-2: Data transfer details
3.2 Research design
Once the Bligg social bookmark button and the Beatles Mini website were
created, the question was how should one use these two tools to find out
whether cognitive effort and social feedback affect the likelihood of using social
bookmarking or not?
50
Firstly, cognitive and learning theories were examined, following the usage
stages. It was helpful to find out at which stage subjects lost their interest and
chose not to use social bookmarking. The group of people that do not use
social bookmarking can be divided into two groups: people who do not know
about social bookmarking and so do not use it, and people who know about it
but choose not to use it. In this analysis, the usage stages began with non-use
of social bookmarking – the subjects were newcomers, who did not know about
social bookmarking and did not use social bookmarking. People noticing a new
object go through four phases – hearing, smelling, seeing, and feeling. In order
to let social bookmarking newcomers to know about social bookmarking and
induce them to use the Bligg social bookmark button, the research was divided
into three phases – unknown social bookmarking, explaining and seeing social
bookmarking and the Bligg social bookmark button, and using the Bligg social
bookmark button. The details are listed below:
Table 3-2: Three phases following usage stages Do Not Know Know
Do Not Use Unknown Social Bookmarking
Explaining and Seeing Social Bookmarking and
The Bligg Social Bookmark Button Use Using
The Bligg Social Bookmark Button Stop Using Using
The Bligg Social Bookmark Button Restart Use Using
The Bligg Social Bookmark Button
How to design the research? In order to detect the stage at which subjects lost
interest and chose not to use social bookmarking, the researcher analysed
each phase. To ensure the research was valid, reliable and generalisable, data
were gathered from precise quantitative studies (questionnaires). Two methods
could have been used, the details of which are displayed below:
Table 3-3: Two optional research methods
Method 1 Method 2
Description All subjects go through the three phases.
The subjects are divided into 3 groups, and each group goes through each phase randomly.
Advantage l Every subject’s task might be
same and the task time taken might be similar
l Takes only a short amount of time
l Might be valid and reliable
51
l Might be orderly and systematic
Disadvantage
l Takes a long time l Might be biased and spurious
l Every subject’s task and task time taken might be different
l Might be disorderly and unsystematic
After comparing the two methods and considering the methods’ advantages
and disadvantages, Method 1 was selected and applied. Although Method 1
tasks took a longer time, all subjects’ tasks were the same. In order to let
subjects use the Beatles Mini website and the Bligg social bookmark button
easily and to avoid it taking a long time and having a bias, help descriptions
were added into these two tools (see Appendix C and Appendix D). The help
descriptions helped subjects realise how the Bligg social bookmark button
works and shortened the time taken to do tasks, thus avoiding biased and
spurious research results.
Secondly, six versions of the Bligg social bookmark button were used in the
phase “using the Bligg social bookmark button”. When subjects lost interest
and chose not to use social bookmarking, the researcher was able to see
whether cognitive effort or social feedback was the reason. This ensured the
results’ veracity and believability. After the experiment, the subjects’ preferred
Bligg version was found; this information could be helpful when developing
current social bookmark buttons.
How to design the research using the six versions of the Bligg social bookmark
button? As mentioned earlier, the six versions of Bligg were created based on
two levels of cognitive effort and three levels of social feedback (see Table 3-1).
In order to find the best version of Bligg buttons and shorten the time taken,
each subject used two versions of the Bligg button, which offered the same
amount of social feedback but required different amounts of cognitive effort.
Moreover, the versions were randomly presented in a different order to avoid
order bias. For example, one group might use Bligg 1 (high cognitive effort and
no social feedback) first and Bligg 2 (low cognitive effort and no social feedback)
next, while another group used the same Bligg buttons in the reverse order. To
ensure that the research is valid, reliable and can be generalised, data was
52
gathered from questionnaires and the time taken was directly measured. The
research method is listed below:
Figure 3-3: Research method design
Because of the research method design, each subjects had to use two versions
of the Bligg social bookmark button. In order to help the subjects realise that
they were using different Bligg buttons, the Beatles Mini website was changed
a little bit. Both versions of the Beatles Mini website displayed five different
songs (see Appendix D). This was more interesting than requiring subjects to
bookmark the same songs using the different Bligg buttons. Moreover, each
subject had to bookmark two different songs from the five songs using one
version of the Bligg social bookmark button. For this research, each subject
had to bookmark four songs in total. As mentioned earlier, the register and login
processing steps could be skipped after subjects had already logged in once.
When the subjects used the Bligg button for the first time, the process order
was register à bookmark à feedback. When the subjects used the same
version of the Bligg button again, the process order was bookmark à
feedback.
53
Initially, all subjects were asked how interested they were in social bookmarking
using a scale. After explaining about social bookmarking and seeing how the
Bligg social bookmark button worked, all subjects were asked the same
questions again to find out whether their opinions had changed or not. Next,
subjects were asked to use two versions of the Bligg social bookmark button.
After using each version, subjects answered the same questions again and
some specific questions about how the Bligg social bookmark button was to
use and which was their preferred version (based on cognitive effort and social
feedback). Finally, one open-ended question was asked. Most of the questions
in the questionnaire were on an interval scale and were asked in an unbiased
way. The procedure plan details are discussed further in Chapter 3.5.
3.3 Subject requirement
As stated earlier, the population had to be newcomers to social bookmarking.
The population had to not know, have heard about, seen or used social
bookmarking, social bookmarking services or social bookmark buttons before
but the population must be able to learn and use new things fairly easily and
they must have computer skills and Internet experience. These requirements
made collecting results for this research easier.
For these reasons, the research samples were taken from univerisites.
Students find it easier to apply new things, have computer skills and Internet
experience, and are expected to learn new things. It was thought that young
students might be hard to control and organise, so the research samples were
taken from universities. To ensure that the research samples were generalised
and unbiased, they were selected randomly from Massey University in
Auckland.
The research method was designed to analyse how certain things (such as
cognitive effort and social feedback) affected the subjects’ desire to use the
Bligg social bookmark button. The sample size should be:
2 (cognitive effort) x 3 (social feedback) x 2 (use/not use) x 4 (phases)
54
= 48 (samples)
The research sample size must be more than 48. After considering the
research method and the Bligg social bookmark button, a sample size of 60
was decided upon.
How to find sixty subjects from Massey University in Auckland? To avoid a
biased approach, the sixty subjects were selected randomly from Massey
University computer labs, in Auckland, New Zealand.
l First, find a student from Massey University in Auckland.
l Then, make sure the student fits the sample requirements.
l Next, ask the student to be a part of the research. If the student is
interested in the research, offer more general research details and allow
enough time for the student to read a “Survey Letter” (see Appendix E).
l After the student has finished reading “Survey Letter”, make an
appointment with the researcher. At that stage, the researcher gives the
student a “Timetable” (see Appendix F), and asks the student to choose a
date and enter their name and email address on the “Timetable” sheet.
After the researcher checks the details on the “Timetable” sheet, the
researcher ticks the chosen date on the “Survey Letter” and asked the
student to keep the “Survey Letter”. The “Survey Letter” reminds the
student of the research time, date and location, and includes the
researcher’s contact details.
l Finally, suggest the student bring friends to participate in the research. This
provides more subjects for the research and creates a social and friendly
environment for the subjects themselves. The student writes the friend’s
name and email address on the “Timetable” sheet using a pencil, in case
the friend changes the plan or the date. The student could also email the
researcher the friend’s details and chosen date. The researcher keeps the
friend’s details for one day.
After sixty subjects were found from within Massey University in Auckland, the
subjects were divided into six groups randomly using the “Table of Random
Numbers”. Each group included ten subjects, who were invited to attend three
main phases. The research location meant that only five subjects could
55
participate at the same time. Moreover, the research took place from 3pm to
4pm, after the subjects’ class time. The procedure plan details are explained
further in Chapter 3.5.
3.4 Questionnaire design
The Questionnaire (see Appendix H) was created based on the research
method and the research question. The questionnaire’s questions were easily
understood, unambiguous, unbiased and inoffensive. Some questions had a
dichotomous scale (nominal scale) and asked for basic, categorical, gross
information and that could be calculated as a percentage. Some questions had
an interval scale (like the Likert scale, numerical scales or semantic differential
scale), which was easy to use, unbiased, exhaustive and sensitive. The interval
scale is a more powerful scale than a nominal scale or an ordinal scale, and it
gives an indication of the magnitude of the differences among the ranks. All of
the interval scales had an unbalanced rating scale (10-point scale), which did
not have a neutral point. This made them more accurate and sensitive in
eliciting unbiased responses. Some questions were open-ended questions that
collected qualitative responses. The details are described below:
Table 3-4: Questionnaire questions’ details
Question Description
Q1 This question included sub questions, and the sub questions were answered depending on the different steps (such as unknown, explaining and seeing, using one version of the Bligg button, and using another version of the Bligg button). The sub questions were about the likelihood of the subject using social bookmarking and how easy and useful each version of the button was. The sub questions told the researcher whether the subjects’ opinions had changed or not and at which point the subjects lost their interest and did not want to use social bookmarking. They helped to ascertain whether cognitive effort or social feedback affect the likelihood of using social bookmarking. These sub questions used a 10-point Likert scale (interval scale).
Q2 This question included two questions, both of which were only answered after the two versions of the Bligg button had been used. The first question used a 10-point numerical scale (interval scale), and asked which version of Bligg buttons was better. The answers helped the researcher find the best version of the Bligg button based on cognitive effort and social feedback. The second question was an open-ended
56
question that asked about some of the reasons that the subjects preferred one version to another. The researcher might find more factors that affect social bookmarking usage from this open-ended question.
Q3 ~ Q6 These questions included sub questions that were answered after the two versions of the Bligg button had been used. The sub questions were about the required cognitive effort on the submit bookmark page, register page and add a comment section. The answers explain which text boxes the subjects were willing to fill in, which text boxes the subjects were not willing to fill in, which automatic functions were very helpful and which were not helpful. The answers provided more details about cognitive effort and helped the researcher realise which text boxes were necessary for the correlating page/section and whether automatic text boxes were helpful or not. These findings were very helpful in terms of improving current social bookmark buttons. These sub questions used a dichotomous scale (nominal scale) or 10-point semantic differential scale (interval scale) that was able to compute the means and the standard deviations of the responses.
Q7 These questions included sub questions that were answered after the two versions of the Bligg button had been used. The sub questions were about the social feedback on the feedback page. The answers explained what information was very useful and what information was not useful, helping the researcher to realise how much information the subjects want. The findings were very helpful for current social bookmark button designers. They provided information on how to attract users and improve the buttons. These sub questions used a 10-point semantic differential scale (interval scale), and measured the magnitude of the differences in the preferences among the individuals.
Q8 This question was an open-ended question to get any general comments. The researcher might find more factors that affect the likelihood that people will use social bookmarking via this question.
3.5 Procedure plan
At the beginning of the research, there was a preparation phase for the
researcher, in which the researcher made sure everything was ready. The
researcher had to put one copy of the “Consent Form” (see Appendix G), one
copy of the “Questionnaire”, one earphone for listening to songs and one pen
for filling in questionnaires on the tables for each subject. Then, the researcher
had to turn each computer on and open the Internet web browser and
“Procedure Navigation Page” (see Appendix I). The “Procedure Navigation
Page” was a webpage that was created to help subjects follow the procedure
easily. The webpage could show the subjects what to do next – the Beatles Mini
website could be opened from it. A green processing bar was added to the
“Procedure Navigation Page” to show the subjects which stage they were at
57
and how many tasks were left.
To ensure the research results were authentic and effective, the researcher did
not reveal any details to the subjects after the research. If the subjects had
problems, they could ask the researcher questions; the researcher was not
allowed to help the subjects with the tasks or the survey.
3.5.1 Unknown phase
Initially, the subjects were asked to sign a “Consent Form”, after which they
were allowed to look at the “Procedure Navigation Page” on the already
opened Internet web browser and answer “Question 1a and 1b for STEP I”
following the procedure.
3.5.2 Explaining and seeing phase
After the subjects answered the questions, the researcher taught the subjects
about social bookmarking, social bookmark buttons and the Bligg button with
that aid of a PowerPoint. The subjects were then asked to answer “Question 1a
and 1b for STEP II”.
3.5.3 Using phase
After the questions were answered and the subject had clicked the “Next”
button on the “Procedure Navigation Page”, the “Try Out The First System” task
was displayed. After the subjects clicked “Try Out The First System”, the
Beatles Mini website was opened in a new tab. A popup notes window was
opened at the same time, reminding the subjects to put on the earphone, listen
to the songs and bookmark two songs using the Bligg button. When the
subjects clicked the Bligg button to bookmark a song, the Bligg popup window
was opened with another popup notes window, which told the subjects to
register because it was their first time using this version of the Bligg button.
58
After the subjects went through the register, bookmark, and feedback pages
(for some versions of Bligg buttons), the Bligg popup window was closed
automatically or by clicking the “Close” button on the Bligg popup window.
When the subjects bookmarked their second song using the same Bligg button,
the register page did not appear and the bookmark page opened. After they
bookmarked the second song, a popup note window was opened automatically,
reminding the subjects to go back to “Procedure Navigation Page” to do the
next task. After the subjects clicked the “Next” button on the “Procedure
Navigation Page”, the next task was displayed – “Answer Question 1a to 1f for
STEP III”.
After answering the questions, the subjects clicked the “Next” button on the
“Procedure Navigation Page” and the task “Try Out The Second System” was
displayed. The task details were the same as those for “Try Out The First
System”, except that the songs and the button were different. After the subjects
finished the tasks and clicked the “Next” button on the “Procedure Navigation
Page”, the task “Answer Question 1a to 1f for STEP IV & Question 2” was
displayed.
After answering the questions, the subjects clicked the “Next” button on the
“Procedure Navigation Page” and the task “Answer Questionnaire: Part 2” was
displayed. After they answered these questions, the questionnaire was
complete and the research was finished.
3.6 Measurement method
The measurement method was reliable and valid in the way it detected and
analysed the data from the questionnaires. The measurement method included
two main steps – getting data and analysing data.
59
3.6.1 Getting data
The researcher got the data ready for analysis after it was collected in the
questionnaires. In this phase, the sub jobs were editing data, handling blank
responses, coding data, categorising data and creating a data file. When the
researcher categorised data, the items had to be reversed so as to be in the
same direction as the positively worded questions. Moreover, a row table was
used to enter data into a data file.
3.6.2 Analysing data
The researcher analysed the data using SPSS analysis data software. In data
analysis, there are two objectives: getting a feel for the data and testing the
hypotheses developed for the research. When testing the hypotheses, t-test
and ANOVA were used to analyse whether cognitive effort and social feedback
affect the likelihood of using social bookmarking or not. T-test was used for Q1
and a mean table was used for Q3~Q7.
60
Chapter 4: First Phase Study Results
4.1 Getting data ready for analysis
After data were collected from subjects, the original data had to go through five
steps (editing data, handling blank responses, coding data, categorising data,
and entering data) before being analysed. SPSS analytical and statistical
software was used to analyse the data.
4.1.1 Editing data
Data have to be edited, especially when the data are responses to open-ended
questions. There were two open-ended questions in the questionnaire. The first
was “Q2. Reasons? (Why do you prefer this system?)”, and the second was
“Q8. Any general comments?”. Because the researchers checked completed
questionnaires in the subjects’ presence, there were no legibility problems.
These answers to these two questions are in Appendix J.
4.1.2 Handling blank responses
Because the researchers checked the completed questionnaires in the
subjects' presence, all of questions were answered reasonably. This step was
skipped.
4.1.3 Coding data
To analyse the data, responses had to be coded. Because most of the
questions were answered on an interval scale, the answers to these questions
were the data. However, Q3a, Q5a and Q6a (sub question – “Willing to do it?”)
had nominal scales with “Yes” or “No” options; in this case “Yes” was coded as
61
1 and “No” was coded as 0.
4.1.4 Categorising data
There were six groups, and each version of the Bligg social bookmark button
was compared twice in different orders (see Figure 3-3). There were three main
groups comparing the six versions of the Bligg social bookmark button. In order
to analyse the data, the order had to be the same. Questionnaire responses
from Group 1, Group 2 and Group 3 were not changed, but questionnaire
responses from Group 4, Group 5 and Group 6 were changed. For Group 4,
Group 5 and Group 6, the answers to the STEP III sub questions were
swapped with the answers to the STEP IV sub questions.
The answers to “Q2. Based on trying the two systems out, which system do you
prefer?” also had to be changed. For Group 4, Group 5 and Group 6, the
answers to Q2 were changed. If the answer was 1, the answer was made 10.
The details are listed below:
1 à 10, 2à 9, 3 à 8, 4 à 7, 5à 6, 6 à 5, 7 à 4, 8 à 3, 9 à 2, 10 à 1
Because some answers were changed, the data were categorised into three
groups – Group1, Group 2 and Group 3. Group 4 became Group 1, Group 5
became Group 2 and Group 6 became Group 3.
The answers to other questions using an interval scale were not changed
because the lower number meant negative and the higher number meant
positive.
4.1.5 Entering data
After being coded and categorised, these data had to be entered into SPSS for
analysis. The raw data table’s columns were questions and the rows were
subjects.
62
4.2 Data analysis
In data analysis, there are two objectives – getting a feel for the data and
testing the hypotheses developed for the research. The results in this section
can be used to interpret in the next section.
4.2.1 Feel for the data
Getting a feel for the data can offer preliminary ideas. The responses to the
open-ended questions (“Q2. Reasons? (Why do you prefer this system?)” and
“Q8. Any general comments?”) could provide a feel for the data. The details are
described below:
• Simple and fast
One subject commented that “bookmarking is supposed to be simple (and)
fast like first one (low cognitive effort and no social feedback)”. How simple
is it? “I just want to bookmark, why so many questions (to be answered)?”,
“shorter process for bookmarking”, “shorter steps in key-in information for
the songs”, “sys2 (low cognitive effort and low social feedback) is easier,
fewer click (and) fewer register info needed”, “(not) too many details to fill
in” and etc. How fast is it? “quicker”, “less time spent on the registration”,
“the system saves a lot of time of typing and thinking about the description”,
etc.
Did the questionnaires’ results prove these points? Did the subjects prefer
low cognitive effort over high cognitive effort? The results from “Q2. Based
on trying the two systems out, which system do you prefer?” are below:
63
Figure 4-1: Mean preference for social bookmark buttons by cognitive effort (N=60)
The results suggest that subjects preferred the low cognitive effort system
to the high cognitive effort system.
Did the subjects think that the details entered already were helpful? The
results from “Q4. Is it helpful if the following details are entered already?”
are below:
Figure 4-2: Mean helpfulness by details entered already (N=60) It appears that the subjects thought details entered already were very
helpful and that having the URL entered already was the most helpful.
Which details were the subjects willing to enter? How much effort to enter
each detail for the subjects? The answers to these questions can be found
in the responses to “Q3. What do you think about social bookmark
submission?”, “Q5. What do you think about social bookmarking
registration?”, and “Q6. What do you think about adding a comment?”. The
details are displayed below:
64
Figure 4-3: Mean willing and effort to enter the details (N=60) The results suggested how willing subjects were to satisfy various social
bookmark requests. Username, Password, Email, Thumbnail and Title
were generally accepted; having to give Full Name, Comment, Gender,
URL and Country were reported as more onerous; subjects were least
willing to give Postal Code and Birthday details or fill in the Captcha form.
The same number were willing as were un-willing to fill in Full Name and
Comment. Moreover, the subjects did not put much effort into Thumbnail,
Username and Gender, but they especially felt that Captcha’s required a
high level of effort.
Did the low cognitive effort system take less time than the high cognitive
effort system? Because the research measured the time taken and saved
these details into the database, the time taken can be analysed using the
database.
Table 4-1: Mean task time taken in milliseconds by cognitive effort (N=60)
Registration Bookmark Comment
The High Cognitive Effort System
92618.47 59424.98 33991.45
65
The Low Cognitive Effort System
31587.23 5920.08 24139.80
As mentioned in Chapter 3, each subject used the different Bligg buttons
two times; the table shows the mean time taken for the different levels of
cognitive effort. The results show that the high cognitive effort case took a
significantly longer amount of time to complete: nearly ten times as long for
the bookmark page, three times as long for the registration page, and 1.5
times as long for the comment section.
• Annoying captchas
One subject stated “Test human is annoying”. The “test human” is captchas;
many of the subjects mentioned this, finding it “ridiculously annoying”.
Did the results prove that the subjects did not like captchas? Were the
subjects willing to do captchas? How much effort was it for them to do
captchas? These answers can be found in the responses to “Q3. What do
you think about social bookmark submission?”, “Q5. What do you think
about social bookmarking registration?” and “Q6. What do you think about
adding a comment?”. The details are displayed in Figure 4-3; captchas
were found to be one of the tasks subjects were most un-willing to
complete.
• Feedback on same topic
One subject wrote that “Informational links about the topic would have been
helpful” after using the high social feedback Bligg buttons.
Did the questionnaires’ results prove that the topic links were helpful? The
answers to “Q7. What do you think about social bookmarking feedback?”
are displayed below:
66
Figure 4-4: Mean usefulness of social feedback (N=60) For the social feedback given, the results show that the most useful
feedbacks were “The Number of Bookmarked”, “Bookmark Details”, and
“Top in Music”. “Top in Music” relates to “informational links about the topic”.
Subjects were less interested in “Related by Source” or “People Who
Saved This Also Saved”.
4.2.2 Hypothesis testing
Hypothesis testing is achieved by choosing the appropriate menus from the
SPSS software programs to test each of the hypotheses using relevant
statistical tests. The results of these tests can determine whether or not the
hypotheses are substantiated. This study’s hypotheses are:
l Social bookmark button’s cognitive effort affects the likelihood of using social
bookmarking on the Internet.
l Social bookmark button’s social feedback affects the likelihood of using
social bookmarking on the Internet.
67
Were these hypotheses proved? Firstly, the questionnaire responses were
used to find out at which stage the subjects lost interest and no longer wanted
to use social bookmarking. The researcher hoped that the subjects would
change their opinions after using the Bligg social bookmark button. The below
table shows the results:
Table 4-2: Likelihood of using social bookmarking by usage stages
Table 4-2 shows that the subjects did not change their opinions during the first
two stages (unknown social bookmarking, and explaining and seeing social
bookmarking and the Bligg social bookmark button), and the six groups of
subjects had non-significant differences with respect to attitude to social
bookmarking (p=0.85; p=0.48; p=0.32; p=0.44), so before without being
exposed to the interface, there were no effects. However, the subjects started
to change their opinions after using the first version of the Bligg social
bookmark button (p=0.17; p=0.04); this change was even more obvious after
they had used the second version of the button (p=0.00; p=0.00).
Did the subjects become more likely to use the Bligg social bookmark button
after they had used the two different versions? Because the differences
between the two versions were clear, the researcher hoped that the subjects
would change their opinions after using both versions. The table below displays
the results:
68
Table 4-3: Likelihood of using the Bligg social bookmark button by Bligg buttons
Table 4-3 proves that the subjects changed their opinions after having used
both versions of the Bligg button.
Did the subjects have the different opinions on Bligg buttons’ ease of use and
usefulness?
Table 4-4: Bligg buttons' ease of use and usefulness by Bligg buttons
The results show that subjects felt that there were clear differences in ease of
use and usefulness among the different versions. Were these differences
based on Bligg buttons’ cognitive effort and social feedback?
Table 4-5: Mean easy to use social bookmark buttons by cognitive effort and social feedback (N=60)
No Social Feedback
Low Social Feedback
High Social Feedback
Low Cognitive Effort 8.25 8.05 7.85
High Cognitive Effort 4.00 3.50 3.05
Table 4-6: Mean usefulness of social bookmark buttons by cognitive effort and social feedback (N=60)
No Social Feedback
Low Social Feedback
High Social Feedback
Low Cognitive Effort 2.30 5.70 8.45
High Cognitive Effort 1.90 4.40 7.25
69
The subjects thought that the buttons requiring low cognitive effort were easier
to use than those requiring high cognitive effort and that buttons offering less
social feedback were easier to use than those offering more (see Table 4-5).
However, the subjects also thought that the buttons offering more social
feedback were more useful than those offering less (see Table 4-6).
It is clear subjects’ opinions on social bookmarking usage and Bligg button
usage were changed by the different Bligg buttons. Moreover, the Bligg buttons’
ease of use and usefulness were different depending on the relative levels of
cognitive effort and social feedback. Did these change based on Bligg buttons’
cognitive effort and social feedback?
Table 4-7: Mean like to use and will use social bookmarking by cognitive effort and social feedback (N=60)
Ø Like To Use: No
Social Feedback Low
Social Feedback High
Social Feedback Low
Cognitive Effort 8.35 7.80 7.65
High Cognitive Effort 5.85 6.15 5.00
Ø Will Use: No
Social Feedback Low
Social Feedback High
Social Feedback Low
Cognitive Effort 8.25 7.70 7.40
High Cognitive Effort 5.65 5.75 4.90
Table 4-8: Mean will use and would recommend social bookmark button by cognitive effort and social feedback (N=60)
Ø Will Use: None
Social Feedback Low
Social Feedback High
Social Feedback Low
Cognitive Effort 8.05 7.65 6.85
High Cognitive Effort 5.20 4.90 4.10
Ø Would Recommend: None
Social Feedback Low
Social Feedback High
Social Feedback Low
Cognitive Effort 7.65 7.65 6.50
High Cognitive Effort 4.95 4.70 3.90
Table 4-7 shows the mean intended social bookmarking usage by cognitive
70
effort and social feedback, and the subjects would like to use “Low Cognitive
Effort” social bookmarking more than “High Cognitive Effort” social
bookmarking in the future. For “Low Cognitive Effort” buttons, the more social
feedback offered, the less likely it is that people will use social bookmarking.
For “High Cognitive Effort” buttons, the likelihood of using “Low Social
Feedback” social bookmarking was higher than the other two sorts of social
feedback, but the subjects preferred less social feedback. For both low and
high cognitive effort, less social feedback (which appears to be the whole point
of social bookmarking) makes subjects more likely to use the button. Table 4-8
shows that the subjects preferred to less cognitive effort and less social
feedback and would recommend this sort of social bookmark button to a friend.
Next, it was necessary to find out and prove whether the Bligg button’s
cognitive effort and social feedback affected subjects’ likelihood of using social
bookmarking or not, and prove the hypotheses. The researcher thought that
social bookmark buttons’ cognitive effort affected the likelihood of using social
bookmarking; this assumption was proved.
Hypothesis 1: Social bookmark button’s cognitive effort affects the likelihood
of using social bookmarking on the Internet. (Significant)
Table 4-9: Likelihood of using social bookmarking by social bookmark button's cognitive effort
As Table 4-9 shows, cognitive effort had a significant effect on social bookmark
intention to use (paired t-test, p<0.001). Social bookmark button’s cognitive
effort was significant in terms of how likely people were to use social
bookmarking in the future (F=27.29, df=118), and was also significant for will
71
use social bookmarking in the future (F=25.70, df=118). This proves that social
bookmark button’s cognitive effort affects the likelihood that people will use
social bookmarking on the Internet. Subjects paid attention to how much effort
they spent when they were using the Bligg social bookmark button. This was
expected.
Hypothesis 2: Social bookmark button’s social feedback affects the likelihood
of using social bookmarking on the Internet. (Not Significant)
Table 4-10: Likelihood of using social bookmarking by social bookmark button's social feedback
From the results of Hypothesis 2, the different levels of social feedback had no
significant effect on social bookmark intention to use (ANOVA, p=0.35; p=0.36).
4.3 Interpretation of results
One subject said “bookmarking is supposed to be simple (and) fast like first one
(low cognitive effort and no social feedback)”; subjects wanted everything to be
easy and simple, to save them time. Subjects did not want to enter lots of
details to register and bookmark, and hated annoying captchas. Subjects paid
attention to the amount of cognitive effort required of them. Subjects
commented: “I just want to bookmark, why so many questions (to be
answered)?” and “shorter process for bookmarking”.
Social bookmark button’s social feedback does not affect the likelihood of using
social bookmarking on the Internet. This was problematic, as social feedback
was the primary reason proposed that subjects used social bookmarking in the
first place. If social feedback level did not affect usage, what did? Certainly it
was expected that less effort would increase the likelihood of future usage, but
72
minimal effort alone is hardly a good reason to use social bookmarking in the
first place. For both low and high cognitive effort, buttons offering less social
feedback, which is apparently the whole point of social bookmarking, attracted
the subjects (see Table 4-7). Moreover, the results of the mean easy to use
social bookmark buttons show similar findings (see Table 4-5). The subjects
preferred less cognitive effort and less social feedback and would recommend
this sort of social bookmark button to a friend (see Table 4-8). However, the
subjects thought buttons offering more social feedback were more useful than
those offering less (see Table 4-6).
How to explain this strange finding? The subjects knew that buttons offering
more social feedback were more useful, but they preferred those with less
because they were easier to use. Is social feedback not important in terms of
social bookmarking usage? Or is social feedback affected by other critical
factors, meaning that it does not work for social bookmarking usage? Table 4-7
offers an answer. There is a slight positive increase from no social feedback to
low social feedback with the high cognitive effort button. If the subjects do not
use social bookmarking for social feedback, then why do they prefer low social
feedback to no social feedback?
On re-analysing the screens presented in the six cases it became apparent that
increasing social feedback may also have increased cognitive effort, as then
subjects had to read more – the high social feedback page was longer than the
low social feedback page. The top most useful feedback “Top in Music” was
listed on the bottom of the feedback page. If the subjects wanted to read “Top in
Music”, they had to scroll down to read more information on the high social
feedback page. This confounding of cognitive effort and amount of social
feedback may have caused the non-significant effect on intention to use social
bookmarking. If more social feedback increased likelihood of future use and
more cognitive effort decreased it, the two effects could have cancelled out. It
was decided to repeat the study to eliminate this confounding.
73
Chapter 5: Second Phase Study Method
5.1 Pilot studies
In order to make sure the second study could prove reading screen effort and
confounded effort (forcing subjects to read social feedback they find
un-interesting) is critical to whether or not social feedback is useful; two pilot
studies were done to achieve this – increasing feedback without also
increasing cognitive effort (i.e. giving more feedback for the same reading
effort).
5.1.1 First pilot study
In the first pilot study, three sorts of Bligg buttons were created. They were
“Low Cognitive Effort and No Social Feedback”, “Low Cognitive Effort and No
Choice of Social Feedback”, and “Low Cognitive Effort and Choice of Social
Feedback” (see Appendix K). Using low cognitive effort version ensured that
the subjects did not put too much effort on Bligg button. Moreover, processing
steps “Register” and “Login” were skipped for less the subjects’ effort puting.
Considering and proving reading screen effort is a critical factor in whether
social feedback works on social bookmarking usage, the three sorts of
feedback pages had the same reading length, images and layout.
When a subject clicked one sort of Bligg button, the Bookmark page would be
opened firstly. Subjects did not need to enter any details into the Bookmark
page because all details were entered already. After clicking the “Submit”
button from the Bookmark page, the feedback page would be opened. For no
social feedback version, there were “Thanks for bookmarked this song”
information displayed, but no social feedback included. For no choice of social
feedback version, Bligg number, song title, song description, first submitted
user’s username, submitted time and topic were list. For choice of social
feedback version, it covered no choice of social feedback’s details, and also
included one link “See other bookmarks in Pop Music”. After clicked the link,
74
five more social bookmarks were listed on a popup window.
In the first pilot study, three subjects were asked to try two different sorts of
Bligg buttons, and they were asked to speak aloud when they were using the
Bligg Buttons. Their voice was recorded for protocol analysis, and the analysis
details were list in Appendix L. From the analysis, the researcher found that the
subjects liked more social feedback and the link from choice of social feedback.
However, the subjects still paid more attention on Bookmark page (such as
changing bookmark’s details), and one subject did not realise the difference
from no choice of social feedback and choice social feedback.
5.1.2 Second pilot study
From the first pilot study’s findings, the researcher improved Bligg button –
skipped Bookmark page, added one more link “See other bookmarks related by
Keyword” into choice of social feedback version, and changed no choice of
social feedback version into “Song information social feedback” and “Bligg
number social feedback” versions. For song information social feedback, it
included song title, song description, first submitted user’s username,
submitted time and topic. For Bligg number social feedback, it covered all
details of song information social feedback, and also included Bligg number.
These changes could make sure the subjects paid attention to social feedback
and the links, and also could analyse the different sorts of social feedback more
accurately. Considering reading screen effort, the four versions of feedback
pages had the same reading length, images and layout.
In the second pilot study, three subjects were asked to try two different sorts of
Bligg buttons, and they were asked to speak aloud when they were using the
Bligg Buttons. Their voices were recorded for protocol analysis, and the
analysis details were list in Appendix M. From the analysis, the researcher
found that the subjects paid much more attention on social feedback and the
links, and the four sorts of social feedback usages were totally different.
75
To ensure the subjects found it easy to follow the study and did not lose steps,
procedure webpage, song list webpage, song details webpage and Bligg
buttons were used and checked using cognitive walkthrough. The results
showed that this study was easy to follow and use, and the details are listed in
Appendix N.
5.2 Last social bookmark button – Bligg
Depending on the second pilot study’s results, the last versions of Bligg buttons
were used for the second study. More details are described in next chapters.
5.2.1 Processing order
In order to prove that reading screen effort and confounded effort are critical to
whether social feedback works for social bookmarking usage, the last Bligg
social bookmark button just had one processing step – feedback. After the
subjects clicked the Bligg button, they did not have to register or login or enter
details to bookmark – the feedback page would just open. This allowed
subjects to pay attention to the feedback and saved time.
5.2.2 Feedback versions
Because reading screen effort and confounded effort are critical to whether or
not social feedback affects social bookmarking usage, four types of social
feedback versions were created. In each case, the amount of social feedback
cumulated, but the reading length was the same. Images and layout were also
the same.
• No social feedback
No social feedback was provided to subjects and “Thanks for using Bligg
button” was displayed. (This mirrors when Digg users log in and the new
bookmark is submitted for the first time.)
76
• Song information social feedback
Subjects could read song information (Title and Description), which was
written by the first user to submit the bookmark. The first user’s
bookmark details (Username, Topic and Submitted Time) were also
displayed. (This mirrors when Digg users log in and the new bookmark is
submitted for the first time.)
• Song information social feedback plus Bligg number social
feedback
This version included the above version’s details, and also included the
Bligg number – this shows how many other Bligg users bookmarked this
song. (This mirrors when Digg users log in and the new bookmark is
submitted for the first time.)
• Song information social feedback, Bligg number social feedback
plus choice of social feedback
This version included the above version’s details, and also included two
links (“See other bookmarks in Pop Music” and “See other bookmarks
related by Keyword”), which could show subjects more social bookmarks.
The subjects were not forced to read the linked social feedback, but
could do so if they wanted to. (This is similar to the Digg version when
Digg users log in and the new bookmark is submitted for the first time.)
5.2.3 Interface design
The last Bligg button’s interface did not change – the first study interface was
used. The interface screenshots are displayed in Appendix O.
5.2.4 Database
For the last Bligg button, the Beatles mini website was used and the only
change was that ten songs were listed on the song list page. This change is
77
due to the changed research method (see Chapter 5.3).
For social feedback, the details created by the researcher were based on the
Digg version. The social bookmark resources used for the links were from a
Beatles song bookmark on the website digg.com; all the details were real and
collected by September 10th, 2009.
Database access was similar to the first study – JavaScript programming
language was used to transfer details and save them into a .txt file using PHP
programming language. In the end, the saved details included the submitted
data and time, the song ID, the Bligg version number, username, title,
description, thumbnail, feedback time taken, clicked link type and linked
feedback time taken.
5.3 Research design
The research method was similar to the first study, which included three phases
(such as unknown social bookmarking, explaining and seeing social
bookmarking and the Bligg social bookmark button, and using the Bligg social
bookmark button) and investigations were carried out after each phase. It
detected at which stage subjects lost interest and stopped using social
bookmarking. Although the first study found that the subjects changed their
opinions on social bookmarking after using the Bligg social bookmark button,
these three phases were applied for letting the subjects to realise social
bookmarking well, and to get the second study subjects to the same stage as
the first study subjects were at. This was to keep the research consistent.
For this study, every subject had to go through the three phases and try the four
versions of the Bligg social bookmark button in a random order. This
experiment was repeated measures and the research data were collected from
questionnaires and the time taken for each version of Bligg button was
measured. The aim was to find out which sort of social feedback affects the
subjects’ social bookmarking usage. Because each subject had to use four
78
versions of the Bligg button, ten songs were listed on the song list page. This
gave the subjects more choices.
5.4 Subject requirement
Twenty-four subjects (newcomers to social bookmarking) were found from
within Massey University in Auckland. This was to keep the research consistent.
Because there were twenty-four different random orders to try the four versions
of Bligg buttons, the sample size was twenty-four. This was a limited sample –
the research samples should be selected from other universities as well for
future studies.
5.5 Questionnaire design
The questionnaire (see Appendix P) was similar to the first study’s
questionnaire. The questions were understandable, unambiguous, unbiased
and inoffensive. In order to get an indication of the magnitude of the difference
recorded, interval scales were used. The interval scale was designed to be
easy to use, unbiased, exhaustive and sensitive; all of the interval scales used
an unbalanced rating scale (10-point scale) that did not have a neutral point.
This was better at eliciting unbiased responses. Some questions were
open-ended, so as to collect qualitative responses. Because the second
study’s questionnaire was similar to the first, just those questions that were
different questions are described below:
Table 5-1: Questionnaire questions' details
Question Description
Q2 This question includes two questions, both of which were answered after the four versions had been used. The first question used a 10-point semantic differential scale (interval scale), and asked the subject’s preference. The answers could show each version of Bligg buttons’ preference point, and found that how much the subjects preferred each version of social feedback. The second question was open-ended and aimed to ascertain some of the reasons that subjects preferred one version to another. The researcher might find more factors that affect the likelihood of the user using social bookmarking from the open-ended questions.
79
Q3 This question included sub questions, which were answered after the four versions of the Bligg button had been used. The sub questions were about each social feedback’s usefulness and attractiveness. These sub questions used a 10-point semantic differential scale (interval scale) that computed the standard deviations of the responses. The answers would suggest which kind of social feedback the subjects found useful and which kind of social feedback the subjects liked to use. The findings were very helpful in improving current social bookmark buttons’ social feedback.
Q4 This question included sub questions, which were answered after the four versions of the Bligg button had been used. The sub questions were about being willing to click and make an effort with the links. These sub questions used a 10-point semantic differential scale (interval scale) that computed the standard deviations of the responses. The answers would show how willing the subjects were to click the links for more social feedback and how much effort the subjects wanted to expend. The findings highlighted the subjects’ opinions on choice of social feedback and improving social feedback.
5.6 Procedure plan and measurement method
The details of the procedure plan and measurement method were the same as
that of the first study (see Chapter 3.5 and Chapter 3.6).
80
Chapter 6: Second Phase Study Results
6.1 Getting data ready for analysis
The same method was used as for the first study.
6.1.1 Editing data
For the second study’s questionnaire, two open-ended questions needed to be
edited. The first was “Q2. Reasons? (How much do you prefer them?)”, and the
second was “Q8. Any general comments?”. Because the researchers checked
the completed questionnaires in subjects’ presence, all of answers were written
clearly. The answers to these two questions are listed in Appendix Q.
6.1.2 Handling blank responses
Because the researchers checked the completed questionnaires in the
subjects’ presence, all of questions were answered reasonably. This step was
skipped.
6.1.3 Coding data
In order to analyse data, responses had to be coded. Because most of the
questions had an interval scale, the answers to these questions were the data
(no coding data). No nominal scales were included in the questionnaire.
6.1.4 Categorising data
In the second study, every subject tried the four versions of the Bligg button in a
different order, so “Q1. Answer the following questions based on your
81
experience, or what you just saw or tried out:” STEP III, STEP IV, STEP V and
STEP VI had to make sure on the order of “No social feedback” à “Song
information social feedback” à “Song information social feedback Plus Bligg
number social feedback” à “Song information social feedback, Bligg number
social feedback Plus choice of social feedback”. For the same reason, answers
to “Q2. Based on trying the four systems out, how much do you prefer them?”
had to be in order, from less social feedback to more social feedback.
For other questions with an interval scale, the answers were not changed
because the lower number was negative and the higher number was positive.
6.1.5 Entering data
After coding and categorising data, these data had to be entered into SPSS for
analysis. The raw data table’s columns were questions and rows were subjects.
6.2 Data analysis
In data analysis, there are two objectives – getting a feel for the data and
testing the hypotheses developed for the research. The results from this
section can be used to interpret in the next section.
6.2.1 Feel for the data
Getting a feel for the data can give one preliminary ideas. From the
questionnaire’s responses, the responses to the open-ended questions “Q2.
Reasons? (How much do you prefer them?)” and “Q5. Any general
comments?” could give one a feel for the data. The details are described below:
• Useful and interesting information
One subject answered “... the more information the better...”, and all of the
subjects liked the 4th system (song information social feedback, Bligg number
social feedback plus choice of social feedback), because “the 4th system gave
82
me more information” and “... is a very interesting system as well as good
information involved”. Moreover, one subject stated that “this system will be
more useful than the others” and “...will prefer to use it (in the future)”.
Did the results prove these points? Did the subjects prefer more social
feedback? Did the subjects like more social feedback? Was more social
feedback more useful? The answers are below:
Figure 6-1: Mean usefulness, liking to use and preference for social
bookmark buttons by social feedback (N=24) The results clearly show that the subjects thought more social feedback were
more useful and they liked to use this sort of social feedback. Moreover, the
subjects’ opinions were definite. For example, the mean like to use social
bookmark buttons by social feedback:
Figure 6-2: Mean like to use social bookmark buttons by social feedback (N=24)
83
Each social feedback’s range of like to use social bookmark buttons was short,
which showed that the subjects had similar opinions on the different sorts of
social feedback.
• Useful links
The subjects who preferred more social feedback had another reason – the
useful links. One subject mentioned “I prefer to use the 4th system (song
information social feedback, Bligg number social feedback plus choice of social
feedback) because of the wonderful links and also user can find out more
songs they like”; other subjects agreed: “(the 4th system) had other options in
the bottom – as a quick link to see other bookmarks”; “(I) would use the 4th
system in the future because there are recommend links down the bottom”.
The subjects preferred the system with more social feedback because of the
useful links; did this affect link usage? Were the subjects willing to click the links
for more social feedback? How much effort did the subjects think to expend
clicking the links? Did all of the subjects click the links? The results are below:
Table 6-1: Mean willing and effort to click the links by gender (N=24)
Gender Willing to Click the Links Effort to Click the Links
Male 8.83 2.33
Female 8.08 2.67
The questionnaire’s results show that all of the subjects were willing to click the
links and that they did not think it was too much effort to click the links. The
male subjects were more willing to click the links than the female subjects, and
they considered it to be less of an effort to click the links than did the female
subjects.
In reality, did every subject click the links for more social feedback? The answer
was in the database, because all the link click records were saved into it. For
the link choices in the final social feedback level, 71% of the subjects clicked
the links for more social feedback. 82% of the subjects clicked the link “See
other bookmarks in Pop Music” and 18% of the subjects clicked the link “See
other bookmarks related by Keywords”. Moreover, more male subjects clicked
the links than female subjects.
84
• Ease of use and time taken
The research found that “71% of subjects clicked the links for more social
feedback”; however did the extra time taken make the subjects think the social
feedback system was hard to use? The answer is no. The details are listed
below:
Table 6-2: Mean time taken in milliseconds by social feedback (N=24)
Time Taken
No Social Feedback 14300.13
Song Information Social Feedback 9192.44
Above, Plus Bligg Number Social Feedback 11267.50
Above, Plus Choice of Social Feedback Feedback Page 12519.72
26205.72 Links Page 13686.00
From this table, it is not hard to see that the subjects took about two to three
times longer with the choice of social feedback system than the other systems.
However, the longer time taken did not affect the subjects’ opinions on how
easy it was to use this system:
Figure 6-3: Mean ease of use Bligg buttons by social feedback (N=24)
85
The subjects thought the choice of social bookmark system was the easiest to
use. This proves that the links were easy to use and that the subjects did not
think a system was easy to use based on the time it took.
6.2.2 Hypothesis testing
For the second study, the hypothesis was:
l Social bookmark button’s social feedback affects the likelihood of using
social bookmarking on the Internet.
Was the hypothesis proved? First, let’s look at the mean of social bookmarking
and social bookmark button usage by social feedback.
Table 6-3: Mean like to use and will use social bookmarking by social feedback (N=24)
Ø Like To Use:
No Social Feedback
Song Information
Social Feedback
Left, Plus Bligg Number Social
Feedback
Left, Plus Choice of Social Feedback
1.88 4.63 6.21 9.46
Ø Will Use:
No Social Feedback
Song Information
Social Feedback
Left, Plus Bligg Number Social
Feedback
Left, Plus Choice of Social Feedback
1.96 4.54 6.25 9.42
Table 6-4: Mean will use and would recommend social bookmark button by social feedback (N=24)
Ø Will Use:
No Social Feedback
Song Information
Social Feedback
Left, Plus Bligg Number Social
Feedback
Left, Plus Choice of Social Feedback
2.00 4.54 6.33 9.38
Ø Would Recommend:
No Social Feedback
Song Information
Social Feedback
Left, Plus Bligg Number Social
Feedback
Left, Plus Choice of Social Feedback
2.13 4.67 6.29 9.33
These tables show that the subjects would like to use the more social feedback
system in the future and that the more social feedback system affects social
bookmarking usage.
86
Because the different Bligg buttons were created based on social feedback, did
the social bookmark button’s social feedback affect the likelihood of using
social bookmarking?
Next, we needed to prove whether Bligg button’s social feedback affected the
likelihood of using social bookmarking or not, and prove the hypothesis. The
researcher thought that the social bookmark button’s social feedback affected
the likelihood of using social bookmarking; this assumption was proved.
Hypothesis: Social bookmark button’s social feedback affects the likelihood of
using social bookmarking on the Internet. (Significant)
Table 6-5: Likelihood of using social bookmarking by social bookmark button's social feedback
Once cognitive effort was controlled for, social feedback had a significant effect
on the likelihood of using and intention to use social bookmarking in the future
(ANOVA, p<0.001 for both criteria). Table 6.5 shows that social bookmark
button’s social feedback is significant for like to use social bookmarking in the
future (F=139.92, df=3), and also was significant for will use social
bookmarking in the future (F=114.17, df=3). This proves that social bookmark
button’s social feedback affects the likelihood of using social bookmarking on
the Internet. The subjects paid attention to how much social feedback they got
when they were using the Bligg social bookmark button.
6.3 Interpretation of results
One subject said “the more information the better”. Subjects preferred to get
more interesting and useful information from others. However, more social
87
feedback did not mean more social bookmarking usage. The results of the first
study and this study, show that reading screen effort and confounded effort
were proved as critical factors affecting social feedback works on social
bookmarking usage.
Based on these two studies, a social bookmark button should include the
following:
Table 6-6: Social bookmark button details
Details Notes
Registration Username, Email Address, Password
Re-type Password and Terms and Conditions can exist for logic.
Bookmark URL, Title, Description, Thumbnail
URL, Title, and Description can be entered automatically.
Comment Comment An un-required entry.
Feedback
The Number of Bookmarked, Bookmark Details, Top in Music (Same Topic Bookmarks), Related by Keyword
Page length should be controlled (not too long). Top in Music (Same Topic Bookmarks) and Related by Keyword can be made links.
This research suggests that a good registration page would be similar to Reddit;
a good bookmark page would be similar to that of Digg; a good comment
section would be like Reddit’s; a good feedback page would be similar to that of
Digg. Generally, a good social bookmark button requires low cognitive effort
and offers a large amount of social feedback. The social feedback page should
not too long and some social feedback should only be able to be read after
having clicked related links.
88
Chapter 7: Discussion
7.1 Conclusions
These results suggest that the reading effort users are willing to put into social
bookmark feedback is less than many designers suppose and may well be
limited to a single screen with no scroll downs. In particular, users had little time
for captchas (having to respond to “are you human” requests). Increasing
feedback length, which increased cognitive reading effort, had no significant
effect. It seemed that users did not even value the feedback upon which social
bookmarking is based.
However when the amount of feedback was kept to a constant single screen
size, then increasing the social bookmark feedback did increase the likelihood
that people would use the button. This suggests that perhaps one reason for
people’s relatively slow uptake of social bookmarking, compared to say social
networking, may be that its social feedback is not cognitively efficient, i.e. users
will not value social feedback that is too hard to get, or conversely, they want
social feedback but will not put a lot of effort into getting it. In sum, reasonable
cognitive effort is a critical enabling factor for the effect of the social feedback
social bookmarks provide.
7.2 Future potential
Future research should address further this critical issue of how social and
cognitive interface factors interact in socio-technical system success.
89
References
Alexa. (2009). Facebook. Retrieved May 25, 2009, from http://alexa.com/siteinfo/facebook.com+digg.com
Alexa. (2010a). Digg.com. Retrieved February 20, 2010, from
http://www.alexa.com/siteinfo/digg.com?p=tgraph&r=home_home Alexa. (2010b). Delicious.com. Retrieved February 20, 2010, from
http://www.alexa.com/siteinfo/delicious.com?p=tgraph&r=home_home Arrington, M. (2007, September 6). Exclusive: Screen Shots And Feature
Overview of Delicious 2.0 Preview. Retrieved January 26, 2010, from http://techcrunch.com/2007/09/06/exclusive-screen-shots-and-feature-overview-of-delicious-20-preview/
Bailey, R. N. (1996). Human Performance Engineering: Designing High Quality
Professional User Interfaces For Computer Products, Applications And Systems (3rd ed.). Prentice Hall PTR.
Bao, S., Xue, G., Wu, X., Yu, Y., Fei, B., & Su, Z. (2007). Optimizing Web
Search Using Social Annotations [Electronic version]. International World Wide Web Conference: WWW '07 Proceedings of the 16th international conference on World Wide Web, 07, 501-510.
Bateman, S., Muller, M. J., & Freyne, J. (2009). Personalized Retrieval in Social
Bookmarking [Electronic version]. Conference on Supporting Group Work: Proceedings of the ACM 2009 international conference on Supporting group work, 09, 91-94.
BBC. (2008). Rugby union on the BBC. Retrieved April 3, 2008, from
http://news.bbc.co.uk/sport2/hi/rugby_union/4797831.stm BBC. (2009). Social Bookmarking Links. Retrieved June 12, 2009, from
http://news.bbc.co.uk/2/hi/help/6915817.stm Beaird, J. (2007). The Principles of Beautiful Web Design. Collingwood, Vic.:
SitePoint Pty. Ltd.. Bernard, M., & Mills, M. (2000). So, what size and type of font should I use on
my website? Usability News Summer. Retrieved January 25, 2010, from http://wsupsy.psy.twsu.edu/surl/usabilitynews/2S/font.htm.
Bernard, M., Liao, C. H., & Mills, M. (2001). The effects of font type and size on
the legibility and reading time of online text by older adults [Electronic
90
version]. Conference on Human Factors in Computing Systems: CHI '01 extended abstracts on Human factors in computing systems, 2001, 175-176.
Bian, J., Liu, Y., Agichtein, E., & Zha, H. (2008a). A few bad votes too many?:
towards robust ranking in social media [Electronic version]. AIRWeb: Proceedings of the 4th international workshop on Adversarial information retrieval on the web, 295, 53-60.
Bian, J., Liu, Y., Agichtein, E., & Zha, H. (2008b). Finding the right facts in the
crowd: factoid question answering over social media [Electronic version]. In WWW '08: Proceeding of the 17th international conference on World Wide Web, 08, 467-476.
Biggs, J. B. (1993). From Theory to Practise: A Cognitive Systems Approach.
Higher Education Research & Development, 12(1), 73-85. Blair, L. (1975). Rhythms of Vision: The Changing Patterns of Belief. London:
Croom Helm. Burkard, J. (2009). Social Bookmarking / Sharing Buttons That Don’t Suck.
Retrieved May 28, 2009, from http://johannburkard.de/blog/www/social-bookmarking-sharing-button.html
Chen, F., Scripps, J., & Tan, P. N. (2008). Link Mining for a Social Bookmarking
Web Site [Electronic version]. Web Intelligence and Intelligent Agent Technology, 2008. WI-IAT '08. IEEE/WIC/ACM International Conference on, 1(9-12 Dec. 2008), 169-175.
Chou, E. (2002). Redesigning a large and complex website: how to begin, and
a method for success [Electronic version]. User Services Conference: Proceedings of the 30th annual ACM SIGUCCS conference on User services, 2002, 22-28.
Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process
[Electronic version]. Cognition, 22(1), 1-39. Cockburn, A., & McKenzie, B. (2001). What Do Web Users Do? An Empirical
Analysis of Web Use [Electronic version]. International Journal of Human-Computer Studies, 54(6), 903-922.
Compete. (2008). Site Analytics. Retrieved April 11, 2008, from
http://siteanalytics.compete.com/digg.com+del.icio.us+stumbleupon.com+reddit.com+furl.net/?metric=uv
Compete. (2009). Digg.com vs Facebook.com. Retrieved May 25, 2009, from
http://siteanalytics.compete.com/digg.com+facebook.com/ Compete. (2009, March). Site Analytics. Retrieved April 25, 2009, from
http://siteanalytics.compete.com/delicious.com/
91
Compete. (2010). Site Analytics. Retrieved February 17, 2010, from
http://siteanalytics.compete.com/digg.com+delicious.com/ Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance of information technology [Electronic version]. MIS Quarterly, 13(3), 318-340.
Delicious. (2008, July 31). Oh happy day — the new Delicious is here.
Retrieved January 25, 2010, from http://blog.delicious.com/blog/2008/07/oh-happy-day.html
Delicious. (2009, August 7). Sharing Made Easier: Email and Tweet Your
Bookmarks. Retrieved January 30, 2010, from http://blog.delicious.com/blog/2009/08/sharing-made-easier-email-and-tweet-your-bookmarks.html
Digg. (2010). Digg Most Recent. Retrieved January 25, 2010, from
http://digg.com/ DoshDosh. (2007). Are Social Bookmarking Buttons Useless?. Retrieved April
3, 2008, from http://www.doshdosh.com/are-social-bookmarking-buttons-useless/
Dover, D. (2008). Reddit, Stumbleupon, Del.icio.us and Hacker News
Algorithms Exposed!. Retrieved January 8, 2010, from http://www.seomoz.org/blog/reddit-stumbleupon-delicious-and-hacker-news-algorithms-exposed
Dugan, C., Muller, M., Millen, D. R., Geyer, W., Brownholtz, B., & Moore. M.
(2007). The Dogear Game: a Social Bookmark Recommender System [Electronic version]. GROUP '07: Proceedings of the 2007 international ACM conference on Supporting group work, 07, 387-390.
Educause. (2005, May). 7 Things You Should Know About Social Bookmarking.
Retrieved March 16, 2008, from http://www.educause.edu/ir/library/pdf/ELI7001.pdf
Farooq, U., Kannampallil, T. G., Song, Y., Ganoe, C. H., Carroll, J. M., & Giles,
C. L. (2007). Evaluating Tagging Behavior in Social Bookmarking Systems: Metrics and Design Heuristics [Electronic version]. GROUP '07: Proceedings of the 2007 international ACM conference on Supporting group work, 07, 351-360.
Farooq, U., Song, Y., Carroll, J. M., & Giles, C. L. (2007). Social Bookmarking
for Scholarly Digital Libraries [Electronic version]. IEEE INTERNET COMPUTING, 11(6), 29-35.
Farzan, R., Coyle, M., Freyne, J., Brusilovsky, P., & Smyth, B. (2007). ASSIST:
adaptive social support for information space traversal [Electronic version].
92
Conference on Hypertext and Hypermedia: Proceedings of the eighteenth conference on Hypertext and hypermedia, 2007, 199-208.
Festa, P. (1999). Net Surfers can Backtrack with Backflip. Retrieved May 2,
2008, from http://www.news.com/2100-1023-233926.html Fishbein, M., & Ajzen, I. (1975). Belief, Attitude, Intention and Behaviour: An
Introduction to Theory and Research. Reading, MA: Addison-Wesley. Flanders, V., & Willis, M. (1998). Web Pages That Suck: Learn Good Design by
Looking at Bad Design. San Francisco, CA: SYBEX Inc.. Florczak, B. (2007). Five reasons why I don’t use Reddit for social bookmarking.
Retrieved April 9, 2008, from http://florchakh.com/2007/08/22/five-reasons-why-i-dont-use-reddit-for-social-bookmarking.html
Ford, G., & Gelderblom, H. (2003). The effects of culture on performance
achieved through the use of human computer interaction [Electronic version]. SAICSIT: Proceedings of the 2003 annual research conference of the South African institute of computer scientists and information technologists on Enablement through technology, 47, 218-230.
Forman, C., Ghose, A., & Wiesenfeld, B. (2008). Examining the Relationship
Between Reviews and Sales: The Role of Reviewer Identity Disclosure in Electronic Markets [Electronic version]. INFORMATION SYSTEMS RESEARCH, 19(3), 291-313.
Freyne, J., Smyth, B., Coyle, M., Balfe, E., & Briggs, P. (2004). Further
Experiments on Collaborative Ranking in Community-Based Web Search [Electronic version]. Artificial Intelligence Review, 21(3-4), 229-252.
Gladwell, M. (2000). The Tipping Point : How Little Things Can Make A Big
Difference. Boston: Little, Brown. Golbeck, J. (2009). Trust and Nuanced Profile Similarity in Online Social
Networks [Electronic version]. ACM Transactions on the Web (TWEB), 3(4), 12:1-12:33.
Golder, S. A., & Huberman, B. A. (2006). Usage patterns of collaborative
tagging systems [Electronic version]. Information Science, 32(2), 198-208. Grudin, J. (2005). Three Faces of Human-Computer Interaction [Electronic
version]. IEEE Annals of the History of Computing, 27(4), 46-62. Guha, R., Kumar, R., Raghavan, P., & Tomkins, A. (2004). Propagation of trust
and distrust [Electronic version]. International World Wide Web Conference: Proceedings of the 13th international conference on World Wide Web, 04, 403-412.
93
Guy, M., & Tonkin, E. (2006, January). Folksonomies: Tidying up Tags? [Electronic version]. D-Lib Magazine, 12(1). Retrieved April 14, 2008, from http://www.dlib.org/dlib/january06/guy/01guy.html
Harwood, P., & Rainie, L. (2004). Use of the Internet in places other than home
or work: A Pew Internet project data memo. Retrieved January 25, 2010, from http://www.pewinternet.org/pdfs/PIP_Other_Places.pdf
Heymann, P., Koutrika, G., & Garcia-Molina, H. (2008, February). Can Social
Bookmarking Improve Web Search? [Electronic version].WSDM '08: Proceedings of the international conference on Web search and web data mining, 08, 195-206.
Hines, K. (2009, April). 8 Reasons to Use Delicious for Social Bookmarking.
Retrieved June 3, 2009, from http://kikolani.com/8-reasons-delicious-social-bookmarking.html
Hix, D., & Schulman, R. S. (1991). Human–computer interface development
tools: A methodology for their evaluation [Electronic version]. Communications of the ACM, 34(3), 75-87.
IBM. (n.d.). Features and Benefits – Lotus Connections. Retrieved May 2, 2008,
from http://www-306.ibm.com/software/lotus/products/connections/features.html
Ivory, M. Y., & Megraw, R. (2005). Evolution of Web Site Design Patterns
[Electronic version]. ACM Transactions on Information Systems (TOIS), 23(4), 463-497.
Jarvis, P., Holford, J., & Griffin, C. (2003). The Theory & Practice of Learning.
Sterling, VA: Stylus Publishing Inc. Jr., K. K. (1984). The Hundredth Monkey. Camarillo, CA: DeVorss & Co. Jr., K. K. (n.d.). The Hundredth Monkey. Retrieved January 26, 2010, from
http://www.worldtrans.org/pos/monkey.html Keane, M. T., O’Brien, M., & Smyth, B. (2008). Are people biased in their use of
search engines? [Electronic version]. Communications of the ACM, 51(2), 49-52.
Kelley, J. (2007). Notes from Lotus Support. Retrieved May 2, 2008, from
http://www-10.lotus.com/ldd/nflsblog.nsf/dx/LC102_Install_Info Kirkpatrick, M. (2006). Tips on Choosing a Social Bookmarking Tool. Retrieved
April 9, 2008, from http://www.netsquared.org/choosingsocialbookmarking Klaisubun, P., Kajondecha, P., & Ishikawa, T. (2007). Behavior Patterns of
Information Discovery in Social Bookmarking Service [Electronic version]. WI '07: Proceedings of the IEEE/WIC/ACM International Conference on
94
Web Intelligence, 07, 784-787. Kuler. (2010). Create From a Color – Triad. Retrieved January 25, 2010, from
http://kuler.adobe.com/#create/fromacolor Kyrnin, J. (2006). Browser Safe Colors – yay or nay?. Retrieved January 25,
2010, from http://webdesign.about.com/b/2006/04/11/browser-safe-colors-yay-or-nay.htm
Kyrnin, J. (n.d.). How Long Should Your Web Page Be – People Do Scroll, But
How Far Will They Scroll?. Retrieved January 16, 2010, from http://webdesign.about.com/od/layout/qt/page_length.htm
Lawlor, J. (2000). Web Services Offer Solutions to Bookmark Overload.
Retrieved May 2, 2008, from http://query.nytimes.com/gst/fullpage.html?res=9B05E3DC1E38F930A25754C0A9669C8B63&sec=&spon=&partner=permalink&exprod=permalink
Leskovec, J., Adamic, L. A., & Huberman, B. A. (2007). The Dynamics of Viral
Marketing [Electronic version]. ACM Transactions on the Web (TWEB), 1(1), 1-39.
Liu, F., Yu, C., & Meng, W. (2002). Personalized web search by mapping user
queries to categories [Electronic version]. Conference on Information and Knowledge Management: Proceedings of the eleventh international conference on Information and knowledge management, 2002, 558-565.
LivLarge. (2009). New Zealand Social Marketing Survey Results. Retrieved
October 5, 2009, from http://www.livlarge.co.nz/2009/04/new-zealand-social-marketing-survey-results/
Madden, M., & Rainie, L. (2003). America’s online pursuits: The changing
picture of who’s online and what they do. Retrieved January 25, 2010, from http://www.pewinternet.org/pdfs/PIP_Online_Pursuits_Final.PDF.
Markus, S. (2008). Purpose Tagging: Capturing User Intent to Assist
Goal-Oriented Social Search [Electronic version]. Conference on Information and Knowledge Management: Proceeding of the 2008 ACM workshop on Search in social media, 08, 35-42.
Mathes, A. (2004). Folksonomies – Cooperative Classification and
Communication Through shared Metadata. Retrieved May 2, 2008, from http://www.adammathes.com/academic/computer-mediated-communication/folksonomies.html
Matsuo, Y., & Yamamoto, H. (2009). Community gravity: measuring
bidirectional effects by trust and rating on online social networks [Electronic version]. International World Wide Web Conference: Proceedings of the
95
18th international conference on World wide web, 09, 751-760. McNeil, P. (2008). The Web Designer's Idea Book : the ultimate guide to
themes, trends and styles in website design. Cincinnati, Ohio: HOW Books. Millen, D. R., & Feinberg, J. (2006). Using Social Tagging to Improve Social
Navigation [Electronic version]. Workshop on the Social Navigation and Community-Based Adaptation Technologies. Conjunction with Adaptive Hypermedia and Adaptive Web-Based Systems (AH’06). June 20th, 2006, Dublin, Ireland.
Millen, D. R., Feinberg, J., & Kerr, B. (2005). Social Bookmarking in the
Enterprise [Electronic version]. Queue, 3(9), 28-35. Millen, D. R., Feinberg, J., & Kerr, B. (2006). Dogear: Social Bookmarking in
the Enterprise [Electronic version]. Proc. CHI’06, 06, 111-120. Murch, G. M. (1985). Colour graphics—blessing or ballyhoo? [Electronic
version]. Human-computer interaction: a multidisciplinary approach, 333-341.
Nature Publishing Group. (2005). Social Bookmarking Tools (I) – A General
Review. D-Lib Magazine, 11(4). Retrieved April 12, 2008, from http://www.dlib.org/dlib/april05/hammond/04hammond.html
NetSquared. (2006). Social Web Survey Results. Retrieved April 3, 2008, from
http://www.netsquared.org/social-web-survey-results Nie, M. D. (1999, September 17). itList. The Scout Report, 6(18). Retrieved
May 2, 2008, from http://scout.wisc.edu/Reports/ScoutReport/1999/scout-990917.html
Nielsen, J. (2000). Designing Web Usability: The Practice of Simplicity.
Indianapolis, Ind.: New Riders Publishing. Norman, D. (1990). The Design of Everyday Things. New York: Doubleday. PCPitstop. (2008, August 01). Social Bookmarking Survey Results. Retrieved
October 5, 2009, from http://techtalk.pcpitstop.com/2008/08/01/social-bookmarking-survey-results/
Sano, D. (1996). Designing Large-Scale Web Sites: A Visual Design
Methodology. New York, NY: Wiley Computer Publishing, John Wiley & Sons, Inc..
Sawyer, P., & Schroeder, W. (2000). Report 4: Links that give off scent. In
Designing Information-Rich Web Sites. Bradford, MA: User Interface Engineering.
96
Scanlon, T., & Schroeder, W. (2000). Report 7: Designing graphics with a purpose. In Designing Information-Rich Web Sites. Bradford, MA: User Interface Engineering.
Schriver, K. A. (1997). Dynamics in Document Design. New York, NY: Wiley
Computer Publishing, John Wiley & Sons, Inc.. Sen, S., Lam, S. K. T., Rashid, A. M., Cosley, D., Frankowski, D., Osterhouse,
J., et al. (2006). Tagging, Communities, Vocabulary, Evolution [Electronic version]. CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, 06, 181-190.
Sklar, J. (2009). Principles of Web Design (4th ed.). Clifton Park, N.Y. : Delmar
Learning ; London : Cengage Learning [distributor]. Sontag, D. (2009, May 24). Why Use Social Bookmarking?. Retrieved June 3,
2009, from http://www.fewsports.com/why-use-social-bookmarking.html Sperber, D., & Wilson, D. (1986). Relevance: Communication and Cognition.
Oxford: Blackwell. Spool, J. M., Klee, M., & Schroeder, W. (2000). Report 3: Designing for scent.
In Designing Information-Rich Web Sites. Bradford, MA: User Interface Engineering.
Spool, J. M., Scanlon, T., Schroeder, W., Snyder, C., & DeAngelo, T. (1999).
Web Site Usability: A Designer’s Guide. San Francisco, CA: Morgan Kaufmann Publishers, Inc..
Stanford, D. (2007). Criteria for Evaluating Social Bookmarking Tools.
Retrieved April 9, 2008, from http://www.iddblog.org/?p=5 Subramani, M. R., & Rajagopalan, B. (2003). Knowledge-sharing and influence
in online social networks via viral marketing [Electronic version]. Communications of the ACM, 46(12), 300-307.
Thebeatles. (2010). Songs: A-Z. Retrieved February 27, 2010, from
http://www.thebeatles.com/#/songs TVB. (2007). Multiplatform Glossary. Retrieved April 11, 2008, from
http://www.tvb.org/multiplatform/multiplatform_glossary.aspx Underwood, L. (2001). Page Content: The Long and the Short of It. Retrieved
May 21, 2010, from http://www.wdvl.com/Internet/Content/ Wann, M. (2008, December 4). Tag Cloud Module – Innovative Idea for Your
E-business. Retrieved December 29, 2009, from http://www.articlesbase.com/ecommerce-articles/tag-cloud-module-innovative-idea-for-your-ebusiness-670855.html
97
Watrall, E., & Siarto, J. (2009). Head First Web Design. Beijing; Cambridge: O'Reilly.
Whitworth, B., Banuls, V., Sylla, C., & Mahinda, E. (2008). Expanding the
Criteria for Evaluating Socio-Technical Software [Electronic version]. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 38(4), 777-790.
Yanbe, Y., Jatowt, A., Nakamura, S., & Tanaka, K. (2007). Can Social
Bookmarking Enhance Search in the Web? [Electronic version]. JCDL '07: Proceedings of the 7th ACM/IEEE joint conference on Digital libraries, 07, 107-116.
Ziegler, C., & Golbeck, J. (2007). Investigating interactions of trust and interest
similarity [Electronic version]. Decision Support Systems, 43(2), 460-475.
98
App
endi
x A
: Com
pari
ng S
ocia
l Boo
kmar
k Bu
tton
s
99
100
101
App
endi
x B:
Diff
eren
t Ver
sion
s of
Cog
niti
ve E
ffor
t and
Soc
ial F
eedb
ack
Col
lect
ed b
y 16
th J
uly
2009
fro
m b
bc.c
o.uk
and
mas
sey.
ac.n
z
102
Col
lect
ed b
y 16
th J
uly
2009
fro
m b
bc.c
o.uk
and
mas
sey.
ac.n
z
103
App
endi
x C:
Blig
g Bu
tton
s In
terf
ace
Scre
ensh
ots
– Fi
rst S
tudy
104
105
106
App
endi
x D
: The
Bea
tles
Min
i Web
site
Inte
rfac
e Sc
reen
shot
s
107
108
Appendix E: Survey Letter
109
App
endi
x F:
Tim
etab
le
110
Appendix G: Consent Form
111
Appendix H: First Study Questionnaire
112
113
114
115
App
endi
x I:
Proc
edur
e N
avig
atio
n Pa
ge
116
App
endi
x J:
Ope
n-En
ded
Que
stio
ns R
espo
nse
– Fi
rst S
tudy
117
118
Note:
Hig
h: H
i; Lo
w: L
o; C
ogni
tive
Effo
rt: E
F; S
ocia
l Fee
dbac
k: F
B
119
App
endi
x K:
Upd
ated
Blig
g Bu
tton
Scr
eens
hots
– F
irst
Pilo
t Stu
dy
120
Appendix L: Protocol Analysis Results– First Pilot Study
121
App
endi
x M
: Pro
toco
l Ana
lysi
s Re
sult
s– S
econ
d Pi
lot S
tudy
122
Appendix N: Cognitive Walkthrough– Second Pilot Study
123
124
125
126
127
128
129
130
131
App
endi
x O
: Blig
g Bu
tton
Inte
rfac
e Sc
reen
shot
s –
Seco
nd S
tudy
No
Soci
al F
eedb
ack
Son
g In
form
atio
n S
ocia
l Fee
dba
ck
Son
g In
form
atio
n S
ocia
l Fee
dba
ck
Plu
s B
ligg
Num
ber
Soci
al F
eed
back
Son
g In
form
atio
n S
ocia
l Fee
dba
ck
Bli
gg N
um
ber
Soci
al F
eedb
ack
Plu
s Ch
oice
of S
ocia
l Fee
dbac
k
132
Appendix P: Second Study Questionnaire
133
134
135
App
endi
x Q
: Ope
n-En
ded
Que
stio
ns R
espo
nse
– Se
cond
Stu
dy
136
137
Statement from the Nature of the Massey University Ethics Committee
The experiments were awarded by Sylvia V Rumball (Professor) Chair, Human
Ethics Chairs’ Committee and Assistant to the Vice-Chancellor (Research
Ethics) on 27th October 2009.