investigating virtual social networking in the context of

DEFENCE DÉFENSE&

Defence R&D Canada – Atlantic

Copy No. _____

Defence Research andDevelopment Canada

Recherche et développementpour la défense Canada

Investigating Virtual Social Networking in the

Context of Military Interoperability

Year 3 Report and Overall Three Year Summary

Jacquelyn CrebolderTania RandallAren HunterDRDC Atlantic

Curtis CoatesGerard TorenvlietCMC Esterline|Electronics

Technical Report

DRDC Atlantic TR 2012-255

August 2013

This page intentionally left blank.

Investigating Virtual Social Networking in the Context of Military Interoperability Year 3 Report and Overall Three Year Summary

Jacquelyn Crebolder Tania Randall Aren Hunter DRDC Atlantic Curtis Coates Gerard Torenvliet CMC Esterline|Electronics

Defence R&D Canada – Atlantic

Technical Report


August 2013

Principal Author

Original signed by Jacquelyn Crebolder

Jacquelyn Crebolder

Defence Scientist

Approved by

Original signed by Francine Desharnais

Francine Desharnais

Head Maritime Decision Support Section

Approved for release by

Original signed by Calvin Hyatt

Leon Cheng

Chair DRP

In conducting the research described in this report, the investigators adhered to the policies and procedures

set out in the Tri-Council Policy Statement: Ethical conduct for research involving humans, National

Council on Ethics in Human Research, Ottawa, 1998 as issued jointly by the Canadian Institutes of Health

Research, the Natural Sciences and Engineering Research Council of Canada and the Social Sciences and

Humanities Research Council of Canada.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2013

© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale,

2013

DRDC Atlantic TR 2012-255 i

Abstract ……..

Over a three-year research program, Defence Research and Development Canada – Atlantic

investigated social networking technology with the aim of understanding this relatively new

collaboration tool. The technology was assessed for potential benefits and issues within the

context of information sharing, finding expertise, and team building for the Canadian Forces

(CF). A requirements analysis was conducted on a number of military groups in Year 1, and an

in-depth analysis was administered to a select group, the Training Development Officers (TDO),

in Year 2. In Year 3 the TDO community was provided with an enterprise-level social networking

platform (SABA People Cloud) in order to examine how the social networking technology was

used, what features were important, and what issues, if any, existed. A number of data collection

methods were employed including, questionnaire, experimentation, workshop, observation, and

visual analytics. The results were compared to similar data collected from the publicly available

Facebook social networking site (Facebook.com), where the TDOs had created their own Group

and where they had become increasingly active. This report summarizes Years 1 and 2 of the

program, and describes in detail the final year, Year 3. The work overall emphasizes the

importance of ease of use and accessibility of social networking tools and suggests that this

technology can be a viable and useful support to the CF.

Résumé ….....

Par un programme de recherche de trois ans, Recherche et développement pour la défense Canada

– Atlantique a étudié les nouvelles technologies que constituent les réseaux sociaux afin de mieux

comprendre ces nouveaux outils de collaboration. Nous avons évalué les avantages et

inconvénients potentiels du réseautage social dans le contexte du partage d’information, du

repérage d’experts et la promotion du travail d’équipe au sein des Forces canadiennes (FC). La

première année, nous avons effectué une analyse des besoins de plusieurs groupes de militaires et,

l’année suivante, une analyse approfondie a été faite sur l’un de ces groupes : les officiers du

développement de l’instruction (ODI). La troisième année, la collectivité des CDI a pu utiliser

une plateforme de réseautage social d’entreprise (SABA People Cloud); et nous avons pu

observer comment cette technologie a été utilisée, quelles fonctions ont été les plus importantes et

le cas échéant quels problèmes se sont manifestés. Nous avons pour cela fait appel à plusieurs

méthodes de collecte de données, notamment des questionnaires, des expériences, des ateliers et

des observations ainsi que l’analytique visuelle. Nous avons comparé les résultats obtenus à des

données semblables recueillies sur le site public de réseautage social Facebook (facebook.com),

car les CDI y avaient créé leur propre groupe et y étaient de plus en plus actifs. Le rapport résume

les deux premières années du programme d’étude, et décrit la troisième en détail. Ces travaux

soulignent en règle générale l’importance de la convivialité et de l’accessibilité des outils de

réseautage social, et appuient l’idée que cette technologie peut servir d’appui utile aux activités

des FC.

ii DRDC Atlantic TR 2012-255


DRDC Atlantic TR 2012-255 iii

Executive summary

Investigating Virtual Social Networking in the Context of Military Interoperability

Jacquelyn Crebolder; Tania Randall; Aren Hunter; Curtis Coates; Gerard Torenvliet; DRDC Atlantic TR 2012-255; Defence R&D Canada – Atlantic; August 2013.

Introduction: With the rapid adoption and popularity of social networking sites many

organizations are replacing traditional methods of communicating with enterprise-level social

networking platforms. Defence Research and Development Canada (DRDC) examined the use of

this kind of technology for the Canadian Forces (CF), with a focus on how social networking

tools might support finding expertise and sharing information amongst groups, particularly those

whose members are not working face-to-face. This report deals in detail with the final Year 3 of

the research program with a summary of Years 1 and 2 included1.

Through a requirements analysis conducted in Year 1, Training Development Officers (TDO)

were identified as a suitable community for more in-depth analysis of use and requirements for a

virtual social networking platform. TDOs are distributed across Canada and are responsible for

coordinating and advising on CF training and education. The community was provided with an

enterprise-level social networking platform called SABA People Cloud, and use of the platform

was analyzed. A comparison was made to a publicly available social networking website

(Facebook).

Results: Data were collected through interviews, surveys, on-line experimentation, and direct

observation of activity on the two social networking web-sites. Overall the number of data points

was small but nevertheless, results emphasized the importance of seamless deployment of the

platform, ease of use requiring little training, and rapid adoption based on a quickly recognized

improvement in workflow by the user. The Facebook platform met all these goals, thereby

progressively boosting activity on the site and resulting in a vibrant and active on-line community

where people could meet, converse, and share. The SABA site on the other hand struggled for a

number of reasons, most of them technical, and after a short period of use eventually went

unused.

Significance: These results show that virtual social networking can be viable and useful within

the CF for members who are distributed and also for those in close proximity to one another. The

most important considerations are that:

The technology has to be easy to use and quick to learn, and it must complement and

integrate seamlessly with current work practices;

Virtual social networking is not so much about file storage and sharing as it is about

people meeting and interacting. As such, a social networking technology should have

1 Full reports of Years 1 and 2 can be found in Torenvliet, Euerby, Scott, & Histon, 2011, [DRDC Atlantic

CR 2010-308]; and Torenvliet & Leal, 2011, [DRDC Atlantic CR 2011-155], respectively.

iv DRDC Atlantic TR 2012-255

features that support on-line community-building and participation. Those features

include: individual profiles preferably with the capability to attach a self-portrait photo,

an easy to read and easy to navigate conversation zone, widespread broadcasting of posts

and comments to the entire community, and not least, a climate that is welcoming, active,

and easy going;

Support by upper management must be demonstrated by their use of the technology;

Full support and commitment from the information technology (IT) sector is required;

and

The deployment strategy must include ongoing interaction with the community,

highlighting benefits of the platform and promoting adoption.

Incorporating this kind of tool into the right military groups could significantly change the way of

doing business, increasing community amongst members not only within the military but

extending to include other organizations.

Future plans: This study examined a community where the workday tempo is relatively steady.

Future research should investigate the feasibility of this kind of technology within a fast-paced

operational environment where other variables may impede use of the tool or where other uses

might be discovered.

DRDC Atlantic TR 2012-255 v

Sommaire .....

Investigating Virtual Social Networking in the Context of Military Interoperability

Jacquelyn Crebolder; Tania Randall; Aren Hunter; Curtis Coates; Gerard Torenvliet; DRDC Atlantic TR 2012-255; R & D pour la défense Canada – Atlantique; août 2013.

Introduction : Vu l’adoption rapide et la popularité des sites de réseautage social, nombre

d’organisations remplacent les méthodes de communication habituelles par des plates-formes de

réseautage social d’entreprise. Recherche et développement pour la défense Canada (RDDC) a

étudié l’utilité de ce genre de technologies pour les Forces canadiennes (FC); plus précisément,

comment les outils de réseautage social peuvent appuyer le repérage d’experts et l’échange

d’information dans un groupe, particulièrement ceux dont les membres ne travaillent pas

ensemble en personne. Le rapport traite de la troisième et dernière année du programme de

recherche, mais il comprend un résumé des deux premières années2.

Une analyse des besoins effectuée la première année a révélé que les officiers de développement

de l’instruction (ODI) seraient une collectivité se prêtant bien à une analyse plus poussée de

l’utilisation et des besoins entourant une plate-forme de réseautage social. Les ODI se trouvent

aux quatre coins du pays; ils coordonnent l’instruction et l’éducation des FC et conseillent à ce

sujet. Nous avons donné aux ODI l’accès à une plate-forme de réseautage social d’entreprise

nommée SABA People Cloud, puis nous avons analysé comment ils l’ont utilisé. Nous avons

aussi comparé cette utilisation à celle d’un site Web de réseautage social grand public,

c’est-à-dire Facebook.

Résultats : Nous avons colligé les données à l’aide d’entrevues, de sondages, d’expériences en

ligne et d’observations directes de l’activité sur SABA et Facebook. Les données recueillies,

quoique limitées en nombre, ont tout de même indiqué clairement l’importance d’un déploiement

transparent de la plate-forme, d’une grande convivialité (ce qui réduit ainsi les besoins en

instruction) et d’une adoption rapide découlant d’avantages clairement perçus par l’utilisateur.

Facebook respecte tous ces critères; par conséquent, l’activité sur ce site a graduellement

augmenté jusqu’à donner une communauté virtuelle dynamique et active où les gens pouvaient se

rencontrer, dialoguer et partager. SABA, par contre, a éprouvé des difficultés pour plusieurs

raisons (souvent d’ordre technique), et après une courte période d’activité, ce site a

essentiellement été ignoré.

Importance : Cette étude démontre que le réseautage social virtuel peut servir d’appui utile au

sein des FC, particulièrement pour les membres géographiquement éloignés, mais aussi pour ceux

qui travaillent ensemble. Voici les grandes conclusions qui se dégagent de cette étude :

2 Vous trouverez le rapport complet des première et deuxième années respectivement dans Torenvliet,

Euerby, Scott et Histon, 2010, [DRDC Atlantic CR 2010-308] et Torenvliet et Leal, 2011, [DRDC Atlantic

CR 2011-155].

vi DRDC Atlantic TR 2012-255

la technologie adoptée doit être facile à apprendre et à utiliser, et elle doit s’intégrer de

façon transparente aux pratiques de travail actuelles;

plus que le stockage et le partage de documents, le réseautage social est surtout axé sur la

rencontre et l’interaction. Par conséquent, toute technologie de réseautage social doit

favoriser la création de communautés virtuelles et la participation à celles-ci. Parmi ces

caractéristiques, mentionnons un profil personnel auquel on devrait pouvoir joindre une

photo personnelle, un espace de conversation de lecture facile et où la navigation est

simple, une large diffusion des messages et commentaires à l’ensemble de la

communauté, et, la dernière mais non la moindre, un environnement accueillant, actif et

décontracté;

les cadres supérieurs doivent, par leur participation, démontrer qu’ils appuient ces

technologies;

il est vital d’obtenir l’appui et l’engagement ferme des responsables de la TI; and

la stratégie de déploiement doit impérativement comprendre une interaction continuelle

avec les utilisateurs afin de souligner les avantages de la plate-forme et en encourager

l’adoption.

Intégrer ce type d’outils aux groupes militaires les plus susceptibles d’en profiter peut

révolutionner leurs pratiques et renforcer la cohésion des membres, non seulement au sein des

Forces, mais aussi dans d’autres organisations.

Perspectives : L’étude a porté sur une communauté dont le rythme de travail est assez régulier. Il

serait bon, dans les études suivantes, d’évaluer le cas échéant la faisabilité de ces technologies

dans un environnement opérationnel fébrile; car d’autres variables risquent d’empêcher

l’utilisation de ces outils, ou encore, pourraient s’y dégager des applications insoupçonnées.

DRDC Atlantic TR 2012-255 vii

Table of contents

Abstract …….. ................................................................................................................................. i

Résumé …..... ................................................................................................................................... i

Executive summary ........................................................................................................................ iii

Sommaire ..... ................................................................................................................................... v

Table of contents ........................................................................................................................... vii

List of figures ................................................................................................................................. ix

List of tables .................................................................................................................................... x

Acknowledgements ........................................................................................................................ xi

1 Introduction ............................................................................................................................... 1

1.1 Background.................................................................................................................... 1

1.2 Research Overview and Objectives ............................................................................... 1

2 Research Program ..................................................................................................................... 5

2.1 Year 1 (December 2009 – March 2010) ........................................................................ 5

2.1.1 Requirements Analysis.................................................................................... 5

2.1.2 Technology Watch .......................................................................................... 6

2.2 Year 2 (April 2010 – March 2011) ................................................................................ 7

2.2.1 In-depth Requirement Analysis ....................................................................... 7

2.2.2 Interdepartmental workshop ............................................................................ 8

2.3 Year 3 (April 2011 – March 2012) .............................................................................. 10

2.3.1 Selection and tailoring of a social networking platform ............................... 10

2.3.2 Installation and Deployment ......................................................................... 11

2.3.3 Pre-SABA People Cloud Deployment Survey .............................................. 12

2.3.3.1 Pre-SABA People Cloud Survey Results ................................... 12

2.3.4 An alternative to an enterprise-level social networking site ......................... 13

2.3.5 Facebook and SABA People Cloud deployment experience ........................ 13

2.3.5.1 SABA People Cloud ................................................................... 13

2.3.5.2 Facebook TDO Group ................................................................ 14

2.3.5.3 Summary of comparison of deployment strategies .................... 15

2.3.6 Data collection .............................................................................................. 15

2.3.6.1 Post-SABA People Cloud Deployment Survey .......................... 16

2.3.7 Symposium .................................................................................................... 23

2.3.7.1 Structured Interview ................................................................... 23

2.3.7.2 Experiment ................................................................................. 25

2.3.8 Social networking platform user data analysis .............................................. 28

2.3.8.1 Data analysis – visualization using HanDles .............................. 29

2.3.8.2 Data analysis – Observation ....................................................... 32

2.3.8.3 Results ........................................................................................ 33

viii DRDC Atlantic TR 2012-255

3 Discussion ............................................................................................................................... 39

4 References ..... ......................................................................................................................... 41

Annex A .. Technology Steward Introduction Letter ..................................................................... 43

Annex B ... Community Introduction Email ................................................................................... 45

Annex C ... Post-SABA Survey ...................................................................................................... 47

Annex D .. TDO Symposium Interview......................................................................................... 59

Annex E ... SABA Platform “How-To” ......................................................................................... 69

Annex F ... Facebook data extraction and preparation ................................................................... 77

List of symbols/abbreviations/acronyms/initialisms ..................................................................... 84

DRDC Atlantic TR 2012-255 ix

List of figures

Figure 1: Visualization of the subsets of the TDO community referred to throughout Table 6. ... 23

Figure 2: A screen shot of HanDles in response to the query ‘George Clooney’.......................... 30

Figure 3: HanDles column titles. ................................................................................................... 31

Figure 4: Visual of 5 months of activity on the SABA site for the top generated handles.

Note initial and final activity rates. ............................................................................. 34

Figure 5: Visual of 4 years of activity on the Facebook TDO Group page for the top

generated handles. Note the increasing activity in July 2011 as the experience with

SABA began. ............................................................................................................... 34

Figure 6: Number of 'likes' over a 2 year period of activity on the Facebook TDO Group

page. ............................................................................................................................ 35

Figure 7: Basic post output excerpt. .............................................................................................. 78

Figure 8: Microsoft Office Excel macro........................................................................................ 80

Figure 9: Organization of Facebook data. ..................................................................................... 81

Figure 10: Separate HanDLes files. ............................................................................................... 81

Figure 11: Separate HanDles documents. ...................................................................................... 82

Figure 12: HanDles comments linked together. ............................................................................ 83

x DRDC Atlantic TR 2012-255

List of tables

Table 1: Pre-SABA survey results – overall average score of familiarity with other TDOs......... 13

Table 2: Deployment strategy. ...................................................................................................... 15

Table 3: Summary of post-SABA survey results. ......................................................................... 17

Table 4: Summary of responses to open-ended post-SABA survey questions.............................. 20

Table 5: Post-SABA network analysis – overall average score of familiarity with other TDOs. . 21

Table 6: Comparison between pre- and post-SABA network analysis – familiarity with other

TDOs ........................................................................................................................... 22

Table 7: Description of headings. .................................................................................................. 78

DRDC Atlantic TR 2012-255 xi

Acknowledgements

The authors would like to acknowledge the assistance of the Training Development Officer

community from within the Canadian Forces. Many thanks to all those who participated in this

research project.

xii DRDC Atlantic TR 2012-255


DRDC Atlantic TR 2012-255 1

1 Introduction

1.1 Background

In the first few years of the 21st century a new class of internet-enabled software applications has

gained prominence. These applications are built on familiar models from email and informational

websites, so they are accessible and have been adopted quickly. Even though they seemed

familiar, there was an important difference from what had come before. These new applications

allow large networks of people (even the entire user population of the internet) to contribute to

and participate in conversations about the information that is posted. Blogs, for example, have

become a major force in news dissemination and commentary, not just because they allow experts

(or, would-be experts) to publish their thoughts, but because they allow anyone to express

agreement with, or improve, or detract from the original, by means of adjoining comments.

At the time of writing, the internet has changed dramatically from what it was just a few years

ago. News websites, while still built on a backbone of news stories, now solicit and publish user

comments, so that the conversations about stories is a source of important additional meaning.

Fully co-creatable websites called Wikis have emerged and entered the mainstream, and the

volunteer-powered Wikipedia (an on-line encyclopedia) is the web’s premier reference source.

Email, the application that powered the emergence of the internet into the mainstream, is at risk

of being eclipsed in importance and usage by social networking sites like Facebook, Twitter, and

LinkedIn. This change in the internet is not just a change in the applications available, but a

fundamental change in their nature and how they can be used. These new applications harness the

power of network effects (O'Reilly, 2005) to make the information and functions they provide

more valuable as more people use them (McAfee, 2009). Moreover, most of these new websites

do not typically have a static imposed structure, but rather use an aggregation of individual-level

input (called a Folksonomy) to allow categories, importance, and expertise to emerge

dynamically. More than ever before, the changes in the internet allow groups of people that are

networked together to act as a collective, easily accessing and working from thoughts developed

by others (Bingham & Conner, 2010).

This changed internet is often called Web 2.0, and the social network applications are often called

social media. While both of these terms are accurate, they are easily misinterpreted. Web 2.0

conveys the sense that the revolution has been all about the public internet, and social media

conveys the sense that somehow this is about socializing, and not real work. However, large

organizations have applied Web 2.0 and social media with significant benefits. Instead of these

terms, McAfee (2009) favours the noun Enterprise 2.0 and the adjective collaborative. These

more formal terms do a better job of conveying that the revolution is also relevant to large,

modern organizations and that it is all about bringing people together for better work outcomes.

1.2 Research Overview and Objectives

Defence Research and Development Canada (DRDC) realized that these new collaborative tools

have strong potential to support the work and operations of the Canadian Forces (CF). There are

potential benefits in at least the following three areas:

2 DRDC Atlantic TR 2012-255

– Networking with familiar and unfamiliar contacts. Like any large and diverse

organization, the CF is composed of groups, where members are strongly connected to one

another (for example, soldiers within a specific unit). In the social networking literature,

these close connections are sometimes referred to as strong ties (Granovetter, 1973).

Individuals in specific groups can also be less closely linked to individuals in other

communities (for example, members who have trained with, served with on foreign

deployment, or worked with on a recent posting). These more distant links, or weak ties,

include connections less frequently in touch, or who may never connect at all, or who, in

fact, may be people an individual is not aware of directly. Acquaintances, friends of friends,

non-co-located colleagues, and other, more distant connections, are examples of

connections that fall into the category of weak ties. While it is relatively easy to stay in

touch with people within a current group, especially if in contact face to face, a virtual

social networking tool for the CF could help members maintain contact with those weaker

ties found in other groups and with members who are not co-located. Virtual social

networking can also be used to tap into network membership (making weak ties stronger)

and to make use of networks to collaborate with members they do not already know

(making weak ties where no ties previously existed). Since weak ties are a critical conduit

for new ideas to spread through an organization (Granovetter, 1973; Hansen, 1999) the CF

stands to benefit significantly if a tool can be found to help CF members form and use these

ties.

– Sharing information. McAfee (2009) reports that it is typical for employees in large

organizations to experience difficulty in finding information on their corporate intranet.

Feedback from CF members indicates that the CF is no different in this regard (Torenvliet

& Leal, 2011; Torenvliet, Euerby, Scott, & Histon, 2011). Virtual social networking can

help in sharing of information in three ways. First, as people’s networks of contacts grow,

the size of the group of people to which specific questions can be posed and shared

increases. On publicly available social networking sites, like Twitter and Facebook for

example, people can use their status updates and tweets to broach questions to their network

or to share insights. This idea shows promise to allow CF members to share information.

Second, as CF members view content on the intranet, it could be possible for them to tag it

with a self-defined category, rate it, or add a bookmark to the content. These tags, ratings,

and bookmarks are ways for records of meaning (from a consumer’s point of view, rather

than the author’s) to be developed, and they allow others to be informed about searches for

information. Third, it may be possible for members to post or store information on their

profile (in a file store, or on a blog) from where it can be shared with others.

– Locating sources of expertise. CF members have increasingly diverse sets of expertise

commensurate with the increasing diversity of operations the CF has participated in; the

increasing diversity and complexity of technology employed; and, the increased

opportunities for participating in exchanges with ABCA3 countries. As a result, a typical

CF member’s expertise is far richer than recorded on a service record. Because the actual

dimensions of members’ expertise are not recorded, the way to find experts is typically

through word of mouth. Word of mouth strictly limits the breadth of dissemination.

Compounding this problem is the limited availability of experts since most often, experts in

a given area are already tasked to important projects. The result is that the best expert that

can be spared, rather than the best expert possible, is obtainable. Virtual social networking

3 America, Britain, Canada, and Australia.


tools show promise to help with both of these problems. For example, by providing CF

members with a profile on an enterprise-level social networking platform, individuals could

record and market their expertise using their profile, thereby advancing the data on their

service record. Even for members who do not take the time to record their expertise, other

artefacts related to their profile (e.g., bookmarks, stored files, etc.) would be available

indicators of their interests and expertise, that would be revealed in expertise searches.

These benefits are all related to military interoperability and, if realized, they could increase the

CF’s capability in general, particularly in distributed team, joint and coalition force, networked

environments where CF members could benefit from the ability to develop and explore

professional connection networks with other CF members.

The objective of DRDC’s research programme was to provide an enterprise-level social

networking tool to be used for collaboration and communication in the military domain. The

research aimed to understand the human performance aspects of social networking technologies.

The work focused on determining appropriate individuals and groups who were already using

social media in their work, or to whom social media might be useful, and to gain an

understanding of the benefits that could be realized, and the challenges encountered using this

kind of technology. While the research dealt with a new class of technologies, its focus was not

on the technologies themselves, but on the user’s interaction with the tools provided.

The research programme consisted of three phases realized over three years. Following are

sections providing research objectives, methods, and findings for each year, although Year 3 is

addressed in more detail than Years 1 and 2. Separate, comprehensive reports for Years 1 and 2

can be found in Torenvliet, Euerby, Scott, & Histon, 2011; and Torenvliet & Leal, 2011,

respectively. A discussion of the collective findings closes the report.


2 Research Program

2.1 Year 1 (December 2009 – March 2010)

2.1.1 Requirements Analysis

In the first year of the research programme a broad requirements analysis was conducted to gain

an understanding of the tasks and the collaborative work conducted by groups within the CF and

the potential for work to be supported by a virtual social networking platform. The research

involved interviewing 8 military communities to discuss their roles and tasks and the problems

that they might be facing in collaboration with other groups or team members who are not co-

located. The interviewed individuals belonged to the following communities: Regional Joint

Operations Center (RJOC); Land Forces Atlantic Area (LFAA), Web Development; Land Forces

Atlantic Area (LFAA), Operations; CF Naval Operations School (CFNOS), Training

Development; Her Majesty's Canadian Ship (HMCS) Trinity; Halifax & Region Military Families

Resource Center (HRMFRC); CF Psychological Operations (PsyOps).

In summary, the analysis revealed a broad requirement for social networking tools across the

groups interviewed, focusing primarily on a need to find and share expertise. Information sharing

was a secondary focus, but it was tightly linked to expertise sharing. On the other hand, finding

and making use of weak links, in other words, social networking as such, is not a current part of

formal CF practices. Other findings from the analysis included the observation that individuals

will only show interest in adopting a social networking system if it does not involve extra work,

and new recruits and those of the future expect these kinds of tools in the workplace.

Through the process of conducting the requirements analysis, three of the interviewed

communities were identified as showing the largest number of requirements for social

networking. They were: RJOC, CFNOS Training Development, and HMCS Trinity. Of these the

CFNOS Training Development Officers (TDO) were selected as the military group best suited for

the research and more in-depth analysis. Further research would also include providing the group

with a web-based social networking platform and observing network development and use of the

platform. The original plan was to work with the TDOs and then extend the work to the RJOC

and HMCS Trinity. However, time became limited as the project progressed, and the work was

restricted to analysing and observing one group, the TDOs.

There were a number of reasons why this group was chosen. First, the community is distributed

and members are often required to seek out individuals with specific expertise, meaning that a

virtual social networking platform might serve them well for connecting and collaborating, and

for making, finding, and maintaining contacts. Second, the group is large enough to support the

development and growth of a virtual social network. Small groups do not usually have the need to

reach out and build a social network. Third, TDOs do not typically work in an operational

environment that is frequently changing and high-paced. The tempo of their workday is relatively

steady and risk-free and is therefore conducive to observation and data collection.

Further to this, the TDOs have a vested interest in social media since these technologies can

provide means for supporting their tasks, such as delivering instruction and training, and


conducting writing boards. The community had unsuccessfully attempted in the past to establish

virtual social networking through other means (e.g., GC Connect, a government of Canada

on-line community). The group was interested in learning about why previous attempts had

failed, and they were hopeful that the enterprise-level platform to be provided to them through the

DRDC research project would be successfully embraced and utilized by the community.

Approximately 150 Training Development Officers serve in the CF across Canada. Their role is

to promote, guide, and coordinate training and education in the CF. To that end, TDOs identify

needs and requirements as well as solutions to support and enhance job performance. One of their

primary tasks is developing courses and conducting working groups and writing boards for

qualification standards (QSWB) and training plans (TPWB), for which they frequently need to

seek out expertise and information from within and beyond the CF.

2.1.2 Technology Watch

Part of the first year’s work included a technology watch to monitor developments in social

networking technologies, and particularly developments that might point to new and beneficial

ways for collaborating within the CF. The technology watch built on earlier work that evaluated

social networking technologies as of 2009 (Pronovost & Lai, 2009), and, apart from gathering

knowledge about progressive technologies, the technology watch was used to inform the choice

of candidate enterprise-level social networking platforms that could be used as a test platform in

the next two years of the project.

This activity included use of a closed-access team social network developed in the freely

available tool, Ning. The Ning site was used by the research team, who were distributed at

various locations across Canada, and the site proved to be a useful way for the team to gain direct

experience in sharing and collaborating on-line using blogs and discussion forums. In addition to

the Ning site, the team gained exposure and experience with as many Enterprise 2.0 technologies

as possible. A key collaboration environment was the IBM4 collaboration toolkit. Lotus

Sametime, Quickr, and Connections, was hosted at DRDC Atlantic and available to a number of

DRDC Atlantic staff including members of the research team. The intention was to obtain

hands-on experience using the numerous social media features on this platform, to observe how

different components and features of the toolset were used, and to monitor the development of

individuals’ social networks in this virtual environment.

To expand a social network naturally, it must be free of boundaries or restrictions in connecting

with others. Social networking is not typically restricted to a small segment but, rather, is able to

reach out and grow naturally. It is by this process that the network builds its character of strong

and weak links. Thus, our intention was to make the IBM platform available to all DRDC offices

across the country, and, as importantly, to other individuals and groups, such as military, industry,

and academic collaborators. It is only by opening up access to the IBM site that the real

development of virtual social networks could be observed accurately and understood.

Unfortunately, security concerns from DRDC IT prevented this from happening and hands-on use

of the IBM platform was limited to a relatively small pool within the walls of DRDC Atlantic.

4 International Business Machines Corporation.


Other commercially available platforms were also trialed or investigated, such as Mango,

Yammer, LinkedIn, Facebook, Igloo, Jive, Microsoft Sharepoint 2010 as well as the option to

design a platform from scratch. With respect to the latter option, while developing a grassroots,

useable social networking test platform for a CF test group might be a viable solution, upon

further investigation the research team decided that the time required to execute this endeavour

exceeded a reasonable timeframe within the project. As a result, we decided to continue to

evaluate the marketplace for commercially available tools that would be suitable for use as a test

platform. The evaluation and research on web-based collaborative tools conducted through

Year 1 highlighted several points about introducing new technology into the workplace.

First the technology must be easily integrated into an individual’s work flow;

Second, it must be easy to use and easy to adopt, not only from an individual’s

perspective but also from an organizational, information technology and security basis;

and

Thirdly, developing a deployment strategy that includes ongoing interaction with the test

community, and that highlights the benefits of the test platform and promotes adoption, is

essential.

The Technology Watch was initiated in Year 1 of the project and continued over Years 2 and 3.

For a full report on Year 1 see Torenvliet, Euerby, Scott, and Histon, 2011.

2.2 Year 2 (April 2010 – March 2011)

2.2.1 In-depth Requirement Analysis

Further to the findings from the technology watch, a particular technology can only be selected in

response to understanding the human requirements that need to be fulfilled and supported by that

technology. Consequently, following the general requirements activity and collaborative tool

evaluation of Year 1, a set of detailed requirements were developed for the TDO test community

in order to ensure they would be provided with appropriate work-supporting features on an

enterprise-level social networking platform. Data to define this activity were collected via an

on-line survey distributed to all TDOs. The objective of the questionnaire was to gather a picture

of the community and its members with respect to demographics and their use and familiarity

with social media tools. The researchers were also interested in gathering knowledge as to the

tasks that TDOs are required to perform and the specific individual and group need that might be

supported by a social networking platform. Of particular interest was how the TDOs structure

their work with respect to the three main research areas of interest: finding expertise, information

sharing, and building inter-personal networks.

The response rate to the questionnaire was approximately 50%. Responses showed that most of

the TDOs were officers with many years served in the CF, although typically less than five as a

TDO. Many, but not all, of the respondents indicated that they were familiar with social media

technologies in their personal lives, and with social networking websites, like Linkedin,

Facebook, and Twitter. Messaging (email) and viewing and posting text and photos to social

networking sites were the activities performed most often. The primary way for TDOs to

communicate at work was via email, and about 25% used a Blackberry.


While TDOs have a number of specialized work processes, much of their work fits into categories

that are generic to all knowledge work. This suggests that social networking tools proven

successful in other work domains are likely to fit well with the TDO community.

Management and providing guidance is an especially important task, and seems to take up a lot of

this community’s time. Social networking tools would give workers more direct access to

information and individuals without needing to work through their managers. A social computing

tool could be made more useful to many TDOs if it included workflow support for conducting

Qualification Standard Writing Boards (QSWB) and Training Plan Writing Boards (TPWB).

A social networking platform could also support mentoring, sharing of lessons learned, and

sharing of experiences. In the CF where individuals are regularly posted from/to a position this

kind of support could be really beneficial in the transfer of responsibility, which is typically done

through sharing of documents, contact lists, and other information shared in one or a series of

meetings at the time of transfer. Once the transfer has occurred, there is little on-going support

from the person transferring out to the person transferring in. As far as we were aware, at the time

of writing, other than face-to-face contact, TDOs had no way to share their informal experiences

and lessons learned. This is an especially challenging problem for many TDOs who work in

relative isolation. Social computing tools provide ways to increase the documentation of the

context of work, providing a trail of information and context that other personnel can refer to.

Furthermore, social computing tools designed to encourage informal contributions of expertise

(blog posts, discussions, etc.) allow informal training materials to be developed as problems are

encountered, and not only when a transfer is imminent.

The survey responses reinforced that TDOs were a good choice as a test group. In short, the

composition of this community, the nature of their work, and the challenges they face, led the

research team to believe that the TDO community would gain quick and important benefits from

the use of social computing tools designed to support their work.

2.2.2 Interdepartmental workshop

Part of the research involved learning about what other groups had done, or were doing, as far as

supporting communities through the use and development of collaborative web-based social

networking environments. A number of examples existed within the Government of Canada

(GoC) itself and so it seemed a valuable pursuit to exploit this resource by bringing representative

together to share and discuss experiences.

The research team hosted an interdepartmental workshop in Ottawa, Ontario in November 2010,

where invited GoC groups met to share experiences such as problems, successes, and lessons

learned around the use and implementation of social networking toolsets and technologies for

large distributed communities. Attendees included:

Treasury Board of Canada – who provided detail on the Government of Canada

Wikipedia-like website, GCPedia, noting that the site has morphed into somewhat of a

social networking site filling a void for this kind of tool.

Library Archives Canada – provided a summary of pointers for implementing and using

social media in the workplace based on their current and past experiences.


Communication, Foreign Affairs – discussed their work on developing government

websites through listening to the community of users so that the sites are improved to

meet the needs of the reader and answer their questions effectively.

Canadian Army – gave an overview of the Army Collaborative Information Management

System (ACIMS) with respect to the stage and status of development and plans for the

future. The ACIMS includes capabilities that break down into four main areas, file

sharing, collaboration (Microsoft Office Sharepoint Server (MOSS)), an information

gateway, and knowledge creation (e.g., wiki).

Centre for Security Science (CSS) – discussed their work with the Royal Canadian

Mounted Police (RCMP) in developing a Major Event Security Framework that

facilitates collaborative planning across a number of different agencies and organizations.

Human Resources Skills and Development (HRSD) Canada – talked about how the

department is reaching out using social media tools, and is gathering together ‘bits’ of

information all over the internet into one concise user-friendly web place. Several pros

and cons for using social networking sites like Facebook as a social media tool in

government were provided.

Individual Training and Modernization, Canadian Defence Academy (CDA) – talked in

general about the use of social media tools and provided pointers for how to support users

throughout implementation of social media in the workplace.

DRDC Toronto – discussed the development of a Web 2.0 hardware and software

architecture forming the testbed, Collaborative Knowledge Management and Platform.

The package was being developed using Open Source software and it contained a wide

selection of collaborative and social media tools.

Formal presentations were followed by round-table discussion in the afternoon segment of the

program. The findings from the workshop supported the DRDC research team’s own

understanding of strategies, issues, and challenges in the use and implementation of collaborative

tools like social networking in the workplace. The knowledge gained was of significant value to

the research program as it moved toward applying and observing the use of these kinds of

technologies in military groups within the Canadian Forces.

Some of the common observations were:

Clear and well-articulated organizational reasons and goals are necessary when applying

social media in the workplace;

Community guidelines and management are required;

Social media is eroding hierarchical structure, particularly for middle management;

Organizational silos and stovepipes are barriers to social media;

Satisfied employees where social media is accessible versus attrition where it is not; and

Reverse mentoring is apparent.


2.3 Year 3 (April 2011 – March 2012)

2.3.1 Selection and tailoring of a social networking platform

Having identified detailed requirements the next step was selection of a social networking

platform. The plan was to develop a platform or take a commercially available platform and

modify it so that it fit with the specific requirements of the TDO group. The intention was to

conduct usability tests on the tailored test platform with a small group in order to validate the

design before full deployment to the TDO community. As stated earlier, it was decided early on

that the best approach would be to use a commercial platform rather than developing one from

scratch.

The survey of technologies over Years 1 and 2 helped inform the selection of a test platform and

provided an understanding of the ways in which some industries make use of social networking

tools. Based on the features of these platforms in relation to the TDO requirement and the

research objectives, and despite the outreach implementation issues during the test period within

DRDC, the research team selected the enterprise-level IBM social computing platform as the best

choice. The IBM Lotus toolset is a mature collaborative environment that provides a high quality

blend of productivity solutions that could support the TDOs’ workflow (e.g., document sharing)

with social networking features (e.g., member profiles, community sharing, the ability to search

for expertise).

The aim was to actually implement and deploy a technology so that the community could use it in

their day to day work. As such, it was not only essential that the platform meet the needs of the

test community and the research objective, but that it also was consistent with the restrictions and

requirements of security and information technology (IT) standards, as well as being generally

accepted by senior officials of the Canadian Defence Academy, which is the organization that

oversees the TDOs. Acquiring the various levels of approval took a considerable amount of time,

often following a pattern whereby agreement for using, for example, the IBM platform, was

obtained at the point of contact level but subsequently denied at a more senior management level.

Likely because the research team was not co-located with the CDA community, issues identified

by CDA were not always communicated, so it was difficult for the research team to understand

the delays and potentially address the problems. Such was the case with the IBM platform. After

initial verbal agreement with CDA on selection of the IBM toolset, final endorsement was slow in

coming and, coupled with setbacks in DRDC procurement (IBM called for revisions to legal

wording within the procurement documents) and at the DRDC information technology level, the

project schedule slipped. As it turned out the IBM suite of collaborative tools was eventually not

chosen by CDA senior management, and all in all, final selection of a platform took considerably

longer than originally estimated.

The process of platform selection in this situation where it was necessary that the user

organization be completely engaged was not helped by frequent replacements of the point of

contact within the organization. Indeed, over the entire course of the three year project, changes

in personnel and points of contact were an ongoing challenge. At one point DRDC rotated

through three CDA/TDO contact personnel in less than a month, in addition to a change in the

contractor’s lead human factors engineer to the project at about the same time. Postings are

routine in the CF but in this circumstance where numerous major and minor changes to the


research plan were necessary, the changeover in contact personnel resulted in fragile continuity in

the history behind those changes and an unclear understanding of the objective and mandate of

the project by the participating community.

In July 2011, a decision was reached by CDA approving a SABA® product, SABA People Cloud,

as the platform to be provided to the TDO community for the purpose of this research.

SABA People Cloud, still in beta version, was demonstrated via webinar to the DRDC research

team and judged to be a suitable environment that would meet the needs of the research

objectives. Features of the platform included individual profiles, a personal configurable

homepage, file sharing, the ability to ‘follow’ other members, RSS feed for notification of

updates, the ability to create groups, and to flag and tag documents and bookmarks, video sharing

in a feature called ‘channels’ as well as the functionality of real-time communication, and a web

meeting feature for holding on-line meetings and desktop sharing. All in all, the SABA People

Cloud was an excellent candidate with a robust set of Web 2.0 tools.

SABA People Cloud was selected by CDA for a number of reasons. The primary motivation was

that SABA is the backbone of the Royal Canadian Air Force Integrated Information Learning

Environment (AFILE), a project launched in 2008 as a means to standardize the accessibility and

management of training programs. AFILE is eventually to be available to the CF as a whole.

SABA People Cloud is an additional module to AFILE and, as such, CDA and DND in general,

were interested in understanding the value of adding the social aspect to the existing AFILE

environment. Furthermore, the CDA technical team who would be responsible for implementing

SABA People Cloud were already familiar with the underlying technology. Branding and

configuration of the platform was performed by CDA, who also performed the duties of system

administrators.

An important consideration to the TDO community and to the research was whether the platform

would continue to be available to users when the project ended and the research team stepped

away. As such, the SABA People Cloud platform was more likely than any other to stand the test

of time. However, having the platform available also means that there needs to be someone

assigned to administrative duties and to technical support.

2.3.2 Installation and Deployment

Beyond the technology itself and the requirements of the user, social networking tools foster a

new and different set of work practices, which means that the process of implementation should

follow a strategy to help nurture adoption and change. The SABA People Cloud platform was

provided to CDA by SABA in August 2011 and access was initially limited to a select group of

TDOs who were assigned as technology stewards and chosen for their general interest and their

experience and motivation in using social media technologies. The concept of technology

stewards is well defined in Wegner (2009), the idea being that a small group populates and

participates in the online community and assists as required to create and maintain a healthy and

active environment. When necessary the stewards also provide technical support on how to use

the site as well as acting as gardeners, maintaining the content of the site. A total of 15 TDOs

were identified by CDA and approached to be technology stewards for the planned SABA People

Cloud site. Technology stewards were provided with an information package (Annex A) and

began using the site in late September 2011.


SABA People Cloud was deployed to the entire TDO community in October 2011. Members

were invited to join by an email (Annex B) that included a hyperlink to an initial survey to be

completed before access was provided to the social networking platform (via a link at the end of

the survey). It was important that participants complete the survey prior to using the platform so

that baseline data could be gathered before using SABA for comparison to data to be collected

after using SABA.

2.3.3 Pre-SABA People Cloud Deployment Survey

The primary objective of the survey, which was piloted by the technology stewards, was to obtain

an idea of each TDO’s individual social network of professionals and to determine whether there

was any impact on the size of the network through using the SABA virtual social network toolset.

This information was important to understand for the selection of an appropriate social

networking platform for the community. An online survey program, FluidSurveys5, was used to

deliver the questionnaire. It included a list of all 146 TDOs and asked individuals to rank their

familiarity with each of the TDO’s based on skills, expertise, and experience. On a scale of 0 to 3,

a score of 0 indicated that the individual did not know a particular TDO at all, while a score of 3

implied that they knew the TDO very well. This indicator was used to gather a picture of each

individual’s contact network and the strength of the ties within that network. To reduce the

chance of response biasing due to order, the list of names of the TDOs was randomly ordered

across respondents.

With the late start date, five months were available for the community to use the site, become

familiar with it, and possibly connect or reconnect with TDOs with whom they were not familiar

or had lost touch. This is a short time span for developing social networks and consequently the

data represents a very limited collection window.

2.3.3.1 Pre-SABA People Cloud Survey Results

Forty-nine TDOs completed the entire survey. To ensure survey data were collected, individuals

were instructed to complete a questionnaire before gaining access to the SABA People Cloud site.

However, it turned out that not all members joined via the email survey hyperlink. As

membership increased on the SABA site some members used a feature on the site to send a

message and invite others to join, thereby bypassing the email hyperlink to the survey. This fact

emerged by comparing the number of people who accessed the survey against the number of

members on SABA. Failing to complete the survey was a concern for data collection since the

invitation email was constructed specifically to maximize survey participation by introducing the

survey before access to the SABA platform was gained. However, the fact that SABA members

were inviting their colleagues to join points favourably to the social linking capability of social

networks and an individual desire for others to join the virtual community.

5 Survey Monkey was the original choice for delivery of the questionnaire, but was problematic because the

Canadian Department of National Defence (DND) requires that data be stored in Canada and Survey

Monkey uses US data storage locations.


Network analysis data

Familiarity based on skills, expertise, and experience that each TDO self-reported with all other

TDOs was calculated by averaging across the mean of each individual`s scores (0-3). The average

overall score was 1.10 out of a maximum score of 3. The results from this analysis, shown in

Table 1, will be used as baseline data for comparison with the same ranking task after the SABA

platform has been installed and used.

Table 1: Pre-SABA survey results – overall average score of familiarity with other TDOs.

Number of completed surveys 49

Average score (out of 3) 1.10

Standard Deviation 0.43

These figures represent a baseline against which a comparison will be made after the SABA

People Cloud platform has been used. The question addresses whether or not a virtual social

networking platform aids in building contact networks and familiarity with contacts.

2.3.4 An alternative to an enterprise-level social networking site

At about the same time that the SABA People Cloud site was being approved and finalized the

research team discovered that the TDOs had created their own virtual social networking place on

a publicly and freely available site, Facebook (www.facebook.com). The Facebook TDO Group

had been created several years earlier by a few TDOs who obviously recognized the need for this

kind of collaborative environment within their work community. The site was relatively dormant

with a handful of participants but the number of members increased dramatically just before and

during deployment of the SABA People Cloud platform. Whether timing of the increase in

membership was coincidental or whether it was spurred by community interest and discussion

around virtual social networking related to this research program is not known, or important for

that matter, but the unanticipated discovery of the site provided the research team with an

opportunity to compare two different social networking platforms that became available to the

same community at about the same time – one being an enterprise-level site, the other a publicly

available social networking site. The circumstance also allowed for a comparison of deployment

approaches – one official, structured, and promoted internally by the organization, the other

unstructured and entirely voluntary.

2.3.5 Facebook and SABA People Cloud deployment experience

2.3.5.1 SABA People Cloud

The SABA People Cloud platform met with many technical difficulties when it was first

deployed and users experienced continuing problems and frustrations throughout the research

period. Most of the issues were related to a few key problems: users initially having difficulty

logging on; the software was not fully compatible with the workplace Defence intranet (Defence

Wide Area Network (DWAN)) used by the majority of TDOs; and the version of SABA People

Cloud deployed to the community was not as rich, comprising fewer features and less capability


than the version previously demonstrated by SABA to the research team (e.g., chat, and RSS,

were not features available in the deployed version).

A solution for the more general login problem was provided (through exploration by one of the

TDOs) but it required individuals to download a software application and reconfigure their

system- a step that many were, understandably, unwilling to take because of time investment.

The DWAN on which the platform was run, was not a friendly environment for the SABA

platform and it often resulted in sluggish response to commands, and features that should have

worked easily sometimes failed or were slow. Automatic logout after a very short idle time was

also a constant frustration. When SABA People Cloud was identified as the preferred platform,

the CDA technical authorities were under the impression that the TDOs would access SABA

through GPNET (a DND approved general purpose internet connection) which most did not have

access to at their desks, or that users would work at home using their personal internet service.

The researchers on the other hand, understood that the tool would be used at work and

consequently that it would be available to the TDOs at their work station, served by the DWAN.

The research team was aware, however, that uploading files to the SABA site was not possible

using the DWAN and that individuals would have to use another network, such as finding a

GPNET served computer, to perform that particular task. There was no apparent resolution to this

problem and it was deemed an acceptable limitation.

It is difficult to pinpoint where the misunderstanding regarding the network and compatibility

with the SABA toolset stemmed. At least in part, it may have been indirectly a result of the fact

that parties were geographically distributed. Not only was dialogue carried on between CDA in

Ontario (Kingston) and DRDC Atlantic in Nova Scotia (Dartmouth), but information had to be

shared and discussed with other members of the research team who were dispersed across the

country, including DRDC Toronto (Toronto, ON), the University of Waterloo (Waterloo, ON),

and the human factors engineering contracting firm, CMC Esterline|Electronics (Ottawa, ON),

working on behalf of DRDC. The authors also note that there may have been an issue with use of

the word ‘internet’. For some, the term referred to any source of networking capability other than

the DWAN, which an intranet. For others, the DWAN was the internet – that is, just another

means to access the internet. Consequently, when talking about the internet in the context of

SABA implementation there may have been a bit of confusion.

2.3.5.2 Facebook TDO Group

The Facebook TDO Group was originally created by a member of the TDO community and was

quickly joined by a handful of other members who obviously recognized the usefulness and

benefit of using social media, like a virtual social networking platform, within their community.

As such there was no official invitation to join and word of mouth was the primary way of

hearing about the site. Because of the mechanism of information dissemination underlying

Facebook, individuals might also have learnt of the TDO Group if they were using Facebook for

other reasons. A request to, and approval by, an existing member was required to join, and full

membership was obtained immediately. Usually a new member was introduced and welcomed

after joining. As far as we are aware there were no technical difficulties experienced and the

process of joining was relatively quick and easy.


2.3.5.3 Summary of comparison of deployment strategies

Comparisons between the method of deployment for SABA People Cloud platform and Facebook

are shown in Table 2.

Table 2: Deployment strategy.

SABA People Cloud Facebook TDO Group

Officially sanctioned TDO group created by themselves for

themselves

Officially announced via CDA email Aware of Group through word of mouth or

by using Facebook for other reasons

Instructed to complete a questionnaire that

was required in order to join (a few people

bypassed this step because they were invited

by others already on the site)

No preliminary required

Encouraged to join by CDA email Through postings on the site, existing

members were encouraged to increase

membership

Technical issues – initial and on-going Technical issues – none

Network access – not smooth functionality

on DWAN; not accessible on mobile

smartphone

Network access – accessible anywhere with

internet including DWAN, and mobile

smartphone

2.3.6 Data collection

Further to the initial on-line survey, data were collected in several ways. The objective of the

surveys was two-fold. First, surveys were designed to gain an understanding of individual social

networks and to compare those networks for number and strength of connections before and after

the social networking sites became available. Secondly, the data would capture how the different

platforms were used and whether there were differences in activity level and use of features on

the two sites. Methods used for collecting data were as follows:

Pre-SABA deployment (as described above) and Post-SABA deployment surveys. Unlike

the initial survey, the Post-SABA survey included questions related to views on SABA

and Facebook;

Structured interview with community members;

A distributed experiment where TDO members utilized SABA;

Observation – researchers were members of both sites for the purpose of observing the

level of activity and content; and


Content analysis – data from both the SABA and Facebook sites were analyzed for

activity level and content using the HanDles visualization tool, developed by DRDC

Toronto and Ohio State University (Kwantes, 2011).

Detailed methodology and the results from each method are described in the following sections,

followed by a summary of the findings.

2.3.6.1 Post-SABA People Cloud Deployment Survey

In late September 2011 a second survey was delivered via email to all TDOs (about 150).

Forty-six, or about one third of the TDO community completed the survey. As in the first survey,

each TDO was asked to rate their level of familiarity with all other active TDOs on a scale of 0 to

3, 0 being ‘do not know this person at all’ and 3 being ‘know this person’s experience and

expertise very well’.

This survey also included questions that focused on use and experience with each of the two

platforms. For example, which platform, if any, individuals were using; how often they used the

platform; whether or not they migrated from one site to the other; what features were useful and

what they were used for; and which ones, if any, were unnecessary; whether or not any features

or functions that the TDOs might find useful were missing in the toolset; and whether or not there

was a sense that the TDO community was better off through using virtual social networking, and

particularly whether individuals considered the community to have stronger ties since having

access to a virtual social site. The response type varied and included a rating scale (e.g., ‘strongly

agree’ to ‘strongly disagree’), selection from a list of response options, and open ended answers

and comments. The survey can be found in Annex C (note that individuals' names are not

included).

2.3.6.1.1 Results – Post-SABA People Cloud deployment survey

At the end of the research period 98 people had joined the SABA site with 150 in the Facebook

Group. More detailed observations on the joining process are discussed fully in 2.3.8.3.1.

Table 3 contains a summary of the final survey results. Note that absolute values are quite small

because the survey branched into 4 possible versions based on the answer to the initial question of

“Please indicate which of the following tools you have used in a work context” where

respondents could select:

1. Both Facebook and SABA

2. Only Facebook

3. Only SABA

4. Neither Facebook nor SABA for work purposes.

Within options 1, 2, and 3 some of the questions were similar in which case responses were

combined for analysis purposes.

Keeping in mind that the data are based on individual subjective response and the number of

respondents is relatively small, the following observations were made.


Quantitative and Qualitative data

Table 3 shows a summary of post-SABA survey results.

Table 3: Summary of post-SABA survey results.

Platform used - 39% used neither SABA or Facebook

- 22% used SABA and Facebook

- 26% used only Facebook

- 13% used only SABA

- 61% joined at least one of the sites

The data shows that many respondents did not use Facebook or SABA, and

the data also showed that slightly more than half of those do not use social

networking sites at home either. The primary reason given for not joining a

social networking site for work was that respondents were too busy and did

not have the time for social networking on-line.

Facebook was used by the largest number of people (23) and SABA-only the

smallest, but most people who joined the Facebook TDO Group were already

using Facebook for other reasons. For those using both Facebook and SABA,

most of them joined Facebook first, or were using that site already, and then

joined SABA.

Invitation ....to SABA

- 6% joined when a colleague told them about it

- 75% joined through DRDC email

- 0% saw reference to SABA on Facebook

- 19% were introduced by ‘other’

....to Facebook

- 42% joined when a colleague told them about the Facebook TDO Group

- 39% were using Facebook already and saw postings on Facebook about

the TDO Group

- 0% saw reference to Facebook on the SABA site

- 19% were introduced by ‘other’

For the TDO Facebook Group, most people joined when another TDO told

them about the Group, while those using SABA said they were informed by

the DRDC email. Most of those who did not use SABA had heard about the

site but chose not to join.

90% of respondents were on Facebook already and some of those joined the

TDO Group after seeing a Group posting on Facebook. Although a small

number (stemming from the overall small number of survey respondents),

this observation points to the networking and communicating strengths of

this social network platform.

No one was introduced by seeing reference to SABA on Facebook. That is

not to say that they there were no such references on Facebook, but only that

no one reported that there were swayed to join SABA through that means.


Both Facebook and SABA were introduced to 19% of users in 'other' ways.

There is no way of knowing what specific means were experienced but

examples might have been introductions from people other than colleagues,

hearsay, and reference made in documents, emails, etc.

The data suggest that Facebook was generally joined because there was a

personal desire to become part of the on-line community, brought on by the

fact that other people were talking about the site, either vocally or virtually.

SABA, on the other hand, was joined because an official invitation arrived

(email) that included encouragement for TDOs to join the site to support the

research project.

That being said, we know through observation that, even though it is not

showing up in this data set, some SABA members had bypassed the

hyperlink to the survey contained in the initial email because they had been

invited to join by TDOs who were already members. Some of those

invitations were conducted using the invite feature on the site while others

might have been through verbal exchange.

Migration

between sites

One person said that they gave up on the SABA platform and moved to

Facebook, and their reasoning was that people were more connected on

Facebook, and Facebook had a better interface and less technical issues than

SABA, and was generally more user-friendly.

Accessibility According to the responses, 91% of the TDOs who completed the survey had

internet access at work.

As mentioned before, the wording in this question might have been

confusing because of the possibility that the definition of the word ‘internet’

was different for different people. For some, ‘internet’ may have meant any

network access other than the DWAN, while for others it may have referred

to any means of connecting to the world wide web.

24% of respondents accessed at least one of the social networking sites at

work using a mobile device. However, none of the SABA-only users used a

mobile device for this purpose so it is likely that it was only Facebook that

was accessed this way.

Community - 70% using both SABA and Facebook felt that the community was closer

- 75% using Facebook only

- 0% using SABA only

Although not all Facebook users experienced a greater sense of community,

most thought that the TDO community was more closely connected now that

they were members of a common Group on Facebook. None of the

SABA-only users rated the question that way.

Login

Frequency

As an approximate estimate of how much time users were spending on the

social networking sites the survey asked how often individuals logged on.


SABA users checked the site less than once a week, whereas Facebook users

were split between checking less than once a week, and checking once a day

or more.

Bearing in mind that this survey was administered at the end of the research

period, after both sites had been experienced, responses may have been

different had they been gathered at earlier times in the research window. That

being said, it is the long-term value that we are interested in, not the effect of

new and novel tools.

Usage According to responses to the survey, Facebook was used to request,

respond, and very often to obtain information through passive reception

(60%). On SABA, 57% of users responded to a posted request by another

member, but only one person stated that they had obtained information

passively, that is, without actively seeking it.

One of the greatest strengths of social networking platforms like Facebook

and SABA is their ability to distribute material to the entire community

making it possible for information to be found without asking. At times such

information might be important.

Preferred

Features

To support their work people liked the discussion thread feature of the sites

most, followed by:

Facebook – media (photo) sharing, news feed, user profiles, private

messaging, and groups (note that chat and notifications were not the

highest score).

SABA – notifications and groups (all other features were scored 0).

Some people increased the privacy level on their Facebook accounts (~50%)

as a result of joining the TDO Facebook Group, and about 75% changed the

way they use Facebook (although the specifics or how that might be were not

captured).

The majority of users (~76%) are comfortable saying they are on Facebook,

believe TDOs are more connected since Facebook, and would be sorry if the

site went down. Overall, Facebook seems to be quite well integrated into

most people’s work day.

The majority of users on SABA (60%) will not be disappointed if the site

goes down and they do not believe that SABA is part of their daily routine,

or that it makes their activities more efficient. Users (61%) feel very

comfortable about telling CF colleagues they are on SABA.

About 75% of all users feel that having a social networking site could make

getting to know someone met on-line easier. Interestingly, all the SABA-only

users agree with this statement (even though they reported that they generally

did not like SABA).

Social Networking – what is of value:

Most people did not think that finding new people was of greatest

importance, but for SABA-only users 40% thought it was.


Of importance was reconnecting with people with whom they had

lost contact.

Seeing what is on status updates was important.

And the continuous dialogue on the sites was the most important

feature for everyone.

Being concerned about the stigma of being on a social networking site was

not a factor for most.

Table 4 shows a summary of the responses to open-ended questions in the survey. Questions are

in italics, responses in regular text.

Table 4: Summary of responses to open-ended post-SABA survey questions.

Are there features or characteristics missing from Facebook that could help you do your work?

- Document sharing; VTC via Facebook (GoToMeeting features); more professional

look; more interactive rating based system (like Stack Exchange); save discussion

posts; organize/reference posts; TDO directory.

Are there features missing from SABA that could help you do your work?

- Chat; as well as more general ease of use, accessibility, participation.

Comments, issues or concerns that affect your use of Facebook at work?

- Facebook privacy concerns

- Facebook stigma while at work

- Only use with people already friends with. Would not pose questions to larger TDO

group that included people not known

- Security issues (will not use at work or for work-related things)

- Can easily use on other devices (Black Berry, Smartphone, Playbook)

- Know some colleagues better, can reach out with questions

- Useful material posted relevant to CF

- Q&A repository, reputation reviews

- Accessible inside and out of DND – means can tap into larger community

- French-English language barrier, mostly because acronyms and sloppy spelling/writing

- Easy to use

- Good place to meet and chat

- Is a time waster

Issues or concerns that affect your use of SABA at work?

- Accessibility

- Not user friendly

- Too slow, cumbersome, was frustrating

- Lack of activity

- Annoying emails

- No user presence

What general comments do you have regarding the use of social networking in the workplace?

- Lack of time for social networking, yet the usefulness is understood

- Frustrated by too many tools, constant starting over

- Need for integrated tool

- Repeat conversations that are discussed in email threads are replicated on line

- Not everyone is involved


- Chain of Command needed

- Needs guidance (more structure needed)

- Is the intent for informal learning or work?

- Needs to be a reason to participate

- Security

- Email, phone, or face to face is good enough

- No time for using this kind of tool

- Professional information/work should happen inside work during work hours on secure

network

If you have any general comments about social networking in the workplace or about

Facebook or SABA in particular, please add them here.

- Notifications – Facebook to personal email, SABA to work email; no user presence on

SABA

- Already comfortable with Facebook so was easy to use

- Can keep track of contacts

- Social networking tool in general is powerful professional development tool

- Privacy – open forum can be hostile and non-welcoming, not free to express opinion

- There has to be a reason for using a tool

- Suggestions – chat that works across DND firewall

Network analysis data

The last section of the post-SABA survey was identical to the initial survey in which

intra-member familiarity was evaluated. Again, the intent was to gather information on how well

the respondents knew other TDOs based on skills, expertise, and experience, on a scale from 0 to

3. The analysis took the average of each individual`s mean score. An overview of the results from

this section of the survey is shown in Table 5.

Table 5: Post-SABA network analysis – overall average score of familiarity with other TDOs.

Number of completed surveys 44

Average score (out of 3) 1.11

Standard Deviation 0.43

2.3.6.1.2 Pre- Post-SABA network analysis data comparison

The results from the pre- and post-SABA survey question on the knowledge of other TDOs were

compared. Results are detailed in Table 6. Since there are numerous subsets of the TDO

community referred to in Table 6, Figure 1 is provided to help visualize the relationship between

the represented sets of people. The alphabetic references in Table 6 are associated with Figure 1

where a visualization of the data is provided.


Table 6: Comparison between pre- and post-SABA network

analysis – familiarity with other TDOs

Response rate/

Use of social

networking sites

19 respondents (a) answered both the initial and final surveys; 14 of

these reported that they used social networking sites (Facebook and/or

SABA) for work; 5 did not use social networking sites for work.

Average rating Looking at all the responses from the beginning survey, the average

rating across all 147 assessed TDOs (b) for the 49 (complete) survey

responders is 1.10 on a scale of 0-3. Looking at all the responses from

the final survey, the average rating across all 147 assessed TDOs (b)

for the 44 (complete) responses (d) is 1.11 on the same scale.

Change over time i) The familiarity level of the 14 TDOs using social networking, changed

on average by +0.13 (4.4%); for those not using social networks, the

change was slightly less (+0.11, or 3.6%).

Change over time ii) Focusing only on the TDOs who were members of the TDO Facebook

group for at least 3 months (i.e., for whom changes due to social

networking might be expected), the group being assessed reduces from

147 to 55 (e) (since you cannot attribute changes to social networking

usage if the person being rated was not using social networking). Nine

(f) of the 19 respondents (a) that answered both surveys also fell into

this category. Their assessment of the 55 TDOs (e) also using the

group for at least three months, showed a network change of +0.21

(+7.1%). For those not using any social networking tool, their

assessment of those same 55 people amounted to only +0.11 (+3.8%).

Subjective view of

familiarity over time

– users of social

networking sites

The 9 respondents (f) who completed both surveys and used social

networking for at least 3 months assessed only 5 of 55 people (e) as

being better known by at least +0.5 (on a scale of 0-3) after using the

social networking sites.

Subjective view of

familiarity over time

– non-users of social

networking sites

In contrast, the 5 responders not using social networking assessed 8 of

the same 55 people as being better known by at least +0.5 (on the scale

of 0-3). None of the 8 people deemed as better known overlap with the

5 people identified as better known by those using social networking.

One point to note in the data is the increase in familiarity with other TDOs in individuals who

were using neither of the social networking sites.

Figure 1 shows a visualization of the subsets of the TDO community referred to in Table 6.


Figure 1: Visualization of the subsets of the TDO community referred to throughout Table 6.

2.3.7 Symposium

In mid-November 2011 the research team was invited to conduct a syndicate session during the

Canadian Defence Academy (CDA) TDO symposium held in early December in Kingston,

Ontario. This event provided a welcome venue for collecting additional data from the TDOs and

over the next three weeks the team prepared a structured interview format (2.3.7.1) and an on-line

experiment (2.3.7.2) designed to investigate the use of social networking tools. The purpose of

the interview was to progress the team’s understanding of how TDOs, as professionals, use

virtual social networks and Web 2.0 tools, and to gather their opinions about using such tools in

their working environment. We wanted to understand from the TDOs’ perspective what their

opinions were about using the Facebook and SABA People Cloud platforms that had been

provided to them, as well as their views on using social networking at work in general. The team

was also asked for their viewpoint on the concept of Technology Stewards. The purpose of the

experiment was to use the SABA platform as a means of assessing the usefulness of social

networking for finding expertise and sharing information across distributed teams.

Ten TDOs, selected by CDA, participated in the syndicate session (2 Lt(N); 2 LCdr; 5 Capt; 1

Maj). They were randomly divided into two groups of five, with one group (Group A)

participating in the experiment first followed by the interview session, and the other group

(Group B) participating in the interview first and then the experiment. The syndicate ran for one

and a half days. The interview and experiment were conducted on Day 1 and a round table

discussion was held in the morning of Day 2. The interview and the experiment were reviewed

and approved for data collection by the DRDC Human Research Ethics Committee (HREC).

2.3.7.1 Structured Interview

The interview questions and response notes are listed in Annex D. The questions were used to

lead discussions but the interview sessions were by no means directed completely by the

questions. For the most part, once a discussion began, the interviewer let it run its natural course,

( b ) 147 TDOs were rated in Survey 1 and

Survey 2

( c ) 49 TDOs completed Survey 1

( d ) 44 TDOs completed

Survey 2

4

9

6

1

( e ) 55 of the 147 TDOs rated in

Survey 1 and Survey 2 used SN for > = 3

months

( a ) 19 TDOs answered both surveys

9

( f ) 9 TDOs completed both surveys AND used

SN for > = 3 months


stepping in to ask pertinent questions for more detail when necessary, or to request the opinion of

less vocal members. Along with the primary interviewer, two members of the research team took

notes during the sessions. After the interviews, the notes were collated and summarized. The

following section provides a synopsis of the findings from the interview sessions.

2.3.7.1.1 Data analysis

Overall the syndicate participants were savvy social network users; only two had no experience

with any social networking tools and seven considered themselves experienced on Facebook. The

experienced users averaged 4 years on Facebook and generally interacted with Facebook daily as

part of their personal lives.

With respect to using social networking as a professional tool they preferred Facebook as it had a

more familiar and easier to use interface. The few who did use SABA site were dissatisfied with

the interface, the slow speed of the network connection to the site, and the issue of being logged

off when inactive on the site.

As a group they felt that a number of Web 2.0 features, not available in social networking sites,

would be of benefit to professional work. These features are the ability to manage documents,

organize information into folders by subject, and thread conversations.

Generally TDOs were familiar with social networking and with other Web 2.0 tools and they

think of themselves as early adopters and embrace the idea of social media at work. Through

discussion the following features were identified as required by TDOs in order to use social

networking to support work: email prompts, threaded discussions, subject search, persistent data,

folder structure, and the ability to open and keep documents open. To enhance professional

relationships the following features are required: professional chat (with presence awareness),

user resume (experience, degrees, past employment), and audio and video feeds.

Overall the TDOs felt that the largest issue with the usage of social media at work was the stigma

of being on a ‘social’ site during the work day. This point was discussed at length and the

individuals being interviewed felt that it was of significant concern.

The participants felt that shepherding new users and site maintenance would require a large

effort, on top of an individual’s already high workload, and would likely be seen by managers as

an additional task and one that might take away from a TDO’s primary responsibilities. Thus, it

was generally agreed that it would be beneficial if technology stewards were self-identifying and

that, with the amount or work required to be an effective technology steward, the job should be

classed as a primary responsibility. The TDOs see themselves as change agents and as suitable

candidates for technology stewards. To be effective they felt that the role of technology steward

would need to be empowered to act as moderators for discussions (threads) and blogs. They also

felt that direct access to technical support would be a necessity.

Overall the TDO’s felt that social networking will add to the sense of community and increase the

sharing of professional knowledge and also that this kind of tool could be used to capture

knowledge from the experienced TDOs who are about to retire. On this thought they felt that

recently retired TDOs should be given access to the professional site used by the TDO

community, even if it is hosted on the DWAN, so that they can remain connected to the


community and be able to provide their viewpoint and advice if needed. Whatever form a social

networking site takes it should be the single repository for data; the site must be associated with a

reactive Community of Practice. Site structure is required including a set time when online

forums will meet. The site would require a dedicated worker / champion / technology steward and

support from the TDO / CDA leadership is a must in order to overcome the stigma of working on

a social networking site during the work day.

2.3.7.2 Experiment

An experiment using the SABA platform was created to evaluate how social networking enables

information sharing and finding experts across individuals that are distributed and differ in the

knowledge they hold. The intent was to better understand the benefits and issues related to on-line

collaboration and information sharing. The experiment required team members to use the SABA

People Cloud platform to connect with experts, gather information from experts, and disseminate

facts to team members, with the objective of solving a collective problem. The nature of virtual

social networking is that communication between members is made visible to all other members.

It was hypothesized that actions carried out and information gathered on-line by one member

could potentially be of use to other members in the group, thus speeding up the process of

gathering information relevant to the problem-solving task.

2.3.7.2.1 Experimental Task

In order to emulate a real-world social network, the SABA platform was populated with volunteer

confederates with whom the syndicate participants had to find, connect, and interact with, to

gather information. To facilitate the need for information sharing and finding expertise a problem

solving task was developed for the syndicate participants. Syndicate participants were told of an

impending terrorist attack and their task was to search for information that would help uncover

certain elements of the attack – namely, who is attacking; what country is being attacked; what

the target is; and when the attack is supposed to take place. Each of the confederates held certain

facts that needed to be retrieved in order to figure out these details of the terrorist attack6.

Confederates were strategically assigned facts based on information listed in their SABA profiles

that were available to the participants. Syndicate participants were given the following

information to determine who they should contact to find certain facts:

– A person might have date information about the terrorist attack if their year of birth is listed

in their profile.

– A person might have target information about the terrorist attack if they listed where they

went to school in their profile.

– A person might have information about ‘the Tiger’ (a hired gun, basically) if they have

where they are from listed in their profile.

– A person might have information about the countries under attack if they have where they

live listed in their profile.

– Finally, all confederates could have group information.

6 The scenario was a modified version of that found in DRDC Human Research Ethics Committee (HREC)

Protocol L-704, Understanding Competitive and Co-operative Problem Solving Behaviour in Meta-Team

Situations, Filardo, 2009.


The assignment of facts to confederates was not mutually exclusive, therefore a confederate could

have information about both when and where the attack could take place. The number of facts

held by each confederate varied. Confederates were instructed to provide facts only when asked a

question by a syndicate participant. The confederates were asked to answer truthfully (e.g.,

'I know that the Tiger will not work with locals' or 'I don’t have any information about the Tiger').

If they were asked a question that they did not have the information for, but they knew who might

have the information, they were instructed to direct the participant to that person (confederate).

The experimental task was designed to last for 3 hours with Group A participating in the first

1.5 hours and Group B participating in the last 1.5 hours. Group A was told to document their

findings in such a way that they could easily hand-off the information to Group B. The intent was

that Group B would pick up where Group A left off and use the connections and facts that they

developed to continue on.

2.3.7.2.2 Participants

Participants. There were ten TDO syndicate participants in total; five participants in Group A and

five participants in Group B. With the exception of one participant that had no social networking

knowledge, all of the other participants had used a social networking platform before (mostly

Facebook).

Confederates. There were 12 confederates. Confederates were volunteers from DRDC Toronto

and DRDC Atlantic. Knowledge of SABA was similar across individuals.

It should be noted that not all profiles listed in the SABA platform were confederate profiles.

SABA was released to the TDO community for a trial phase, and at that time a number of TDO’s

joined SABA to try it out.

2.3.7.2.3 Procedure

Prior to the syndicate, participants and confederates were asked to login to the SABA social

platform to populate their profile with information and to get a “feel” for how SABA works. A

SABA “how-to” PowerPoint slide deck was provided to the confederates via email and to the

syndicate participants as a presentation to facilitate the use of SABA. Syndicate participants were

asked to read and sign a DRDC HREC informed consent form prior to beginning the task.

The syndicate session took place at the Royal Military College (RMC) in Kingston, Ontario. A

laptop computer was provided to each of the syndicate participants for use during the experiment.

All of the laptop computers were networked to the DWAN. Syndicate participants were required

to login to the DWAN and access the SABA social platform. The syndicate participants sat

around one large table. They also had access to a computer with a projector and a flipchart.

Once the syndicate participants were presented with the task and provided with the instructions

they were left alone to develop their own strategies and collaborations. They were allowed to talk

amongst themselves and to take notes in order to keep track of information found and experts

identified in the on-line confederate community. Two researchers were in the room during the

session to answer technical questions and address problems as they arose during the session. The


researchers were also documenting behaviours, communications and difficulties that surfaced

during the experiment.

2.3.7.2.4 Results and Discussion

The results from the SABA platform experiment are qualitative in nature and are based on

observations noted by the research team during the experimental session.

Most of the participants had not familiarized themselves with SABA as instructed prior to the

experiment.

Technical Issues and Usability Problems

The SABA login and set-up time took longer than expected (approx. 30 min) which left less time

for the actual fact finding task. Most of the syndicate participants had not had time to join and

become familiar with SABA prior to the experiment day. There were also some technical issues

during the session, mainly with computers requiring a restart as a result of security scans. In both

groups much of the conversation during the session was related to usability issues in the SABA

platform – How do I…? ; Where did you see...?; Do you know where...? While participants were

given a “how-to” tutorial immediately before the experiment, it appeared as though the platform

was not intuitive or easy to use. In particular, information seemed to be easily lost and not easily

shared amongst other members of the group. For instance, one syndicate participant noted that

while he was receiving notifications about confederate replies via email notification on his

BlackBerry he was unable to find where they had been posted in SABA. Similarly, participants in

Group B were telling each other when they or someone else received a message, suggesting that

SABA was not efficient at notifying people when they receive messages/posts. A number of

participants made mention of the fact that it was difficult to find things and information seemed to

get “lost” once they had posted it. It was also noted by syndicate participants that there was a lot

of irrelevant information included in the SABA platform, such as "who was following who",

making it difficult to find valuable factual information about the terrorist attack.

Group A: Collaboration Concerns and Strategies

Collaboration and networking were key to successfully determine the – who, what, when, where,

why and how of the terrorist attack. Participants were told that confederates were likely people

they did not know so that “adding all contacts” was the best way to connect with potential

confederates and view comment threads. In light of this, syndicate participants were notably timid

to reach out and connect with individuals they did not know. A significant amount of the session

was spent trying to decide which of the individuals on SABA were most likely to be confederates.

Being unfamiliar with the individuals seemed to be of concern, as was soliciting information in a

'cold-call' manner. Comments about 'bugging' people that may not be confederates were made

multiple times. Eventually they narrowed down the list of potential confederates by looking at

who had recently posted on the SABA wall or for people that were recently added to SABA.

Syndicate participants also expressed concern about the number of emails they (and those they

were connecting with) would have to weed through as a result of the SABA default to send email

updates when contacts were added. This may have created some reluctance to connect with

everyone.


Information began to flow between confederates and participants once the group built a sufficient

network. Syndicate participants tried to set up a dedicated Group in SABA to coordinate and

share information, but unfortunately a technical issue in SABA would not allow them to create a

group. As a result, participants verbally announced to everyone in the room when they contacted

someone, and when they received information about the attack. Once facts were being received

with some regularity, and therefore becoming difficult to keep track of, a member of the

syndicate session began logging them on a piece of paper. The method of reaching out for

information differed where some chose to reach out directly to potential confederates while others

decided to broadcast their questions as part of their status, therefore leaving the question open for

any confederate to answer.

Transition from Group A to Group B

A member of Group A provided a verbal handoff to all members of Group B. An explanation of

the task, the status of the task, and the paper copy of the collected facts were provided to Group B

during this time. At this time, the PowerPoint SABA “how-to” and instructions discussed above

were also presented to Group B.

Group B: Collaboration Concerns and Strategies

Group B’s first strategy was to compile all facts into the “Idea” area in the SABA platform so that

all members had access to the facts that had already been collected. Even with the facts compiled

in one central area the syndicate participants used a flipchart to strategize and determine how/if

the information was coming together in way that would allow them to solve the problem. Overall,

Group B seemed to be more tactical in their approach and with the type of questions they were

asking. They spent more time reading into the information, asking elaborate questions and

creating theories about the scenario.

Group B seemed to pick-up the task and develop strategies quite quickly. Having said that, it can

be speculated that Group B was quicker in picking up the task because the network was already

developed, therefore allowing them to dive into the task without having to consider who was a

confederate. It should also be noted that Group B did not expand their network of confederates;

instead they relied on the network developed by Group A.

2.3.7.2.5 Conclusion

Overall, syndicate participants in both groups enjoyed the task and found it engaging. While there

were some technical and usability issues related to the use of SABA we found the task to be

useful in determining how information is shared and how the social networking platform can be a

useful tool in finding experts who are not co-located. Participants had obvious concerns about

contacting people they did not know and bothering people that were not involved in the task. In

this case we were forcing participants to develop a network which may have made the “cold-call”

feeling more prominent than a network that evolves over time.

2.3.8 Social networking platform user data analysis

The two virtual social networking sites were analyzed for activity level and content by data

extraction and analysis (2.3.8.1) as well as by direct observation (2.3.8.2). For the former, a data


visualization tool was used and for the latter, research team members observed activity on the two

sites through membership on each site.

The objective of analyzing activity level and content was to understand several aspects of

communication and collaboration in a distributed community using an on-line platform, such as,

whether the site was busy with conversation, and if so was the activity on-going or intermittent;

what kinds of topics were being discussed and which topics stimulated conversation; whether

there were a handful of contributors or whether many were participating; how many were being

proactive and posting initial dialogue, and how many were simply responding to existing posts

but not initiating posts themselves, as well as how many were silent7. In other words, how was the

platform being used and for what purpose.

2.3.8.1 Data analysis – visualization using HanDles

A data analysis tool, HanDles (Kwantes, 2011), was used to visualize the data collected from the

two social networking sites. HanDles is designed to provide a visual representation of semantic

relationships between documents and their content. HanDles assesses and displays the content of

the documents, or web-site activity in this case, as points on a scatterplot. The more semantically

similar, the closer the points. The analysis puts documents sharing a query term in close

proximity to each other but it also identifies concepts, called ‘handles’, that help the analyst

understand what topics are being discussed and which ones are common between documents.

‘HanDles’ can show which topics are being discussed frequently in the documents and, as such,

provides the analyst with an in-depth picture of content. As an example of a visual representation,

below in Figure 2 is the output for the query “George Clooney” presented to 17000 articles from

the Internet Movie Database (Kwantes, 2011).

7 It is not possible to tell whether those who were silent were observing but not actively contributing or

absent altogether.


Figure 2: A screen shot of HanDles in response to the query ‘George Clooney’.

Documents are typically built on a number of different topics and the relationship between one

document and another will change depending on the topic of interest. The advantage of HanDles

over other data visualization tools is that the HanDles software allows documents and ‘handles’ to

be interactive. Dragging a ‘handle’ (using the mouse cursor) from one location on the screen to

another will move other documents and other ‘handles’ that are related to that one particular

‘handle’, while those that are not related remain still. This feature allows the user to manipulate

the semantic organization of the documents that have been returned by a query and consequently

to gain a more in-depth understanding of the topic or concept of interest. It is also possible to find

all related documents containing a specific ‘handle’ (by clicking on that particular ‘handle’).

Figure 3 contains a list of the column titles recognized by HanDles (from the HanDles help

documentation).


Figure 3: HanDles column titles.

Of relevance to the current research were fields that provided information about the kind of

material being posted, the number of people posting and commenting, and the different ways

people were engaged in use of the site. Although specific names of contributors were not of direct

interest, valuable information was obtained from knowing how many unique initiators of posts

(that is, the number of people who started post threads), and how many unique contributors there

were (that is, the number of people who contributed to an existing post by commenting on it).

These data were important for understanding how the site was being used, for what kinds of

information and tasks, and for providing indices of the number of people actively using the site

and the different ways they were contributing. Of interest to the research was the way networks

operate and particularly how activity initiated by one member is disseminated throughout the

network. That action of dissemination and passive reception of information is an important one

because it has the effect of exposing information to all network members and opening up the lines

of communication and collaboration that otherwise may not have existed had the information

been shared with a limited group selected by the user.

One of the features on the Facebook platform is the ability to favourably tag a post or comment

using a ‘Like’ tab (displayed under each individual post or comment entry)8. The number of times

an entry was ‘liked’ was used as another measure of the number of contributors.

The date of the post was also required to understand the timeline and to determine whether the

level of activity changed across time. Changes across time could give an indication of consistency

of use of the platform. For example, heavy initial use that tapered off considerably across time

might be indicative of a novelty factor.

8 This feature was not available on the SABA People Cloud platform.

Title: A title for the document.

Fulltext: Full text is the body of text contained in the document.

Handles: Handles or keywords must be entered as a semi-colon separated string.

Authors: Authors are formatted in a semi-colon separated string. For example: "Laurel, S;

Hardy, O"

Locations: Similar to handles and authors, locations are also given as a semi-colon separated

string: "London; England"

Dates: The format for dates is more complex, a "String" with YYYYMMDD is currently used,

however this will be extended to YYYYMMDDHHMMSS in the near future and you are

welcome to use this extended format now.

Longitude: These coordinates need to be in decimal format using negative for West.

Latitude: These coordinates need to be in decimal format using negative for South.


2.3.8.1.1 Data extraction for analysis

HanDles was originally designed for document analysis and so it was modified for this research

to analyze text in the form of users’ posts and comments from the SABA and Facebook TDO

Group social networking sites. A post is defined here as an initial parent entry whereas comments

are responses to that entry. In order to conduct the analysis, the data from the two sites were

extracted and prepared in a HanDles acceptable format.

Two different methods were used to extract the data from the two sites. The work was conducted

by CMC Esterline|Electronics under contract to DRDC Atlantic.

Data extraction SABA People Cloud

With SABA People Cloud the data collection window ran from October 2011 to February 2012.

A first attempt was made to import the data from a *.dmp file provided by SABA. However the

file was very large, containing much more data than required for the analysis, such as system

tables, and the contractor estimated that the task of extracting relevant data from this file would

take a significant amount of time. Since the SABA data contained a relatively small number of

posts and comments it was deemed more efficient to manually copy the data into an Excel

spreadsheet, formatted like the Facebook data spreadsheet. This meant the same script could be

used to convert both the Facebook and the SABA spreadsheet into the Handles-specific *.xml

format.

Data extraction Facebook TDO Group

The Facebook data collection window ran from November 2007 until February 2012. Data

extraction for Facebook was a multi-stage process, that included i) detailing a method of

extraction; ii) extracting the data using Facebook API; and iii) developing a Microsoft Excel

macro. The data were then further formatted for use in the HanDles program. The full process is

described in more detail in Annex F.

2.3.8.2 Data analysis – Observation

The project team members joined the Facebook and Saba People sites and were able to gather a

general picture of the amount and kind of activity going on by passively observing the sites and

noting the frequency of postings and comments and the topics that were being discussed or

shared. Observation on both sites began when SABA People Cloud was provided to the TDO

community at large and continued until five months later. A longer observation period of ten

months was originally planned but the delay in identifying an appropriate platform for the TDOs

reduced the amount of time available since the project was scheduled to end in March 2012. The

findings from the observation data are reported in 2.3.8.3.1.


2.3.8.3 Results

2.3.8.3.1 Quantitative and Qualitative results

Looking at the visualization data first, the analysis is at a higher, less detailed level, than the

HanDles software merits because the sample size and number of resulting data points are small.

Capturing a more comprehensive data set might have been possible if there had been more time

for data collection but even if that were the case, the limited number of participants would have

constrained the output. Despite the small data set the HanDles software proved very valuable for

the analysis and in providing a means to visualize the data.

The data showed that 98 people joined the SABA site and that, apart from the technology

stewards who had access earlier, the majority joined within the first month of the platform being

available. At the start of the data collection period, when the SABA People Cloud platform was

first deployed, there were less than 50 members in the TDO Facebook Group, and by the end of

the five month data collection window there were 150 members.

In total, 16 posts and 39 comments, for a total of 55 entries, were made on the SABA People

Cloud platform within a 5 month period (October 2011 to February 2012); and 273 posts with

892 comments, totalling 1165 entries were made on the Facebook TDO Group page between

January 2008 and February 2012 (January 2008 being the approximate timeframe in which the

Facebook Group was created). Limiting data collection on Facebook to the 5 month window in

which SABA was monitored, the number of posts was 166 with 568 comments, totalling

734 entries. Although the Facebook time window for overall use is much larger than SABA`s it is

clear from the data that the majority of entries on Facebook were made just before, or after the

introduction of the SABA People Cloud platform. In fact, one post on Facebook in April 2011

asks members for feedback as to what an appropriate virtual social networking tool might be for

the TDO community and 26 comments make up an on-going discussion on the subject. As

previously stated, the number of members tripled over the 5 month data collection period. The

increase in membership and activity on Facebook just before and after the SABA platform arrived

signified to the research team that the platform was new to the majority of the community and

that making comparisons to the similarly new SABA People Cloud platform was fair and the

results valid.

Figure 4 shows a timeline for the summary of SABA activity, and Figure 5 the same for

Facebook. The different coloured lines are not relevant for this phase of the analysis but each one

represents a different handle generated automatically by the HanDles program from the content of

the posts and comments.


Figure 4: Visual of 5 months of activity on the SABA site for the top generated handles.

Note initial and final activity rates.

Figure 5: Visual of 4 years of activity on the Facebook TDO Group page for the top generated

handles. Note the increasing activity in July 2011 as the experience with SABA began.

Engagement and level of activity was also measured by the number of `likes` entered on the

Facebook site. As shown in Figure 6, activity, as measured by `like`, increased around September

2011, just as the SABA platform was about to be deployed to the community. Unfortunately the

`like` feature was not available on the SABA site so it is not possible to use this measure for

comparison purposes.


Figure 6: Number of 'likes' over a 2 year period of activity on the Facebook TDO Group page.

Looking at membership activity and the content of the contributions to the sites in more detail, on

the SABA site, of the 98 members, 22% (22) contributed. Important to note is, of those 22, seven

were members of the DRDC research team organizing and participating in an on-line experiment

conducted in collaboration with CDA in November 2011 (2.3.7.2). Of the 56 total entries on the

site, 59% (33) were on work-related topics, 12 (21%) posts and comments were associated with

the on-line experiment, and the remainder (23%) were devoted to problem-solving and queries on

the functionality and usability of the platform.

For the purpose of comparison, this part of the Facebook analysis is limited to data collected

between October 2011 and February 2012. Of the 150 Facebook members, 74 (50%) actively

contributed to the TDO Group by posting or commenting, and 41 (55%) of those were initiators

of posts. The level of activity for those who contributed varied on an individual basis and so,

there were 13 primary contributors (8.6% of the total membership population), defined as having

made 10 or more entries over the course of the data collection period. Two of those (1.3%) were

heavy contributors with 103 and 175 entries respectively during the 5 month window. These data

fit with the generally observed pattern of behaviour in on-line communities, such as Wikipedia

and YouTube, where member activity tends to fall into one of three categories. Often referred to

as the 90-9-1 rule (McConnell & Huba, 2006) where 90% of members observe but do not

contribute, 9% contribute on occasion, and 1% contribute heavily and thus account for most of

the site activity. Looking at SABA, the overall amount of activity is small resulting in very low

power. Nevertheless, the results showed that no member showed a particularly heavy contribution

rate but 1person (1%) contributed more than the rest with 5 entries, 22 (16%) contributed on

occasion, leaving about 83% as non-active. As pointed out previously, it is not possible to

determine whether the non-active members were passive observers or simply not logged on. The

results also do not include the `likes` on Facebook, of which there were 402. Those people might

be counted in the non-active group when in fact they were active.

It is not clear if the 90-9-1 rule applies to all kinds of on-line communities in the same way. For

example, is there a difference in sites like Wikipedia, that are more about sharing and viewing


and less about conversation and dialogue, compared to social networking sites, like Facebook and

SABA, that are designed to promote collaboration and communication amongst members in

many ways. Furthermore, what prompts people to contribute and actively participate is a question

worth investigating.

On the Facebook site, during the period October to February, only two topics appeared that were

not work-related, and those were the welcoming of new members, and reporting on the number of

current members as membership increased. On occasion comments included encouragement to

try to reach the next goal with respect to number of members. The majority of posts and

comments revolved around the general categories of: questions, opinions, discussion on specific

topics, reporting on events, reporting on current work assignments, and requests for information.

A couple of posts used Facebook as a platform to discuss SABA. One questioned the

functionality of SABA with regards to whether it could be accessed on a smartphone, and one

individual commented that they were coming back to Facebook because SABA was problematic.

Some individuals expressed their sentiment on having the community available on Facebook, all

of which were positive posts, such as how good it was to be connected and how information

broadcast via Facebook would otherwise not have been known. Of interest too were numerous

tweet-like9 `status updates` commenting in real-time about presentations and activities during the

TDO Annual Conference in December 2011.

To get a broader feel for this social networking site, we examined activity and content on the

whole Facebook data set from 2008 to February 2012. The general topic categories were the same

as in the five month window data. Over the whole data set there were 273 posts with 892

comments and the average number of comments on a post was five. Significant deviations from

that average were present for a few posts and at least two of those were discussing social media

tools, the role of virtual social networking for TDOs, and the best choice of platform that would

fulfil the community`s needs. Both of those discussions happened in April 2011 as the decision

for a research platform for the TDOs was being made, indicating that the group was using the

social networking tools that they already had at their disposal and suggesting that this kind of tool

was a strong requirement for the community. As an indication of the kinds of topics that

prompted group discussion, the posts with the largest number of comments were:

– Asking for ideas on the most cost effective way to conduct review boards with distributed

participants and which collaborative toolsets might fit the bill (31 comments; April 2011).

– Asking for feedback on virtual social networking for the CF Training Development

Community and what might be the best platform to use (26 comments; April 2011).

– Discussion on course terminology (25 comments; October 2011).

– Asking for experience with a particular performance measurement tool used in the army

(24 comments; September 2011).

– Seeking out who is the TDO at a particular location (18 comments; January 2012).

According to the findings from the survey data, interviews, and content analysis, the most

attractive benefits to using the Facebook TDO Group were reconnecting with people with whom

a member had lost contact, and the continuous dialogue (in the form of posts and comments) that

9 A `tweet` is a text-based limited character message sent and received on-line. The term originates from

the on-line microblogging service, Twitter.


was on-going within the community. The TDOs found this to be a valuable asset because

dialogue on a platform like Facebook is visible and available to all, which means everyone can be

part of the conversation and everyone is aware of the information being posted and shared.

Conversations on Facebook often focused on bettering the working environment and the need for

social media and collaborative tools in the TDOs` workplace, but much of the dialogue

acknowledged that tools themselves are not the solution. Refreshingly, TDOs as a group appear to

recognize the importance of the human component in technology and their hope is to supplement

the user and support their roles with appropriate technologies. A considerable amount of

discussion followed this line of thinking as members offered suggestions and options. Many of

the other topics addressed, not surprisingly, revolved around terms associated with learning.

Despite the positive comments about the Facebook environment, many commented that it was not

a 100% solution. Some of the TDOs said that they would prefer a more professional look, and

some had concerns with privacy and security while using the site for work. Having the ability to

organize and save discussion posts was a feature on the TDO wish list, as well as having real-time

chat that works across the workplace firewall. Having the site integrated with a TDO directory,

and the capability for virtual teleconferencing were other desired attributes. All in all, an

integrated toolset was an important concern, so that all essentials are in one place working

together. Another desirable feature was the inclusion of an interactive rating-based system where

posts and comments from members could be rated for relevance and importance by peers. This

process would bring the most beneficial information to the top so that other searchers acquire

access to the best references quickly, and are assured by the confidence ratings of their cohort.

Based on users’ comments on the SABA site, it was clear from the start that the SABA interface

was difficult to use with features that are useful in a social tool missing (e.g., real-time chat).

Learning how to use the technology took time, something few professionals have room for in

their busy days, as well as a significant amount of effort. Achieving tasks using the platform was

often difficult and cumbersome, or proved unsatisfactory. In addition, the computer network and

connection were slow and sometimes erratic, and the software exhibited irritating characteristics

such as automated logout after a short time, requiring the user to re-login several times a day.

Some of this was related to the DWAN that most TDOs were using at work, but many of the

problems were also experienced on other network connections, such as DRDC`s Defence

Research Establishment Network (DRENet). Much of the content activity on the SABA site early

on was related to questions and comments about technical issues and discussion about possible

workarounds. A series of comments and posts revolved around the `chat` feature which was

expected to be functional but was not. Something like the IBM`s Sametime chat application was a

feature the TDOs clearly wanted in their workplace environment.

In comparison, the Facebook Group site was immediately and easily available with no

deployment or technical problems. Individuals, if not Facebook users already, simply had to

register and join the TDO Group through a request to the site administrator that was responded to

promptly. The interface was relatively easy to use, having been progressively developed and

improved on since the platform was launched several years earlier, and the features and functions

were varied, integrated, and working, regardless of the network (DWAN, DRENet, or other)

connected to.

The majority of users felt that TDOs are more connected as a community since becoming

members of a social networking site, although none of the SABA-only users felt this way.


Facebook TDO Group members said that they would be sorry if the site went down (this was not

true for most of the SABA users), and that they were comfortable saying they were on Facebook

at work, as were most SABA users. This latter point was gathered in the survey data but it is in

contrast to commentary at the interview sessions, where TDOs clearly acknowledged a stigma at

work to being on any social networking site, in spite of the activity being work-related.


3 Discussion

Over the course of a three year research program the usefulness, feasibility and impact of

providing virtual social networking to CF groups was examined using quantitative and qualitative

measures. Much of the work was based on collecting data from military groups and particularly

from a test group, the TDOs, who were selected for more in-depth analysis based on the dispersed

nature, the size of the community, and the work that they do. An enterprise-level social

networking platform, SABA People Cloud, was provided to the group and a number of aspects

pertaining to use and implementation were compared to a free and publicly available platform,

Facebook, that the test community had set up themselves and were using independently.

Variables of interest to the research included, level of activity and content on the sites as well as

the effect the social networking tool may have had on the ability of the users to build contact

networks and become more familiar with their counterparts, especially those who were not

co-located. Also of interest was understanding, from the users` perspective, viewpoints about

using the platforms provided to them, as well as their opinion on using social networking at work

in general. We were also interested in the functionality and usability of both these platforms –

what tasks and activities did this collaborative tool lend itself to, and whether or not it met the

needs of the community. Finally, we wanted to explore content to learn about the kinds of

material discussed and shared on the sites.

First and foremost it is important to point out that the TDO community, which was our primary

focus, had initiated use of virtual social networking, through Facebook, on their own, independent

of the DRDC research project. This fact in itself clearly points to a need within the community

and a desire amongst its members to improve the collaborative space in their work environment.

To fulfil the purpose the community chose the freely available Facebook which, based on our

data, proved to be a rich environment, where the community conversed about work-related topics

and where questions, answers, and information were shared freely and easily. Of interest to note

is that Facebook was one of the first recommendations put forward by DRDC as a platform for

the test group to use that would meet the needs of the community as well as the research program.

As reported, the SABA People Cloud platform met with many difficulties including technical

problems and usability issues. In the end the platform died a natural death, no doubt because the

problems and effort were more than they were worth. That is not to say that the SABA

technology is sub-standard when used in other environments. Within a receptive computer

network domain the platform is undoubtedly a sophisticated high-level collaborative toolset. In

this particular instance though, the firewalled network and the lack of functionality in the beta

version delivered (as compared to the version initially demonstrated) brought on-going challenges

both in implementation and overall use and adoption.

Introducing new technology into an existing workflow is a challenge at the best of times,

requiring a comprehensive requirements analysis and a well thought out, streamlined

implementation and integration process around a technology that is easy to use right from the

start. Today`s social media consumers have no time for problem-solving (neither literally or

figuratively) the technology that is supposed to be providing them benefit. For adoption to be

successful new technologies must provide the user with clear and immediate benefit over existing

practices. In this comparison study it appears that the Facebook TDO Group did just that, by


supporting the community through an open environment that fostered conversation and a sense of

community along with providing the necessary tools and platform for sharing.

A key criterion for the successful launch of new technology into an established community is

support of upper-level management, not only in verbal backing but in actual use of the

technology. We do not believe that the introduction of the enterprise social networking tool

suffered from lack of management support. That being said, explicit direction or promotion from

senior staff after the site was up and running may have been minimal. Whether this ultimately

affected the experience of using SABA is unlikely as the Facebook site became successful

without any particular push from management or senior staff. The Facebook Group was initiated

by TDOs and subsequently populated by TDOs and the data in fact show that senior officers were

members of the Facebook TDO Group10

, using the platform just like other members. No

hierarchical structure appeared to exist on the site, nor were management identified as site

promoters. The advocates of the Facebook TDO Group were, and continue to be, the TDO

members themselves.

The research also served to highlight security and information technology hurdles and the levels

of approval required when attempting to implement new software into a secure defence domain.

These were major stumbling blocks not only within the military, but as much in the DRDC

research establishment where the research team tried to test out and learn about social networking

platforms in order to increase their own knowledge of the technology. Our goal of learning about

different social networking applications and what they were capable of, as well as gathering an

understanding of how social networking might be used in a multi-organization project-oriented

setting, was severely curtailed by IT boundaries and restrictions.

The research conducted over a three year period showed that virtual social networking can be a

useful tool to members of the Canadian Forces. Most notable observations from the use of the

Facebook site were: that the social networking members were able to connect with others who

were not co-located; that members were able to find and share information with the appropriate

people as needed; and that a sense of community developed in this virtual environment. These

findings could be generalized to groups that are similar in composition and structure to the TDOs,

being distributed with a moderately paced workflow. Whether or not the tool would be as useful

in a high intensity operational environment cannot be determined without a dedicated

requirements analysis because the benefits provided by technology in one environment cannot be

assumed to transfer to others, and this is particularly true for social media technologies where

their effectiveness is environment and task specific. However, it is highly conceivable that such a

platform could be very beneficial in a high paced operational situation if an individual`s social

contact network was already established. Such a tool would not only be valuable for connecting

within one`s own community but also, and just as importantly, with other groups and

organizations. The social network framework forms an indispensable contact database and a

platform where information is broadly distributed amongst a widespread and diverse population,

thereby making visible all active members, information about them, and all content exchanged.

The complete package allows for quickly finding specific people or expertise and being able to

relay and share information with the right people at the right time.

10

A post of the Facebook TDO Group in November 2011 pointed out that a number of Lieutenant Colonels

were using the site.


4 References .....

Bingham, T., and Conner, M. (2010). The new social learning: A guide to transforming

organizations through social media [Kindle Edition]. Alexandria, VA: ASTD Press.

Granovetter, M. S. (1973). The strength of weak ties. The American Journal of Sociology, 78(6),

1360-1380.

Hansen, M. T. (1999). The search-transfer problem: The role of weak ties in sharing knowledge

across organization subunits. Administrative Science Quarterly, 44(1), 82-111.

Kwantes, P. (2011). A document visualization tool customized to explore DRDC reports.

DRDC Toronto TR 2011-131.

McAfee, A. P. (2009). Enterprise 2.0: New collaborative tools for your organization's toughest

challenges. Cambridge, MA: Harvard Business Press.

McConnell, B., Huba, J. (2006). The 1% Rule: Charting citizen participation. Retrieved

May 13, 2011, from: http://www.churchofthecustomer.com/blog/2006/05/charting_wiki_p.html

O'Reilly, T. (2005). What is web 2.0: Design patterns and business models for the next generation

of software. Retrieved July 9, 2011, from:

http://oreilly.com/pub/a/web2/archive/what-is-web-20.html.

Torenvliet, G. L., Euerby, A., Scott, S. D., and Histon, J. (2011). Investigating virtual social

networking in the context of military interoperability: Year 1 Report. Defence Research and

Development Canada – Atlantic CR 2010-308.

Torenvliet, G. L., and Leal, J. (2011). Investingating virtual social networking in the context of

military interoperability: Year 2 Report. Defence Research and Development Canada – Atlantic

CR 2011-155.

http://web.archive.org/web/20100511081141/http:/www.churchofthecustomer.com/blog/2006/05/charting_wiki_p.html

http://www.churchofthecustomer.com/blog/2006/05/charting_wiki_p.html

http://oreilly.com/pub/a/web2/archive/what-is-web-20.html


Annex A Technology Steward Introduction Letter

<Name>,

We are writing to request your help with an upcoming CF Learning Technologies initiative. You

may have heard that we are planning on deploying a new Learning Management System and

Learning Content Management, called Saba. As a part of this deployment, we are also planning

on making a related professional social networking platform, called Saba Live, available to the

CF TDO community. We’re hoping that this platform will help us to do a better job of working

together as a community, and that using it will:

Help each of us to maintain a larger network within our community and use that network

more effectively.

Make it easier for us to share our experiences and benefit from others’ experiences

through status updates and blog posts.

Increase our ability to find information and experts related to our jobs.

This new platform is a social platform, so for it to be useful it’s important that people actually use

it in their interactions with others. To this end, we would like to enlist your help in fostering good

use of this platform by asking you to serve as a Technology Steward. This sounds complicated,

but it isn’t. A Technology Steward is just someone who takes a special interest in using, and

championing the use of, technology for the sake of their community. At the most basic level, this

means acting as an early adopter and working with the tool to populate your profile, post your

status, and start following colleagues that you know or think might have something interesting to

say. As use of the tool matures, acting as a technology steward means encouraging your

colleagues to use the tool, thoughtfully assessing how it is being used, and encouraging more and

better use in whatever way you think is best.

We hope you’re able to accept this job, and that you will find it enjoyable and useful. If you do,

here’s what we’re expecting in more detail:

Being an early adopter. We plan to deploy the tool to a select pool of early adopters in late June,

and we would like you to work with and live in the tool as much as possible. We want to deploy

to the overall community in early September, and we’re hoping that if the tool looks lived-in, the

deployment will be more successful. During this period, you may find issues with the system, and

we’ll want to work with you to make sure that those issues are identified and resolved before

launching to the whole community.

Stewarding use. Once the tool has been deployed, we would like you to take ownership of it and

do what you can to help the TDO community’s experience with this tool to be productive. This

means continuing to use it in your everyday work, but also observing how your colleagues and

other TDOs are using it, and encouraging them (by example, and by solicitation) to make better

use of it where necessary.

Collaborating with other technology stewards. We hope to develop a healthy group of

technology stewards across the TDO community, and we would like you to collaborate with them

periodically to share best practices and experiences. Part of this collaboration will happen on the

social platform, but we’re also hoping to bring you all together for a number of discussion

sessions.

We expect that this deployment will provide some important benefits to the TDO community, and

we’re hoping to capture our experiences with the Saba Live platform for the benefit of the rest of

the CF. To help with this, a team of researchers led out of Defence Research and Development


Canada (DRDC) will be observing our use of the Saba Live platform. If you sign up as a

technology steward, you will also be a partner in their research. They are ready to help you in

your role as technology steward, and will want to hear from you about your experiences. This will

involve participating in a handful of interviews, and perhaps filling out a few questionnaires. The

DRDC researchers will also be watching how use of the Saba Live platform develops, and will

listen to and participate in the interactions that you tech stewards engage in together.

If you are interested or would like more information, please contact the undersigned.

Thanks for your time!


Annex B Community Introduction Email

<Name>,

We are writing to request your help with an upcoming CF Learning Technologies initiative

investigating the use of social media in the workplace. As a part of this study a professional social

networking platform, called SABA Social, is being made available to the CF TDO community.

As an indication of the importance of this project to the community one of the TDO champions of

this project is LCol Gilles Emond. We are hoping that this platform will help us to do a better job

of working together as a community, and that using it will:

Help each of us to build and maintain a larger network of contacts within our community and

enable us to use that network more effectively.

Make it easier for us to share our experiences and benefit from others’ experiences through status

updates and blog posts.

Increase our ability to find information and experts related to our jobs.

This SABA software is a social platform, so for it to be useful it is important that people actually

use it in their interactions with others. To this end, if you need help getting started or have

questions, Technology Stewards will be available to help you. The Technology Stewards are a

small group of TDOs that have had access to the site for the past month or so, and they will be

able to provide help, as needed, in using the tool. The Technology Stewards are: (provide a list).

We expect that this deployment will provide some important benefits to the TDO community, and

we’re hoping to capture our experiences with the SABA Social platform for the benefit of the rest

of the CF. To help with this, a team of researchers led out of Defence Research and Development

Canada (DRDC) will be observing our use of the SABA Social platform. Your involvement will

include filling out a questionnaire at the beginning of the project and one at the end. It may also

involve participating in a handful of interviews. The DRDC researchers will also be watching

how use of the SABA Social platform develops, and will listen to and participate in the

interactions that you engage in.

Please complete the consent form – required by DRDC for this study – and the questionnaire. On

completion you will be provided a link to the site.

Thanks for your time!


Annex C Post-SABA Survey

There are three versions of the same questionnaire based on the response to Question 1 (which of

the following tools have you used in a work context).

Final Virtual Social Networking Questionnaire

DRDC Consent Form

Protocol Number: L-740 Amendment 2 Research Project: Understanding requirements for virtual

social networking in the Canadian Forces Principal Investigator: Dr. Jacquelyn Crebolder,

DRDC Atlantic Co-Investigators: Tania Randall and Dr. Aren Hunter, DRDC Atlantic Research

Associates: Curtis Coates, CMC Esterline|Electronics; Dr. Jonathan Histon and Anson Ho,

University of Waterloo. I hereby volunteer to be a participant in the study ‘Understanding

requirements for virtual social networking in the Canadian Forces’. I may obtain information

about the research by contacting Jacqui Crebolder at 902-426-3100 x296, jacqui.crebolder@drdc-

rddc.gc.ca. My participation in this study involves participating in a short online questionnaire. I

understand that the research findings resulting from my participation may be used for publication

or commercialization purposes. I understand my identity will remain anonymous and my

participation confidential. I understand that I will not receive remuneration for my participation in

this research. I understand that by providing consent I have not waived any legal rights I may

have as a result of any harm to me occasioned by my participation in this research project beyond

all risks I have assumed. My name is:

None

and I hereby volunteer to participate in this study and I have read and understood the above

information.

Yes

No

Page 1 of 3

Please indicate which of the following tools you have used in a work context:

Both Facebook and SABA

Only Facebook

Only SABA

Neither Facebook or SABA for work purposes

Page 2 of 3

Do you have internet access on your computer at work?

Yes

No

Do you access the Facebook TDO Group or the SABA site from a mobile device at work?

Yes

No


Do you feel the TDO community is more closely connected in general since using the

Facebook TDO Group and/or SABA?

Yes

No

How were you introduced to the SABA site?

I got a DRDC email about it asking me to join

A colleague told me about it

I saw reference to SABA on Facebook

Other ______________________

How were you introduced to the Facebook TDO Group?

I was using Facebook already and saw postings about the TDO Group


I saw reference to the Facebook TDO Group on SABA

Other ______________________

In what order did you join SABA and the TDO Facebook Group?

TDO Facebook Group first

SABA first

Both at about the same time

Were you already using Facebook before joining the TDO Facebook Group?

Yes

No

Have you migrated from one platform to the other?

Yes – from SABA to Facebook

Yes – from Facebook to SABA

No, I have not migrated from one to the other

If you have migrated from one platform to the other, what was the primary factor(s) that

made you move? (select all that apply)

privacy issues

more people to connect with

easier/more intuitive interface

more functionality – such as, ______________________

easier to access


speed of connectivity

less technical difficulties

Other, please specify: ______________________

How often do you check the following (for work purposes)? Please select the closest answer.

less than 1x per

week

1x per

week

a couple times per

week

1x per

day

more than 1x per

day

Facebook

SABA

Did you personalize your SABA home page?

Yes

No

What features have you found useful in each social networking platform for the purpose of

supporting your work? (select all that apply)

Facebook SABA

photos / media sharing

news feed

event scheduler

user profiles

chat

private messaging

notifications

groups

discussion/comment threads

presence awareness (knowing who's online)

Are there features missing from Facebook that could help you do your work?

None


None

Which, if any, apply to your usage of Facebook? Through this tool:

I have requested information from the TDO community at large

I have responded to another member's post/request for information


I have obtained information relevant to my work duties without actively seeking it (e.g., by

reading a news feed)

Which, if any, apply to your usage of SABA? Through this tool:





Have you changed the way you use Facebook in your personal life as a result of your

membership in the TDO Facebook group? Check all that apply. I have:

increased the privacy level of my account

chosen my profile pictures more cautiously

written my status updates more cautiously

deleted something that friends wrote on my wall

restricted access to specific content (e.g., albums, tagged photos, etc)


Rate your level of agreement with the following statements with respect to the TDO

Facebook Group:

Strongly

Disagree

Disagree Neutral Agree Strongly

Agree

Facebook makes my everyday activities

more efficient

I feel out of touch with my colleagues

when I haven't logged onto the TDO

Facebook group in a while

I am comfortable telling colleagues in

the CF I’m on Facebook

Facebook has become part of my daily

work routine

I feel more connected to the TDO

community than I would have without

the TDO Facebook group

I would be disappointed if the TDO

Facebook group shut down

Rate your level of agreement with the following statements with respect to SABA:

Strongly

Disagree


Agree

SABA makes my everyday activities

more efficient


I feel out of touch with my colleagues

when I haven't logged onto SABA in a

while

I am comfortable telling colleagues in

the CF I’m on SABA

SABA has become part of my daily

work routine


community than I would have without

SABA

I will be disappointed when the SABA

TDO site shuts down

Do you think a social networking tool could help you get to know a colleague you met

online?

Yes

No

How important are the following uses of social networking tools to you?

Not

important at

all

Not

important

Neither

important or not

important

Important Very

Important

Finding new people.

Reconnecting with

people I've lost contact

with.

Seeing what people

have put as their

'status'.

The continuous updates

and dialogue.

If you have any general comments about social networking in the workplace, or about


None


Page 2 of 3


Yes

No

Do you access the Facebook TDO Group from a mobile device at work?

Yes

No

How were you introduced to the Facebook TDO Group?

I was using Facebook already and saw postings about the TDO Group


Other ______________________

Do you feel the TDO community is more closely connected in general since using Facebook?

Yes

No

Were you already using Facebook before joining the Facebook TDO Group?

Yes

No

Which of the following best describes your reason for not being on SABA?

I don't know what SABA is

I heard of SABA but chose not to join

I joined SABA but didn't find it useful

I joined SABA but found it difficult to use

I didn't want to invest time into a product that was only available temporarily


How often do you check Facebook for work purposes? Please select the closest answer.

less than 1x per week

1x per week

a couple times per week

1x per day

more than 1x per day

What features have you found useful in Facebook for the purpose of supporting your work?

(select all that apply)



news feed

event scheduler

user profiles

chat

private messaging

notifications

groups


presence awareness (knowing who's online)

Are there features missing from Facebook that could help you do your work?

None

Which, if any, apply to your usage of Facebook? Through this tool:





Have you changed the way you use Facebook in your personal life as a result of your

membership in the TDO Facebook group? Check all that apply. I have:

increased the privacy level of my account

chosen my profile pictures more cautiously

written my status updates more cautiously

deleted something that friends wrote on my wall

restricted access to specific content (e.g., albums, tagged photos, etc)


Rate your level of agreement with the following statements:

Strongly

Disagree

Disagree Neither

Agree or

Disagree

Agree Strongly

Agree

Facebook makes my everyday

activities more efficient

I feel out of touch with my

colleagues when I haven't logged

onto the TDO Facebook group in a

while


I am comfortable telling colleagues

in the CF I’m on Facebook

Facebook has become part of my

daily work routine


community than I would have

without the TDO Facebook group

I would be disappointed if the TDO

Facebook group shut down


online?

Yes

No

How important are the following uses of Facebook to you?

Not

important at

all

Not

important

Neither

important or not

important

Important Very

Important

Finding new people.

Reconnecting with

people I've lost contact

with.

Seeing what people

have put as their

'status'.


and dialogue.



None


Page 2 of 3


Yes

No

Do you access the SABA site from a mobile device at work?

Yes

No

How were you introduced to the SABA TDO site?

I got a DRDC email asking me to join?

A colleague told me about it.

Other ______________________

Do you feel the TDO community is more closely connected in general since using SABA?

Yes

No

Which of the following best describes your reason for not being part of the Facebook TDO

Group?

I don't use Facebook at all

I didn't know this group existed

I am aware of the group but chose not to join

I joined this group but didn't find it useful

I am uncomfortable using Facebook in a work environment


Do you use Facebook for purposes other than work (i.e., in your personal life)?

Yes

No

How often did you check the SABA site? Please select the closest answer.

less than 1x per week

1x per week

a couple times per week

1x per day

more than 1x per day

Did you personalize your SABA home page?

Yes


No

What features did you find useful in SABA for the purpose of supporting your work? (select

all that apply)


news feed

event scheduler

user profiles

private messaging

notifications

groups



None

Which, if any, apply to your usage of SABA? Through this tool:





Rate your level of agreement with the following statements:

Strongly

Disagree

Disagree Neither Agree

or Disagree

Agree Strongly

Agree

SABA makes my everyday

activities more efficient

I feel out of touch with my

colleagues when I haven't logged

onto SABA in a while

I am comfortable telling

colleagues in the CF I’m on

SABA

SABA has become part of my

daily work routine


community than I would have

without SABA

I will be disappointed when the

SABA TDO site shuts down



online?

Yes

No

How important are the following uses of SABA to you?

Not

important at

all

Not

important

Neither

important or not

important

Important Very

Important

Finding new people.

Reconnecting with

people I’ve lost contact

with.

Seeing what people

have as their 'status'.


and dialogue.



None

Page 2 of 3

Please rate your agreement with the following statements. I do not use Facebook or SABA

for work purposes because:

Strongly

Disagree


Agree

I do not perceive any benefit to being

'online' with my colleagues

I am too busy at work to use a social

networking tool

I am concerned about the stigma of

being on Facebook or a similar social

platform

I am concerned about security issues

Are there additional factors that prevented your usage of these tools? If yes, please explain:

None


Do you use social networking sites (Facebook, LinkedIn, MySpace for example) outside of

work?

Yes

No

Page 3 of 3

The following survey is a continuation of information captured in a similar survey distributed in

September 2011; therefore it is possible that not all TDOs are listed. Even if you did not answer

the initial survey in the fall, we are interested in your response to this final question set.

We are interested in how well you know people at work. Please indicate how well you know each

of the people in the list below by clicking in the appropriate column, where:(0) indicates that you

do not know this person at all(1) indicates that you know of this person, but know nothing about

their skills, experience, or expertise(2) indicates that you somewhat know about this person’s

skills, experience, or expertise(3) indicates that you know a lot about this person’s skills,

experience, or expertise You should base your ratings on how well you know these people at the

current time. Please refrain from looking people up (e.g., on a central database or a web search)

while you complete this questionnaire. (If your name appears on the list, please leave that answer

blank).

List of names 0 1 2 3

XXX XXXX


Annex D TDO Symposium Interview

The syndicate work conducted as part of a TDO Annual Conference consisted of two parts – i)

structured interviews and ii) an on-line experiment using the SABA People Cloud platform.

Interview Questions

The questions for the structured interview follow:

Have you been using either the SABA or TDO Facebook site?

a) Which did you use first?

b) What were you hoping to get from the sites?

c) Why one over the other if applicable?

Have you found most useful in a social networking platform for the purpose of supporting your

work duties?

a) photos / media sharing

b) news feed

c) being able to schedule events

d) others

What features are useful in a social networking platform for the purpose of supporting your work

relationships? (select all that apply)

a) photos / media sharing

b) news feed

c) being able to schedule events

d) others

What functions/features do you find most useful in Facebook?

What functions/features do you find lacking in Facebook? Are there features from SABA that you

find yourself missing?

What are the most important features that help you build trust in a colleague known only through

the Facebook environment?

Rate your level of agreement with the following statements

Facebook makes my everyday activities more efficient

1(Strongly disagree) 2 3 4 5(Strongly agree)

I am proud to tell colleagues in the CF I’m on Facebook


Facebook has become part of my daily routine



I feel out of touch with my colleagues when I haven’t logged onto the CDA Facebook group for a

while


I would be sorry if the CDA Facebook group shut down


How important are these factors to you for a social network environment at work?

Privacy (having control over who can see your profile, content you post, etc.)

1(not important at all) 2 3 4 5(very important)

Customizable controls (having the ability to express convey your own views)


Simplicity (having a simple user interface, perhaps sacrificing some customization)


Rich content experience (the site provides a range of media (text, images, video, etc.)


Contacts/peers


How important are the following uses of Facebook to you personally?

Chatting to people I otherwise would have lost contact with


Virtual people-watching.


Reconnecting with people I’ve lost contact with


Finding people you haven't seen for a while


Seeing what people have put as their 'status'





Interview Notes

The following are structured interview notes transcribed directly from the syndicate proceedings.

Day 1 –December 8th 2011

Syndicate Participants and profiles:

Participant Org Social Media Usage Profile

1 CFNES Experience with Facebook, Twitter, SABA

Daily use both professional and personal

Find VSN useful

Uses SABA to follow links

Accesses VSN through smartphone

2 CFTDC Personal use of Facebook, Twitter, and Linkedin – does not use SABA

Uses approximately every 3 days

Visits TDO Facebook group about once a week – does not post

Is aware of who is posting the most

3 CFTDC Personal use of Facebook, Twitter, and SABA

Uses daily – not SABA

Does not post to TDO Facebook group

Feels there is a stigma to viewing social network sites at work.

Uses a smartphone

4 ADM MAT Early digital adopter

Personal use of Facebook and IRC for post graduate work at Royal

Roads

Does not post to TDO Facebook group

Feels there is a stigma to viewing social network sites at work.

Uses a smartphone

5 Uses SharePoint to manage project data

6 ADM MAT Facebook user for 5 years; SharePoint for 2 years; wiki, skype, go

to meeting, SABA – just signed up, Lotus notes

Understands there is a stigma to using Facebook at work

7 DGPR –

NESOPS

Facebook usage for a couple of years, both personal and the TDO

group

Just signed into SABA – not a user

Uses Wikis and the TDO community of practice (Group Owner)

8 Public

Affairs

School

Power user of Facebook since 2005, uses it to keep in touch with

colleagues

Uses TDO Facebook group – posts information

VSN at Athabasca University

SABA active user

Linkedin, Twitter, SharePoint

9 DGPR No Facebook or virtual social network usage

10 CDA –

R&D

Facebook user since 2007, personal usage and in the TDO group –

not an active participant

Blackboarding

SABA

Chat application


Questions and answers:

Which VSN site, Facebook or SABA did you use first? Why? How do you use the sites?

- Just joined Facebook a week ago – does not use SABA

- Driven to Facebook by email updates

- Interested in what is going on

- General makes a good review of the wall once a week

- TDO Facebook works better than SABA

- Facebook first- the TDO site is good for professional discussions, however it would be

beneficial to keep it inside the DWAN firewall

- Facebook is far more user friendly than SABA

What features do you like with respect to work and the online communities?

- Directed to open SharePoint daily – work news updates and announcements are posted

- Schedules on SharePoint

- Blogs on SharePoint

- Email is still a priority

- Sites need better categories for discussions and threaded discussions

- Current workload is a deterrent to using the sites at work (KLS)

- The technology needs a history feature

- Needs infrastructure to support it

- Needs cultural acceptance

- Use of chat once a personal relationship has been developed

- Linkedin type data that shows professional history

- Experience capturing area – University degrees etc.

- List of key positions / jobs held

- Linkedin type information

- Audio or video feed – immediate feedback

- Skype

- Facebook type chat feature

- Instant messenger

- Threaded discussions

- Social presence, something that makes the work enjoyable

- Chat

- Email alerts

- Asynchronous communications

- There is a feeling that a Facebook post needs an immediate response

What functions/features do you find most useful in virtual social sites to support work

relationships?

- Maintaining acquaintances made face to face

- Keeping up some regular contact – annual/monthly/weekly

- It does take work to keep online relationships alive

- Knowing someone’s specialty, their education

- Currently there is no critical mass in the TDO community so some method to achieve

increased use

- Site would need the convenience of Facebook – to go from one group to another in the

same app


What functions/features do you find most useful in Facebook?

- Being able to hide a conversation

- Filter out what you don’t want to see

- Email alerts

- Personalization

- News feeds

- Tidy interface

- Links

- Knowing the data is persistent

- User friendly – loads quickly

- Group forming

- Event announcement

What features / functions do you find lacking in Facebook?

- Video capability

- Mandatory field such as year in job, work experience, education, specialty, previous

employment

- File sharing and file management such as folders, similar to SharePoint

How many people, professional colleagues or personal, do you know only through Facebook or

SABA?

- Distant relatives

- Learned about an early Facebook user through the TDO group

- TDOs with the branch through the site

- The following discussions were about the Technology Steward concept.

- Technology Steward is already a secondary job for many TDOs.

- Formal coursing would not be required.

- There has to be an interest in the digital community before it is embraced.

- The community must have a draw to get participation.

- A LCdr is effectively the Technology Steward for the Facebook group.

- Technology Stewards are change agents – TDO already do this. They lead by example.

- Need to have expertise similar to e-learning – TDOs do this.

- The schools will pull the resource to do something else if they are posted as a Technology

Steward.

- Not a primary job.

- Will have to be someone that wants to do the job.

- Difficult to determine if the technology needs a steward.

- Tech Steward would need a focus – would need to be named the champion – primary job

- Tutorials for the technology would be required.

- There is always an expert in the unit – whether they are willing or able is another issue.

Other points

- Feel no need to use a social media as he is on a computer all day.

- Sites must avoid too many changes – a static interface is important.

- A work related site must be quick and must be intuitive – no time to learn yet another app.

- Online chat may take away from the usefulness or purpose of the app.


Group A Interview: 4 participants – SABA/Facebook usage / experience.

- has Facebook/Twitter/SABA – Facebook for TDO Group and use for personal – is new

and finds the Facebook Group very helpful – links, docs, etc.

- SABA worked well at home but not on DWAN.

Not on Facebook at work – doesn’t like the ‘stigma’ – trying to promote use but not

overuse – where do you draw the line. Is probably task specific.

- Facebook – TDO Group – on 3 x week – skims through topics. Joined Twitter because of

one interaction but not on there. On LinkedIn – but doesn’t check. Got SABA email to

log in but is too busy teaching – e.g., if email is not clearly relevant to the course then has

no time to check.

- has Facebook checks daily – and is member of TDO Group on Facebook and uses it for

personal as well – l has Twitter – has iphone. Doesn’t use Facebook at work because of

‘stigma’.

- Created SABA account – not logged in thought because of problems with SABA and

other things to do.

- ADM(MAT) major projects – early digital immigrant – using various systems. Has a work

BB – so not on Twitter. On SABA but not much used. Facebook and TDO group – not at

work because of ‘stigma’. Job keeps away from desk so uses phone a lot rather than on

line. Can catch people that way.

Facebook TDO Group – do you go on for something specific or just go on the check?

- Get email when someone makes a change – if it’s interesting will look further.

- Question – is SABA used for business, so people aren’t on Facebook?

- None that they know of.

- Relationships are built better on the phone or face to face.

- If links are made on Facebook there’s Chat – as long as you know the person.

What are the best features to support duties?

- Sharepoint is mandated when you log on – which means everyone is on the same page at

a time, which is good. But problem is it’s not used very often and needs to be refreshed to

see new stuff.

- A hash tag on Twitter might be good – that way we’d all be talking about a specific topic.

- There is an age difference in use of these tools – fear factor for older people.

- Things to consider when implementing technology:

o Is the infrastructure there?

o Is there cultural acceptance?

- SM could allow for informal learning – would prefer chat room on line rather than

random postings so that everyone would be talking about the same topic.

- Forum chat has advantages and synchronous chat does too – depends on the rate of work

and the need.

- Also Skype-like feature would be nice.


- Audio-Visual is important – especially for immediate feedback

- Those on Facebook use the Chat feature.

- Issue is to know the TDOs area of expertise – so names are in database.

- LinkedIn type platform is good for this.

What is the most useful feature?

- ‘Hide’ – shrinking down what’s not relevant.

- Only checks to get email – doesn’t read it all. Depends if it’s

- New in the trade so can check stuff out in case it’s relevant, now or later.

Is there info shared on Facebook that brings you closer?

- Distant relative introduced through 3rd

party

- Yes, has introduced himself in person because saw someone on Facebook.

Many of those interviewed said that they are not on SABA because of trouble with the platform

and the timing of the Facebook TDO Group (note that the group was formed a couple of years

earlier but had few members until more recently).

Questions on Technology Stewardship

If assigned as a tech steward would it be a primary or secondary job?

- We are doing that already (KHAN Academy]) – there are a few TDOs exposing

tools/technologies (the TDO fills that role because of the kind of job they have already

which is to facilitate learning).

Should there be formal training?

- But there has to be interest in the digital community. Show by example – TDOs already do

that.

- Change is driven by the needs of the group.

- But formal doesn’t work – fades.

- Formal Tech Steward – given to someone and it has nothing to do with their training –

higher ranks would say you should be using the person for something better.

- Has to be passionate person – not told.

Group B Interview: 5 participants – SABA/Facebook usage / experience

- Facebook 5 years – for personal use, not in TDO Group (and not going on because things

can be discussed on there that shouldn’t be), Sharepoint, Wikis, COPS (DND forum

website). SABA – just signed up

- [with reference to openness of Facebook Group] – but is accessed to information

- Access to information would be through proper channels.

- Some pilot sites (404) are ok but it’s the trace – but where‘s the line

- Used Facebook 2 yrs – member of TDO Group, is on SABA and checks sometimes. Wiki

use – group involved has one (COPS).


- Facebook 6 years – TDO Group – on SABA but no action so not inspired. But Facebook is

fabulous – her work place is just her as TDO – has posted on Facebook and got loads of

responses and can get to know colleagues because sees their posts.

- New and alone and can reach out through Facebook.

- Is on Twitter – follows but doesn’t post. Is on LinkedIn and Sharepoint

- Uses Facebook at work for TDO Group but is cautious. It’s convenient though.

- Not on Facebook or Sharepoint. Not on Facebook or anything because doesn’t need for

work and on computer all day so don’t like to be on after work.

- Facebook 4 years – early member of TDO Group – but inactive there because uses

Facebook for private, not work. Uses another app for long distance chat so no need for

chat on Facebook.

- Is on SABA.

What features are needed to support your work duties?

- A platform for dynamic discussion is needed

- And should be a threaded discussion – so that comments are linked and don’t get lost

- Who’s on line (social presence) + Chat on line

- What happened to good old email?

- I used to like instant manager

- I like to phone people

- Same

- Email vs Social networking – you always have to have your eye on the SN – or you might

miss something.

- Hierarchy doesn’t understand how younger people use these tools.

- Likes putting names to face.

Building – maintaining relationships at work….

- Be good to see specialities and work experience – as on LinkedIn but there you don’t get

that community interaction experience – would need all TDOs to be on there.

What about a Group on LinkedIn?

- Being able to form a Group is important. It’s easy on Facebook – but maybe need more

control on who can join. And needs more profile information added.

- Chat is important.

- Profile – with job, specialites, previous employment

- File sharing

- Has to be accessible on DWAN and on Internet.

Do you have friends on Facebook that you only know on Facebook.

- Yes – and you feel like you know them – because of status updates.

Should Technology Steward be primary or secondary role?

- What if Tech Steward gets posted?

- Too many tools out there and we don’t have a champion for any. Have enough trouble

getting AFILE and DLN up.

- Need policy in place first


- Has to come from the top.

Would a wiki be useful?

- Yes – but there are so many – difficult to know who’s on where.

- Existing platforms are COP and Blackboard.

Day 2 – December 9th 2011

Day two was a round table with TDOs summing up their views and opinions on social media in

the work place. The conversation points are not attributed to any particular individuals but are a

general summary of the comments provided by the group.

- Overall the TDOs are familiar with virtual social networking and Web 2.0 tools

- TDOs who self-designate as early adopters embrace the idea of social media at work

- Facebook was embraced by the majority

- In order to conduct business the following features are required:

- Email prompts (email is the primary online communication tool)

- Threaded discussions

- Subject search

- Persistent data

- Folder structure

- Ability to open and keep documents open

- In order to enhance business relationships the following features are required:

- Professional chat – with presence indication (who is online0

- User C.V. with experience, degrees, past employment

- Audio and video feeds

- The largest issue is the stigma of the usage of social media at work.

With respect to Technology Stewards:

- Shepherding and site maintenance is seen as a large effort

- Must be self-identifying

- TDOS are already change agents therefore would be suitable candidates as Technology

Stewards

- Must be able to act as moderators for discussions (threads)

What can TDO’s take away from the use of Virtual Social Networking sites:

- Sharing

- Community

- Use of VSN to capture knowledge as a large personnel turnover is coming

- Make VSN open to retired TDO’s and to personnel outside the TDO MOSID

- Need a single repository for data

- Need to create the reactive Community of Practice – expand to allies an retirees

- Need to structure interaction, a set time when an online forum will meet

- CDA or the Training Development Centre in Borden must show leadership

- Need a dedicated worker / champion / Technology Steward

- Need to force change

- Transfer discussions to wikis once a discussion is done


Annex E SABA Platform “How-To”


Annex F Facebook data extraction and preparation

Data extraction for Facebook was a multi-stage process, that included i) detailing a method of

extraction; ii) extracting the data using Facebook API; iii) and developing a Microsoft Excel

macro. The data were then further formatted for use in the HanDles program.

Determining a method of extracting Facebook data

As a first step the best method for extracting the posts and comments from the ‘TDO Network –

Training Development Officer Facebook Group’ was identified. Three methods were explored:

Direct copy and paste from the Facebook page;

Copy and paste each comment individually; and

Run a primary script on the Facebook API.

Methods 1 and 2 posed a problem with regards to formatting and extracting exact dates. Both

methods appeared to be more time consuming than Method 3 because of the manual extraction

aspect. Method 3 required a deeper understanding of the Facebook API in order to effectively

extract the data but it seemed to be the most effective of the three methods and ultimately was

selected as the best way forward.

Data extraction using Facebook API

In order to extract data using the Facebook API, a series of steps had to be performed to gain

access to the postings and comments. Through membership in the Facebook TDO Group access

was gained to the website:

https://developers.facebook.com/tools/explorer?method=GET&path=505782733, which was the

Graph API Explorer webpage. The following procedure was then conducted:

- “Get Access Token” was selected;

- The desired user groups were selected;

- “Get Access Token” selected a second time.

On the resulting page, under “Connections”:

- “Groups” was selected;

- The “TDO Network – Training Development Officer” Group was selected;

- “Feed” was selected.

This procedure resulted in a list of all postings and comments to date made to the TDO Group on

Facebook. Figure 7 is an excerpt of the output of a basic post.

https://developers.facebook.com/tools/explorer?method=GET&path=505782733


{

"id": "[Removed for anonymity]",

"from": {

"name": "[Removed for anonymity]",

"id": "[Removed for anonymity]"

},

"message": "You can send it to me at my DWAN address.",

"created_time": "2012-03-22T11:01:32+0000"

}

Figure 7: Basic post output excerpt.

4.1.1.1.1.1 Phase 3: Excel Macro (Formatting)

Excel Macro (Formatting)

The HanDles software program can read Microsoft Excel files (*.xml), so the data for the

Facebook TDO Group at this point was manually copied and pasted into an Excel spreadsheet. A

Microsoft Excel macro was developed to remove the unnecessary data fields from the Facebook

API output, leaving only the relevant fields described in Table 7. The macro deleted all lines from

the Facebook API that did not contain the relevant objects' names. For the Excel Macro refer to

Figure 8.

Table 7: Description of headings.

Heading Description

Title Generated using the combination of post number and comment number.

For example: Post number 10 comment 3 will have a title of “10_3”

Author Author of the post

Body Content of the post

Date Date of the post in the format of: YYYYMMDDHHMMSS. For example:

A post on 1:51:06pm on the 25 of December 2010 will have a value of

"20101225135106"

Website Website link if a link was posted

Likes Number of likes

Pictures A number 1 represents a user posting a picture, while a 0 represents no

photo was posted

Mobile A number 1 represents a user posting to the site from a mobile device,

while a 0 represents a user posting from a computer

Location Location of post


Heading Description

Post/Comment Contains the value of a post or comment. A high level or parent message

is known as a post, while its children are known as comments

Post Number Value of the post number

Comment Number Value of the comment number

Response to If a message is a post, the value will be its own title. If a message is a

comment the value will be to the previous post’s title


Sub ExtractingData()

Dim Rng As Range

Dim X As Long

Dim i As Integer

Dim n As Integer

Dim row As Integer

row = 1

Set Rng = Range("A1:A" & Range("A65536").End(xlUp).row)

'Facebook API code is placed on Column A

'Search for all relevant post and do not delete

'Delete all irrelevant post

For X = Rng.Rows.Count To 1 Step -1

If InStr(Rng.Cells(X, 1).Value, "likes") < 0 Or InStr(Rng.Cells(X, 1).Value, "message") < 0

Or InStr(Rng.Cells(X, 1).Value, "count") < 0 Then

Rng.Cells(X, 1).EntireRow.Delete

End If

If InStr(1, Rng.Cells(X, 1).Value, "Like") > 0 Then

If InStr(1, Rng.Cells(X + 3, 1).Value, "name") > 0 Then

Rng.Cells(X + 2, 1).EntireRow.Delete

End If


End If

Next X

'Delete all Facebook links

For X = Rng.Rows.Count To 1 Step -1

If InStr(Rng.Cells(X, 1).Value, "facebook.com") > 0 Then


End If

Next X

End Sub

Figure 8: Microsoft Office Excel macro.


A second macro was used to format and sort all the objects into a 13 column table as shown in

Figure 9.

Figure 9: Organization of Facebook data.

Tit

le

Auth

or

Bo

dy

Dat

e

Websi

te

Lik

es

Pictu

re

Mobi

le

Locati

on

Post/

Comm

ent

Po

st

#

Comm

ent #

Respon

se To

All other formatting was performed manually, such as highlighting of all posts.

Data preparation for analysis

Once in Excel the SABA and Facebook data were further formatted for use in the HanDles

software program. The process is described in the following subsections.

Further refinement removed extra columns, such as Mobile or Pictures that would not be used in

this data analysis. Also the format did not allow for posts and comments to be linked together and

as a result it was decided to use a Lua script to parse and reorganize the data. Three different file

formats were created for this purpose.

The files were first created as separate files, each containing the following, shown in Figure 10.

<?xml version="1.0" encoding="UTF-8"?>

<article>

<title> Post or comment title </title>

<author> Author name </author>

<fulltext> Text of the post – including a website address if one was posted </fulltext>

<Date> 200711316323 </Date>

<Location> Location, if available </Location>

<Handles> Handles include each post’s own title, and if a comment, the original post’s

title. It also includes Likes, Mobile and Picture if the post was liked, posted from a mobile phone,

or included a picture; Facebook post also had the handle Facebook, and SABA posts, SABA

</Handles>

</article>

Figure 10: Separate HanDLes files.

A drawback to creating each post or comment as separate *.xml files emerged when attempting to

upload the files to HanDles. It was thought that all of the *.xml files could be compressed into a

*.zip file and uploaded in a single step, however, HanDles could only upload file type Portable

Document Format (PDF) files as part of a *.zip file. As a solution a second format was created


where all the posts and comments were in one *.xml file, but were defined as different documents

(see Figure 11).

<?xml version="1.0" encoding="UTF-8"?>

<corpus>

<document>




<Date> 200711316323 </Date>




or included a picture</Handles>

</document>

<document>




<Date> 200711316323 </Date>




or included a picture</Handles>

</document>

<document>

…

</document>

</corpus>

Figure 11: Separate HanDles documents.

HanDles recognized this file format as containing separate documents11

, and the approach

satisfied the task of uploading the data, but the link between posts and comments was not strong

enough for them to appear related when doing HanDles searches.

11

A document in this case is defined as an item (a post or a comment).


The final revision to the format consisted of a single *.xml file, where comments were included in

the text of the post as shown in Figure 12. This format lost some data, such as the date and title of

the comments, but it allowed for a better link between posts and comments.

<document>

<title> Title of top post </title>

<author> Author of top post </author>

<fulltext> Full text of top post

Comment 1: Full text of first comment

Comment 2: Full text of second comment

…

Comment n: Full text of lastcomment

</fulltext>

<Date> Date of top post </Date>

<Handles> Has the same handles as previous formats, plus all the names of the

comment authors, formatted as follows: First_Last </Handles>

</document>

Figure 12: HanDles comments linked together.


List of symbols/abbreviations/acronyms/initialisms

ABCA America, Britain, Canada, Australia

ACIMS Army Collaborative Information System

AFILE Airforce Integrated Information Learning Environment

CDA Canadian Defence Academy

CF Canadian Forces

CFNOS Canadian Forces Naval Operations School

CSS Center for Security Science

DND Department of National Defence

DRDC Defence Research and Development Canada

DRENet Defence Research Establishment Network

DWAN Defence Wide Area Network

GoC Government of Canada

GPNET General Purpose Network

HMCS Her Majesty's Canadian Ship

HRMFRC Halifax and Region Military Family Resource Center

HRSD Human Resources Skills Development

IT Information Technology

LFAA Land Force Atlantic Area

Q&A Question and Answer

QSWB Qualification Standard Writing Board

PsyOps Psychological Operations

R&D Research and Development

RCMP Royal Canadian Mounted Police

RJOC Regional Joint Operations Center

TDO Training Development Officer

TPWB Training Plan Writing Board

VTC Video Teleconferencing

DOCUMENT CONTROL DATA (Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Designated)

1. ORIGINATOR (The name and address of the organization preparing the document.

Organizations for whom the document was prepared, e.g. Centre sponsoring a

contractor's report, or tasking agency, are entered in section 8.)

Defence R&D Canada – Atlantic, 9 Grove Street, P.O. Box 1012, Dartmouth, Nova Scotia, B2Y 3Z7

2a. SECURITY MARKING (Overall security marking of the document including

special supplemental markings if applicable.)

UNCLASSIFIED

2b. CONTROLLED GOODS

(NON-CONTROLLED GOODS) DMC A REVIEW: GCEC JUNE 2010

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U)

in parentheses after the title.)

Investigating Virtual Social Networking in the Context of Military Interoperability: Year 3 Report and Overall Three Year Summary

4. AUTHORS (last name, followed by initials – ranks, titles, etc. not to be used)

Crebolder, J.; Randall, T.; Hunter, A.; Coates, C.; Torenvliet, G.

5. DATE OF PUBLICATION (Month and year of publication of document.)

August 2013

6a. NO. OF PAGES

(Total containing information,

including Annexes, Appendices,

etc.)

100

6b. NO. OF REFS

(Total cited in document.)

9

7. DESCRIPTIVE NOTES (The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report,

e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)

Technical Report

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.)

Defence R&D Canada – Atlantic 9 Grove Street P.O. Box 1012 Dartmouth, Nova Scotia B2Y 3Z7

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research

and development project or grant number under which the document

was written. Please specify whether project or grant.)

14dl

9b. CONTRACT NO. (If appropriate, the applicable number under

which the document was written.)

10a. ORIGINATOR'S DOCUMENT NUMBER (The official document

number by which the document is identified by the originating

activity. This number must be unique to this document.)


10b. OTHER DOCUMENT NO(s). (Any other numbers which may be

assigned this document either by the originator or by the sponsor.)

11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.)

Unlimited

12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the

Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement

audience may be selected.))

Unlimited

13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable

that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification

of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include

here abstracts in both official languages unless the text is bilingual.)

Over a three-year research program, Defence Research and Development Canada – Atlantic

investigated social networking technology with the aim of understanding this relatively new

collaboration tool. The technology was assessed for potential benefits and issues within the

context of information sharing, finding expertise, and team building for the Canadian Forces

(CF). A requirements analysis was conducted on a number of military groups in Year 1, and an

in-depth analysis was administered to a select group, the Training Development Officers (TDO),

in Year 2. In Year 3 the TDO community was provided with an enterprise-level social

networking platform (SABA People Cloud) in order to examine how the social networking

technology was used, what features were important, and what issues, if any, existed. A number

of data collection methods were employed including, questionnaire, experimentation, workshop,

observation, and visual analytics. The results were compared to similar data collected from the

publicly available Facebook social networking site (Facebook.com), where the TDOs had

created their own Group and where they had become increasingly active. This report

summarizes Years 1 and 2 of the program, and describes in detail the final year, Year 3. The

work overall emphasizes the importance of ease of use and accessibility of social networking

tools and suggests that this technology can be a viable and useful support to the CF.

---------------------------------------------------------------------------------------------------------------

Par un programme de recherche de trois ans, Recherche et développement pour la défense

Canada – Atlantique a étudié les nouvelles technologies que constituent les réseaux sociaux afin

de mieux comprendre ces nouveaux outils de collaboration. Nous avons évalué les avantages et

inconvénients potentiels du réseautage social dans le contexte du partage d’information, du

repérage d’experts et la promotion du travail d’équipe au sein des Forces canadiennes (FC). La

première année, nous avons effectué une analyse des besoins de plusieurs groupes de militaires

et, l’année suivante, une analyse approfondie a été faite sur l’un de ces groupes : les officiers du

développement de l’instruction (ODI). La troisième année, la collectivité des CDI a pu utiliser

une plateforme de réseautage social d’entreprise (SABA People Cloud); et nous avons pu

observer comment cette technologie a été utilisée, quelles fonctions ont été les plus importantes

et le cas échéant quels problèmes se sont manifestés. Nous avons pour cela fait appel à plusieurs

méthodes de collecte de données, notamment des questionnaires, des expériences, des ateliers et

des observations ainsi que l’analytique visuelle. Nous avons comparé les résultats obtenus à des

données semblables recueillies sur le site public de réseautage social Facebook (facebook.com),

car les CDI y avaient créé leur propre groupe et y étaient de plus en plus actifs. Le rapport

résume les deux premières années du programme d’étude, et décrit la troisième en détail. Ces

travaux soulignent en règle générale l’importance de la convivialité et de l’accessibilité des

outils de réseautage social, et appuient l’idée que cette technologie peut servir d’appui utile aux

activités des FC.

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be

helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model

designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a

published thesaurus, e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select

indexing terms which are Unclassified, the classification of each should be indicated as with the title.)

social, network, collaboration, distributed teams, connecting