Make an Impact with Benchmarking, Analytics and Engagement ReportingA Benchmarking Collaboration between Higher Logic and Marketing General

Andy Steggles – President & Chief Customer Officer

Goal is to Identify:• Best Practices for Engaging Members• Quantify Benchmarking Metrics for Different Sized

Organizations• Engagement Potential• Correlation Between Engagement and Satisfaction

Methodology• Perform Correlation Analysis• Create a Composite Engagement Score (CES)• Compare with 2014 MGI MM Benchmark• 54 Engagement Variables (plus Ratios Between

Variables)

2013 Engagement Data Analyzed from:

• 20,314,584 Users (12m qualified)• 6,607,897 Members (4m qualified)• 2,376,321 Subscribers (1.8m qualified)

• 2,205,496 Members (1.6m qualified)• 170,825 Non-Members (154k qualified)

• 419 Orgs (256 qualified)• Opted In (not anonymous)

• 97 Qualified Orgs and 19 Non-Qualified Orgs

4

Timeframe and Assumptions• Point in Time Data for Some Elements• Focus is on Subscribers vs. Members• Date range is 2013 (where applicable)• Size Categories by Number of Members:

Small 0 - 1,499Small/Medium 1,500 - 4,999Medium 5,000 - 19,999Medium/Large 20,000 - 49,999Large 50,000 – 99,999X-Large 100,000+

Inclusion Criteria• Org must have ALL of the below to qualify. Greater

than:• 50 members• 50 subscribers• 50 messages• 0.005 Message per Member Ratio

• Organizations that have posted their 100th message after 1/1/2013 were flagged in the CES ranking because they had less than a full year of engagement maturity.

7

Every Organization has different goals. What is important to your organization?

Photo credit: jepoirrier

681,209 Discussion Messages• 238,333 Threads (avg 931)

• 93,045 with Replies to Group (avg 363)• 62,738 with Replies to Sender (avg 246)• 114,759 with Replies to Group or Sender (avg 448)

• 442,242 Replies (avg 1,728)• 328,259 to Group (avg 1,282)• 114,617 Replies to Sender (avg 448)• 412,127 to Group or Sender (avg 1,610) (where there is at least one

group reply)

111,346 Authors• 61,825 Thread Authors (avg 242)

• 30,943 Created Only One Thread (avg 121)• 30,882 Created More than One Thread (avg 121)

• 62,928 Replies to Group Message Authors (avg 246)• 30,978 Replied Only Once (avg 121)• 31,950 Replies More than Once (avg 125)

• 44,847 Replies to Sender Authors (avg 175)• 26,993 Replied Only Once (avg 105)• 17,854 Replies More than Once (avg 70)

Success is Subjective• Most Common Discussion Success Metrics

• Number of Discussion Messages (Activity)• Percentage of Members Subscribed (Reach)• Number of Responses per Thread (Value)• Number of Authors (Distribution)

• What is Important to Your Organization?

"In most online communities, 90% of users are lurkers who never contribute, 9% of users contribute a little, and 1% of users account for almost all the action."

- Jakob Nielsen (2006)Nielsen Norman Group

11

What are our stats?• 1.8m subscribers• 111k authors

What are our stats?• 1.8m subscribers• 111k authors

Result: 94-6

Average Number of Users, Subscribers and Contributors by Size

*In some cases, subscribers may include non-members.

**Example of “Ratio of Subscribers per Member”: The ratio for a small organization is 0.84. This means that there are 84 subscribers for every 100 members, while a Large Associations with a ratio of 0.21 means there are 21 subscribers for every 100 members.

Of those who Contributed, percentage who:

Small Small/Medium Medium Medium/Large Large X-Large

Replied to Group oneor more times

Replied to Senderone or more times

Started a Thread 49%

39%

57%

54%

32%

57%

52%

41%

59%

58%

41%

54%

60%

41%

57%

62%

35%

61%

Bios and Photos• Subscribers with Bios are 28% more likely to author

threads than an average subscriber.

• Subscribers with Photos are 5% more likely to author threads than an average subscriber.

Threads

*For threads with at least one group reply or reply to sender

**For threads with at least one group reply

***For threads with at least one reply to sender

0 2 4

% Threads with No Response

Avg. No of Replies to Group or Sender*

Avg. No of Replies to Group or Sender**

Avg. No of Replies to Group**

Avg. No Replies to Sender***

52%

3.8

1.8

4.4

4

Percentage of Threads with at least one:

0.0 0.1 0.2 0.3 0.4 0.5

Group Reply

Reply to Sender

Group or Reply to Sender

39%

26%

48%

Thread Statistics Broken out by Organization Size*

• *Size Determined by Total Number of Members

• Note: Groupings do not add up to 100 because the threads without a reply is not displayed.

0.0 0.1 0.2 0.3 0.4 0.5 0.6Percentage of Threads

Small Group Reply

Reply to Sender

Group or Reply to Sender

Medium Group Reply

Reply to Sender

Group or Reply to Sender

Large Group Reply

Reply to Sender

Group or Reply to Sender

Medium/Large Group Reply

Reply to Sender

Group or Reply to Sender

Small/Medium Group Reply

Reply to Sender

Group or Reply to Sender

X-Large Group Reply

Reply to Sender

Group or Reply to Sender

38%

22%

46%

40%

26%

49%

41%

24%

48%

38%

29%

48%

38%

24%

47%

38%

31%

48%

Small 0 - 1,499Small/Medium 1,500 - 4,999Medium 5,000 - 19,999Medium/Large 20,000 - 49,999Large 50,000 – 99,999X-Large 100,000+

Percentage of Threads with at least one:

Composite Engagement Score (CES)• CES Formula: CES = [(ratio of subscribers to members x weight) + (ratio of messages to members x weight) +(ratio of responses per thread x weight)] x 100

• Ratio of Subscribers to Members tells us on average how many subscribers per each member there is in an organization

• Ratio of Messages per Member tells us on average how many messages a member generates

• Ratio of Responses per Thread tells us on average how many responses each thread

Who has the Better Performance? (Large Orgs)

Note: ACA went live in June 2013

OrgName TNOM TNOS TNOT TNORTG TGM CES

American Counseling Association58,552 46,503 835 1,736 2,571 17.91687

Institute of Management Accountants66,365 30,372 1,565 1,965 3,530 13.41126

National Association of College and University Business Officers

53,826 13,168 997 1,474 2,471 12.73087

OrgNameTNOM TNOS TNOT TNORTG TGM CES D100MP

American Academy of Periodontology (AAP) 7,701 6,890 349 747 1,096 27.28743 2013-06-17

Construction Financial Management Association 6,793 6,658 309 625 934 27.27245 2013-01-21

National Business Officers Association 9,967 2,285 399 1,350 1,749 26.86057 2009-09-09

Dynamics NAV User Group 7,106 5,722 345 642 987 25.83747 2011-03-18

Dynamics AX User Group 15,556 12,552 747 1,355 2,102 25.27181 2013-03-26

The Institute of Continuing Legal Education 11,768 8,863 733 1,095 1,828 25.20535 2012-09-21

Regulatory Affairs Professionals Society 11,947 10,678 458 911 1,369 24.72974 2011-06-21

Who has the Better Performance? (Medium Orgs)

Who has the Better Performance? (Small Orgs)

OrgName TNOM TNOS TNOT TNORTG TGM CES

Alliance of Comprehensive Planners 242 236 1,808 4,056 5,864 2195.70

Avectra Users Group 1,095 547 788 1,136 1,924 167.83

The ALS Association 392 381 205 269 474 120.25

National Association of State Procurement Officials 1,190 1,260 612 491 1,103 92.43

League of Historic American Theatres 921 754 276 480 756 86.67

Discussion Activity by Quartile

Of those who Replied to Group, percentage who:

Average Composite Engagement Score (CES) by Organization Size

Small

Small/Medium

Medium

Medium/Large

Large

X-Large

143.3

19.5

12.5

61.9

10.9

27.6

Average Composite Engagement Score (CES) by Quartile

Average Composite Engagement Score (CES) broken out by Organization Size and Quartile

Discussion Based Email Open Rates• Average Daily Digest Open Rate: 26.9%

• Average Real Time Open Rate: 34.13%

Correlation of Renewal & Engagement

Associations by 2014 MGI MMBR & Associations with an Engaged Online Community who Agreed to Participate in a Blind-Comparison

IMO TRADE

2014 MGI Membership & Marketing Benchmark Report

76% 85%

Associations with Engaged Online Community

79% (+3%) 92% (+7%)

% of Improved Retention Correlated with Online Engagement 5%

Example: American Society of Association Executives (ASAE)

• Use Net Promoter Methodology• 21,533 members• 13,272 subscribed to Collaborate• 3016 Completed Survey• 81.6% (2462) of the Respondents were Collaborate

Users

The Hall of Fame

OrgNameTNOM TNOS TNOT TNORTG TGM CES Size Category

Alliance of Comprehensive Planners 242 236 1,808 4,056 5,864 2195.70 Small

National Employment Lawyer's Association 2,146 1,998 4,874 6,230 11,104 476.74 Small/Medium

American Society of Ophthalmic Administrators 2,640 2,562 3,181 7,731 10,912 386.85 Small/Medium

Piano Technicians Guild 3,579 3,377 2,970 11,543 14,513 379.67 Small/Medium

Veterinary Hospital Managers Association 2,170 2,036 1,240 3,796 5,036 223.57 Small/Medium

International Association of Rehabilitation Professionals 2,661 2,570 2,000 3,949 5,949 215.90 Small/Medium

National Society of Accountants 9,438 9,525 3,027 14,997 18,024 186.88 Medium

Avectra Users Group 1,095 547 788 1,136 1,924 167.83 Small

Higher Logic Users Group 2,837 1,235 1,272 2,849 4,121 142.86 Small/Medium

University Risk Management and Insurance Association, Inc., 2,178 2,546 1,156 1,920 3,076 140.41 Small/Medium

The ALS Association 392 381 205 269 474 120.25 Small

The American Association of Nurse Assessment Coordinators 13,553 13,567 6,735 9,065 15,800 116.65 Medium

Your Personalized Benchmarking Data

Andy StegglesPresident & Chief Customer Officerandy@higherlogic.com