nemertes pilot house awards uc 2011 12

N e m e r t e s R e s e a r c h w w w . n e m e r t e s . c o m + 1 8 8 8 . 2 4 1 . 2 6 8 5

Nemertes Research PilotHouse Awards Unified Communications The Nemertes Research annual PilotHouse Awards provide insight on the performance of technology vendors, according to feedback from IT decision-‐makers who use their products or services.

Q3 11

08 Fall

©Nemertes Research 2011 ± www.nemertes.com ± 888-‐241-‐2685 ± DN 1530 2

TABLE OF CONTENTS Unified Communications ....................................................................................................... 3 Award Definition ............................................................................................................................... 3 Overview .............................................................................................................................................. 3 Market Classification ....................................................................................................................... 3 Ratings .................................................................................................................................................. 4 Ratings Categories ............................................................................................................................ 5 Technology ......................................................................................................................................................... 5 Customer Service ............................................................................................................................................. 5 Value ...................................................................................................................................................................... 5

Results Summary ............................................................................................................................... 6 Analysis ....................................................................................................................................... 7 PilotHouse Market Challenger Winner ............................................................................ 9 Siemens ................................................................................................................................................. 9 Technology ...................................................................................................................................................... 10 Customer Service .......................................................................................................................................... 10 Value ................................................................................................................................................................... 10

PilotHouse Market Leader Winner ................................................................................. 11 Cisco .................................................................................................................................................... 11 Technology ...................................................................................................................................................... 12 Customer Service .......................................................................................................................................... 12 Value ................................................................................................................................................................... 12

Pilothouse Finalists: Market Leaders ............................................................................ 13 Avaya .................................................................................................................................................. 14 Microsoft ........................................................................................................................................... 16 IBM Lotus .......................................................................................................................................... 18

PilotHouse Finalists: Market Challengers .................................................................... 20 ShoreTel ............................................................................................................................................ 21 Mitel .................................................................................................................................................... 23

Other Market Challengers ................................................................................................. 26 Conclusion ............................................................................................................................... 27 Methodology ........................................................................................................................... 28 Sample Frame ................................................................................. Error! Bookmark not defined. Planned Sample Size ...................................................................... Error! Bookmark not defined. Survey Sub-‐Groups/Stratification ............................................ Error! Bookmark not defined.

Awards .............................................................................................. Error! Bookmark not defined. Timing ............................................................................................... Error! Bookmark not defined. Incentives to Participate & Time Commitment .................. Error! Bookmark not defined. Future Plans ....................................................................................... Error! Bookmark not defined.


UNIFIED COMMUNICATIONS By Irwin Lazar VP and Service Director, Nemertes Research

Award Definition The Nemertes PilotHouse award for Unified Communications (UC) recognizes vendors identified and rated by IT professionals as their strategic partner for delivering UC products, which integrate voice, video, conferencing, messaging, and presence with office and business-‐process applications to improve collaboration. IT professionals who use these services rated their providers on technology, value, and customer service.

Overview ± The goal of PilotHouse awards is to provide analysis of vendor and service-‐

provider performance from the perspective of their business users. ± Many research firms offer market ranking; Nemertes’ research and analysis is

unique, based 100% on the views and experience of actual Unified Communications users.

± Research is wholly independent and not sponsored; Nemertes has no influence over vendor or service provider performance.

± Opinions are those of the IT professionals who have selected, designed and deployed the technology or service.

± By combining benchmarking (direct user interviews) and surveys, Nemertes is able to provide unique insight into why IT professionals rated vendors the way they did.

For this award, Nemertes gathered ratings on UC system and application providers with a range of offerings. (More detail on the program, and demographics of participating IT professionals is available in the methodology at the end of this report.)

Market Classification We segmented UC providers into two categories: Market Leaders and Market Challengers, and offered awards within each category. To determine the categorizations, Nemertes’ analysts evaluated UC market presence (looking at revenue, device shipments, and number of customers) based on our own research


and publicly available data. Analysts also examined natural breakpoints in the data, and segmented the Market Leaders as those who collectively accounted for the vast majority of each market, and Market Challengers who accounted for a smaller percentage of the overall market.

UNIFIED COMMUNICATIONS Market Leaders Market Challengers

Avaya, Cisco, IBM Lotus, Microsoft Alcatel-Lucent, Mitel, NEC, ShoreTel,

Siemens

Table 1: Vendor Classification, Unified Communications, 2011

Nemertes defines the UC market as “segmented,” meaning that no vendor controls more than 30% of market share, and no two vendors control more than 50%. As published in various public sources, Cisco, Avaya, IBM Lotus, and Microsoft account for more than 50% of the UC market. However the UC market is often difficult to classify, thanks to varying definitions of UC. Nemertes defines UC as the integration of various forms of real-‐time and non-‐real time collaboration (e.g. voice, video, messaging, conferencing) into a set of applications sharing presence, and enabling establishment of any mode of communications (e.g. escalating an IM into a phone call, video chat, or Web conference, all through the same user interface). Vendors typically classify any of their standalone offerings in the UC space (e.g. voice, unified messaging, Web conferencing, etc.) as “UC,” making it difficult to determine specific components of a UC implementation. The Market Leaders reflect the dualities of the UC market: Those with large market share in voice (Avaya, Cisco) that are broadening into the desktop; and those with large market share in desktop collaboration (IBM Lotus, Microsoft) that are broadening into voice. Market Challengers have smaller market shares, or have traditionally focused on the small/midsize business market or a limited set of verticals. It’s worth noting that our data set predominantly reflects U.S.-‐centric enterprises, thus we classify vendors such as Alcatel-‐Lucent and Siemens, with large market share outside the U.S., as Challengers.

Ratings We asked IT professionals to rate UC providers using a 5-‐point scale, where 5 is excellent, 4 is good, 3 is fair, 2 is poor, and 1 is unacceptable. Nemertes then used these raw scores to compute average scores for each category.


The maximum possible score is a 5.0 (and although some vendors did receive perfect scores from individual IT participants, none received a perfect score when all ratings were averaged).

Ratings Categories The participants rated their technology providers in three areas: ⇒ Technology ⇒ Customer Service ⇒ Value ⇒ Overall Rating (average of Technology, Customer Service, and Value)

Technology Technology ratings gauge how customers view the sophistication, features, and implementation of the UC product they’re rating. Additionally, this score reflects how much of a leader a vendor is in the UC industry, from the perspective of the customer.

Customer Service Customer-‐service ratings cover how providers perform in areas such as technical support, responsiveness to deployment problems and concerns, sales support, and general customer care. Additionally, technology users considered the willingness and ability of the vendors to answer questions effectively and promptly.

Value Value ratings are essentially the way customers perceive what they get for what they pay for. In other words, are they getting their “bang for the buck?”


Results Summary

Chart 1: Overall Scores, Unified Communications, 2011

⇒ Among Market Challengers, Siemens wins the PilotHouse Award.

• Siemens’ overall score is 4.27.

⇒ Among Market Leaders, Cisco wins the PilotHouse Award. • Cisco’s overall score is 4.17.

⇒ A total of nine providers received enough responses for us to include them in

this year’s analysis. • Four providers are Market Leaders; five are Market Challengers. • Market Leaders’ overall average score is 4.06. • Market Challengers’ overall average score is 3.90.

4.27

4.17

4.11

4.05

3.95 3.93

3.79 3.78

3.70

3.50

3.60

3.70

3.80

3.90

4.00

4.10

4.20

4.30

Siemen

s

Cisco

Avaya

Microso

ft

ShoreTel

IBM Lotus

Mitel

NEC

Alcatel

-Lucent

Unified Communications: All Vendors, Overall Scores

Market Leaders Market Challengers


ANALYSIS

Table 2: Vendor Scores, Unified Communications, 2011

Overall, scores for UC lagged slightly behind those for other technology areas, with only four vendors scoring higher than a 4.0. Winners significantly outperformed non-‐winners, especially in the Market Challenger category where Siemens was the only vendor to score higher than a 4.0 in all categories. Notable is the lack of any consistency of scores among ratings categories. Cisco, for example saw its highest score in technology, while Siemens scored best in value. Absent are any consistently poor scores in a single category (e.g. “customer service consistently lagging behind value”). What does this mean? Each vendor has strengths and weaknesses that it must focus on improving, rather than a weak area (or areas) existing for the entire UC marketplace. Cisco’s biggest weakness (customer service) isn’t the same as Avaya’s (value) or IBM Lotus’ (technology). It also reflects the relative “newness” of the UC market and the fact that it’s rapidly changing in terms of product capabilities. Like last year, those who approach the UC market from a telephony perspective (Cisco, Avaya) outscored those who have moved from IM/messaging into telephony

Winners Overall TechnologyCustomer Service Value

Siemens 4.27 4.27 4.18 4.36Cisco 4.17 4.23 4.16 4.12

Other FinalistsAvaya 4.11 4.19 4.10 4.03Microsoft 4.05 4.13 3.96 4.06ShoreTel 3.95 3.92 3.85 4.08IBM Lotus 3.93 3.88 3.90 4.03Mitel 3.79 3.64 4.05 3.68NEC 3.78 3.67 3.67 4.00Alcatel-Lucent 3.70 3.72 3.67 3.72

Nemertes 2011 PilotHouse AwardsUnified Communications

Market Leaders Market Challengers

Avaya, Cisco, IBM Lotus, Microsoft Alcatel-Lucent, Mitel, NEC, ShoreTel, Siemens

Rating Scale: 5=Excellent; 4=Good; 3=Fair; 2=Poor; 1=Unacceptable


(Microsoft, IBM Lotus). This reflects the continued struggle for IM/messaging vendors to play in a space where customers focus far more attention on reliability, resiliency, performance, and support than in the desktop applications space. This year, like last, both UC winners come from telephony backgrounds. IM/messaging vendors must continue their efforts to demonstrate to IT leaders that they understand the real-‐time application requirements, of telephony and increasingly video. ⇒ Overall, Market Leaders score a 4.06; Challengers earn a 3.90. Siemens is the

exception to the rule, outperforming all vendors regardless of size. • Reason: The primary gaps between Market Leaders and Market Challengers

exists in technology and customer service. Market Leaders have deeper pockets, larger support networks, and a large installed base from which they can cross-‐sell; using bundled licensing programs to deliver UC as an add-‐on to existing telephony, video, or IM/messaging or other application licenses.

⇒ In technology, Market Leaders compile a score of 4.11 and Challengers get a 3.84. • Reason: Market Leaders typically have the broadest product offering, the

largest R&D budgets and often the greatest support for mobility, a hot area among IT leaders these days.

⇒ In customer service, Market Leaders receive a score of 4.03 and Challengers earn a 3.88. • Reason: Market Leaders typically have the fiscal wherewithal to devote

substantial resources to not just first-‐tier customer service, but higher-‐level technical support. Market Challengers tend to more often sell and support customers through channels, where support may vary. Many IT leaders say they see differences in the support they receive from multiple channels for the same vendor.

⇒ In value, Market Leaders garner a 4.06 and Challengers receive a 3.97. • Reason: Here is where Market Challengers are placing their competitive

emphasis, using innovative approaches such as cloud, or support for virtualization to offer compelling services as a reduced cost. Although they aren’t yet demonstrating greater value than Market Leaders, they are close.

⇒ No single score factored into winner’s advantages, rather all winners outscored their competitors across the board in all scores. • Siemens has the top overall scores among all vendors, in all scoring

categories. • No Market Leader outscored Cisco in any of the three ratings categories.


PILOTHOUSE MARKET CHALLENGER WINNER

Chart 2: PilotHouse Market Challenger Winner: Siemens, Unified Communications, 2011

Siemens Summary: Siemens’ 4.27 is the top overall score among Market Challengers, and it’s also the highest overall score among all UC vendors. In fact, Siemens receives the highest score in each of the three categories measured. Siemens is somewhat of a pioneer in the UC space, having introduced arguably the first UC product, OpenScape, back in the early 2000s (now knows as OpenScape UC Server). Siemens continues to innovate, offering both on-‐premise and hosted solutions, as well as a cloud-‐based service. Though Siemens lacks the U.S. market share of Cisco and Avaya, its offering is just as broad, featuring a full suite of UC and telephony services covering small and large offices; contact centers, and specific vertical solutions (e.g. trading floors). “OpenScape is a great tool/service for us,” says the IT manager of a state government agency. ⇒ Then and Now: Siemens improved its scores across the board from last year,

from an overall 3.75 to this year’s 4.27 with the largest jumps coming in technology (3.71 to 4.27) and value (3.71 to 4.36). These scores reflect Siemens’

4.27 4.27

4.18

4.36

4.00

4.10

4.20

4.30

4.40

Overall Technology Customer Service Value

Siemens Ratings


efforts to not only broaden its solution set, but to differentiate itself from competitors by offering delivery models such as cloud-‐based services.

⇒ Future Direction: The big challenge for Siemens is to expand its reach in the U.S. market beyond the SMB and specific verticals such as healthcare, education, and government, where it has had historical strength. This year, about 80% of the firms rating Siemens were smaller than $50 million in revenue. The good thing for Siemens is that customers evenly praise its performance across all ratings areas. IT leaders in hospitality and manufacturing both praise its quality, while the IT director of a small education institution says, “It’s all about service.”

Technology ⇒ Siemens’ 4.27, like all its scores, is the highest among all UC vendors, Market

Leader and Market Challenger alike. With its broad product set, and ability to support multiple delivery models, Siemens’ customers consider it to be a technology leader.

Customer Service ⇒ Siemens’ customer-‐service score of 4.18 is its lowest rating, considerably lower

than its other scores but overall still the highest among all vendors. Siemens’ score is particularly impressive given the change the company has gone through over the last few years as Siemens AG sold a majority stake in the firm to investment firm Gores Group. • One strength for Siemens is its international footprint. “We choose Siemens

because it is international, able to support us in multiple countries,” says the CIO for a midsize professional-‐services firm.

Value ⇒ Value is Siemens top-‐scoring area, with a 4.36, and again, the highest score

among all rated firms. Customers perceive Siemens as providing tremendous bang for the buck. • “For us, it’s price that makes us choose Siemens,” says the IT buyer for a

small construction company.


PILOTHOUSE MARKET LEADER WINNER

Chart 3: PilotHouse Market Leader Winner: Cisco, Unified Communications, 2011

Cisco ⇒ Summary: Cisco’s 4.17 is the second-‐highest overall score, and the top overall

score among Market Leaders. Cisco wins its second consecutive Market Leader PilotHouse award. Customers routinely cite reliability, service and support, and Cisco’s breadth of products as key buying criteria. • Says the IT director for a global publishing and media company, “Cisco

provides us good range of options for voice, video, conferencing, and messaging.” Adds the IT manager for a regional healthcare firm, “Cisco means reliability.”

⇒ Then and Now: Cisco continues to demonstrate continued improvement; raising

its score from a 4.08 in 2010 (after increasing from a 3.88 in 2009). Cisco’s increasing scores demonstrate its payoff from recent acquisitions to broaden its UC portfolio including Jabber and Tandberg. Overall Cisco is a reliable vendor with good support services, says the IT director for a regional manufacturing firm.

4.17

4.23

4.16

4.12

4.00

4.10

4.20

4.30


Cisco Ratings


⇒ Future Direction: For Cisco, the key challenge is branching out beyond voice and selling its customers on its vision of video emerging as the key component of a collaboration strategy. “Cisco envisions presence and video and all this usability. We aren't really seeing the value out of it yet,” says the telecom manager for a global manufacturing organization. However Cisco’s push to drive technology change is resonating with its customers, its 4.23 technology score is its highest, and the highest of all Market Leaders.

Technology ⇒ As noted, Cisco scores a 4.23 on technology, highest of all Market Leaders, and a

solid improvement over last year’s 4.08. Cisco continues to differentiate its UC offerings through support for ubiquitous video and extensibility beyond the enterprise firewall, an effort that is resonating with its customers. • “Cisco makes it easier for us to stay connected to the outside world,” says the

CTO of a midsize manufacturing organization.

Customer Service ⇒ Cisco scores a 4.16 on customer service, just .02 below overall winner Siemens,

but ahead of its fellow Market Leaders. Here again, Cisco improved on its 2009 score of 3.99. Customers continue to praise Cisco’s service and support, which is a departure from recent years and a clear, concerted effort on Cisco’s part. • “With Cisco we always get fast response times whenever we need support,”

says the CIO of a small manufacturing company. • “Cisco provides great service across the board,” says the head of IT for a

midsize education organization notes.

Value ⇒ Cisco’s value score of 4.12 was its lowest overall score in any rating area, but

even here Cisco beats all of its Market Leader competition (and improves from a 3.86 in 2009, its largest improvement in any area). We’ve often heard the perception that Cisco is a premium brand, more expensive, than other products, but that “nobody gets fired for buying Cisco.” Cisco’s improved score this year is evidence that it is increasingly able to demonstrate the value of its products, even in cases where the actual price is more expensive than competitors. Perhaps more importantly, we hear praise for Cisco’s cost competitiveness, something we rarely heard before. • “We looked at ShoreTel, and ShoreTel was more expensive. We looked at

video this year with Lifesize; Cisco was cheaper there, too,” says the senior architect of a global manufacturing company.


PILOTHOUSE FINALISTS: MARKET LEADERS

Chart 4: PilotHouse Market Leaders: Unified Communications, 2011

4.17

4.23

4.16

4.12 4.11

4.19

4.10

4.03 4.05

4.13

3.96

4.06

3.93

3.88 3.90

4.03

3.80

3.90

4.00

4.10

4.20

4.30


Unified Communications: Market Leaders

Cisco Avaya Microsoft IBM Lotus


Chart 5: Market Leader: Avaya, Unified Communications, 2011

Avaya ⇒ Avaya and Overall Scores Avaya closes the gap between it and Cisco this year, narrowing its overall disadvantage to just .06, versus .13 in 2010. It’s impressive that it improved its competitive positioning despite the upheaval of executing on its integration of Nortel, shifting to a channel-‐based model for the SME, and delivering a new line of products focused around its Aura SIP session manager. Avaya’s scores are marked by significant disparity between its technology (4.19) and value (4.03) ratings, reflecting on both the success of Avaya in using Aura, it’s expanded mobility products, new video offerings, and its recently launched Flare user experience to portray itself as a technical leader; and the failure of Avaya to improve its value score as much as its other scores (value is the one ratings category where Avaya trails the average within its market classification). For comparison, the margin between Avaya’s highest and lowest score in 2010 was .13 where this year it grows to .16. While Avaya improved its technology score from a 3.95 to 4.19, it only improved value from 3.82 to 4.03. ⇒ The average overall score of all Market Leaders is 4.06; Avaya’s is 4.11. ⇒ The average technology score of all Market Leaders is 4.11; Avaya’s is 4.19.

• “I like the fact that Avaya is an open system and will integrate with other applications,” says the IT director of a midsize services firm.

4.11

4.19

4.10

4.03

4.00

4.10

4.20


Avaya Ratings


⇒ The average customer service score of all Market Leaders is 4.03; Avaya’s is 4.10. • “We have great account team. They find us answers, and are willing to work

with us. You can't ask for much more than that,” says the IT manager of a large professional-‐services company

⇒ The average value score of all Market Leaders is 4.06; Avaya’s is 4.03. • “There are better phones out there at a lower price point, but Avaya’s

products work,” says the manager of IT at a small educational institution.


Chart 6: Market Leader: Microsoft, Unified Communications, 2011

Microsoft ⇒ Microsoft and Overall Scores Microsoft’s overall score is a 4.05, trailing Avaya by only .06, narrowing the gap between it and Avaya from .12 in 2010. Microsoft continues to gain tremendous momentum for its UC offering, with more companies citing Microsoft as their strategic vendor for UC than any other vendor. With the introduction of Lync in 2011, Microsoft has taken direct aim at not only the desktop messaging and conferencing portion of the UC market, but at the core voice services largely owned by Market Leaders Cisco and Avaya. Microsoft still has work to do to convince IT buyers that it is able to offer reliable, feature rich voice services, though its slightly-‐higher-‐than-‐average technology score demonstrates that Microsoft is convincing its customers that it is driving technology change. Microsoft’s biggest challenge is in customer service, where it trails the mean Market Leader score by .07. As in 2010, customer service remains Microsoft’s Achilles heel, though its score improved from 3.66 last year to 3.96 this year. If Microsoft can continue to improve on customer service, it can challenge Market Leaders for the 2012 PilotHouse award. ⇒ The average overall score of all Market Leaders is 4.06; Microsoft’s is 4.05.

• “We haven't had many problems; their overall support has been good. They are proactive in including us in demos/pilots, have allowed us to collaborate

4.05

4.13

3.96

4.06

3.50

3.60

3.70

3.80

3.90

4.00

4.10

4.20


Microsoft Ratings


with other large companies,” says the director of telecom for a global energy firm.

⇒ The average technology score of all Market Leaders is 4.11; Microsoft’s is 4.13. • “It works well for what it does, but the voice side is a little shaky,” says the

senior architect for a financial-‐services firm. ⇒ The average customer-‐service score of all Market Leaders is 4.03; Microsoft’s is

3.96. • “They are very confusing and difficult to work with. Sometimes they want to

work directly with you and sometimes they want to send you to a partner. Sometimes they send you to an account team that gives you wrong information,” says the IT director for a global manufacturing firm.

⇒ The average value score of all Market Leaders is 4.06; Microsoft’s is 4.06. • “The fact that we've been able to take this product and offer so many

capabilities is huge value; never thought I'd say Microsoft and value in the same sentence,” says the IT manager for a global technology company.


Chart 7: Market Leader: IBM Lotus, Unified Communications, 2011

IBM Lotus ⇒ IBM and Overall Scores IBM improved its scores across the board from 2010, with the greatest gain coming in value, where it rose from a 3.56 in 2010 to a 4.03 in 2011, tying Market Leader runner-‐up Avaya though still trailing Microsoft. IBM Lotus, like Microsoft, comes at the UC market from a history of providing messaging and non-‐real-‐time collaboration applications. Unlike Microsoft, Lotus isn’t competing for the telephony market, rather its strategy is based on delivering presence, Web conferencing, instant messaging and softphone/video clients that integrate with standards-‐based voice/video backend platforms from others. Given its more narrow focus, it’s not surprising that technology is still the area where Lotus greatly trails the competition. Customers like the value in what they get for their money, but they don’t perceive IBM Lotus as a technical leader. ⇒ The average overall score of all Market Leaders is 4.06; IBM’s is 3.93. ⇒ The average technology score of all Market Leaders is 4.11; IBM’s is 3.88.

• “Great integration with Lotus Notes, good IM, but lousy integration with other products,” says the IT architect of a global manufacturing firm.

⇒ The average customer-‐service score of all Market Leaders is 4.03; IBM’s is 3.90.

3.93

3.88 3.90

4.03

3.50

3.60

3.70

3.80

3.90

4.00

4.10


IBM Lotus Ratings


• “We have IBM Lotus UC products, and they work well, but getting good customer support is challenging,” says the IT director of a midsize manufacturing company.

⇒ The average value score of all Market Leaders is 4.06; IBM’s is 4.03. • “IBM has a much better licensing strategy, and is less aggressive in auditing

our licensing, especially compared to Microsoft,” says the director of telecom for a global manufacturing firm.


PILOTHOUSE FINALISTS: MARKET CHALLENGERS

Chart 8: PilotHouse Market Challengers: Unified Communications, 2011

4.27 4.27

4.18

4.36

3.95 3.92

3.85

4.08

3.78

3.67 3.67

4.00

3.79

3.64

4.05

3.68

3.70 3.72 3.67 3.72

3.50

3.60

3.70

3.80

3.90

4.00

4.10

4.20

4.30

4.40


Unified Communications: Market Challengers

Siemens ShoreTel NEC Mitel Alcatel-Lucent


Chart 9: Market Challenger: ShoreTel, Unified Communications, 2011

ShoreTel ⇒ IBM and Overall Scores ShoreTel comes in as the Market Challenger runner up, trailing Siemens in all categories, but leading all other Market Challengers in every ratings area other than customer service, where it trails Mitel. As a relative newcomer in the UC space compared with Siemens, ShoreTel is still increasing its portfolio, recently improving its mobility and messaging services. ShoreTel’s customers largely praise the value and feature sets of what they are buying, but ShoreTel must address customer-‐ service concerns (historically a strong spot for it in the IP telephony market) to increase its UC success. ⇒ The average overall score of all Market Challengers is 3.90; ShoreTel’s is 3.95. ⇒ The average technology score of all Market Challengers is 3.84; ShoreTel’s is

3.92. • “ShoreTel’s solution is reliable, and provides a complete feature set,” says the

manager of a small software firm. ⇒ The average customer-‐service score of all Market Challengers is 3.88 ShoreTel’s

was 3.85. • “They need to improve on their software engineering practices. It seems like

there are patches. Fix a few, break a few. And that's caused a little it of

3.95 3.92

3.85

4.08

3.50

3.60

3.70

3.80

3.90

4.00

4.10


ShoreTel Ratings


frustration with us. You put in a patch, and a new issue. I just patched last night after identifying three issues. One, receptionists couldn't get to certain calls,” says the CIO of a midsize professional-‐services company.

⇒ The average value score of all Market Challengers is 3.97; ShoreTel’s is 4.08. • “For us ShoreTel’s cost was the biggest driver,” says the head of IT for a small

professional-‐services firm.


Chart 10: Market Challenger: Mitel, Unified Communications, 2011

Mitel ⇒ Mitel and Overall Scores Save not for the second-‐highest score among Market Challengers in customer service, Mitel would have finished last among all others in its category, trailing other Challengers in value and technology. Although Mitel improved its customer-‐service score from 2010 (3.99) to this year’s 4.05, it’s technology score dropped from a 4.05 to 3.64, and its value score also dropped; going from 3.86 in 2010 to 3.68 in 2011. Despite recent turmoil that have resulted in management changes at the top, these scores suggest Mitel and its channel partners are overall improving their customer service and support. If Mitel addresses technology and value concerns, it will position itself among the top of the Market Challengers. ⇒ The average overall score of all Market Challengers is 3.90; Mitel’s is 3.79. ⇒ The average technology score of all Market Challengers is 3.84; Mitel’s is 3.64.

• “Mitel’s products come in at a competitive price, but their technology is behind the competition,” says the IT director for a small education organization

⇒ The average customer-‐service score of all Market Challengers is 3.88; Mitel’s is 4.05.

3.79

3.64

4.05

3.68

3.50

3.60

3.70

3.80

3.90

4.00

4.10


Mitel Ratings


• “Our voice provider pushed us to go with Mitel, Mitel provides great Level 2 support,” says the IT manager of a midsize healthcare firm.

⇒ The average value score of all Market Challengers is 3.97; Mitel’s is 3.68. • “Mitel won our RFP, coming in $15k less than anyone else, and with more

functionality,” says the IT manager of a small healthcare firm.


Chart 11: Market Challenger: NEC, Unified Communications, 2011

Chart 12: Market Challenger: Alcatel-‐Lucent, Unified Communications, 2011

3.78

3.67 3.67

4.00

3.50

3.60

3.70

3.80

3.90

4.00

4.10


NEC Ratings

3.70

3.72

3.67

3.72

3.50

3.60

3.70

3.80


Alcatel-Lucent Ratings


OTHER MARKET CHALLENGERS ⇒ NEC and Alcatel-‐Lucent finish as the bottom two finalists in the Market

Challenger category; NEC did post a strong value score, good for third overall among Market Challengers; Alcatel-‐Lucent struggles in all categories. Compared to 2010, NEC sees strong improvements in technology (3.43 to 3.67) and customer service (3.50 to 3.67), while its value score improves by a whopping .64 (3.36 to 4.00). If NEC can continue to improve in both technology and customer service, it stands a chance of contending for next year’s Market Challenger award.

Alcatel-‐Lucent’s scores fall across the board from 2010. Technology drops from a 3.80 to 3.72, customer service falls from 3.96 to 3.67, and value declines from a 3.96 to a 3.72. The overall score thus declines from 3.91 to 3.70, representing the biggest decline of any UC vendor. Alcatel-‐Lucent has its work cut out for it if it wishes to improve its market position.


CONCLUSION ⇒ The UC market continues to consist of largely three types of vendors:

• Enterprise telephony companies (Avaya, Cisco) fighting to put their applications onto the desktop.

• Desktop IM/messaging vendors (IBM Lotus, Microsoft) attempting to either supplant, or exist alongside telephony vendors.

• Market Challengers offering integrated solutions aiming to grow market share, serve vertical markets and/or size segments, and challenge their larger competitors.

⇒ For another year, the telephony centric Market Leaders post the highest overall scores, but the gap is narrowing as Microsoft continues to show improvement, while IBM Lotus again trails other market leaders overall. Continue to evaluate a wide variety of UC vendors, paying attention to areas including customer service, long-‐term vision, value, and demonstrated implementation success.

⇒ Vendor Selection: Based on the outcome of the PilotHouse program, here is Nemertes’ guidance (with vendors listed in priority order): • Evaluate a minimum of four providers. Cisco, Siemens, Avaya and Microsoft

are solid options. For smaller and mid-‐size firms, consider ShoreTel, as well. • If technology is your key concern, consider Siemens, Cisco, Avaya, Microsoft,

ShoreTel, and IBM Lotus. • If customer-‐service is your key decision criteria, consider Siemens, Cisco,

Avaya, and Microsoft. • If value is your key goal, consider Siemens, Cisco, ShoreTel, Microsoft, Avaya,

and NEC. ⇒ Differentiation of Leaders and Challengers: Overall Market Leaders outscored

Market Challengers across the board. Challengers may offer a better solution for small and midsize businesses concerned with getting lost among many larger companies. And, based on other services you may have with the Challengers, they may be a good fit based on minimum annual revenue commitments.

⇒ Improvement Outlook: Watch out for continued development of new services as

vendors expand their offerings into voice and/or desktop collaboration. Also keep your eye on expanding hosted offerings that will increasingly provide an alternative (or complement) to the on-‐premises solutions evaluated for this award.


METHODOLOGY The population includes individuals primarily from U.S. companies (based in the U.S., but many of which are global multinationals) who are responsible for selecting, or influencing the selection of, suppliers of data-‐center and communications products and services.

Sample Frame In selecting the sampling frame, Nemertes has asked individuals in the following populations to rate their providers:

± U.S. business subscriber lists, including individuals who have opted to participate in surveys and who have been pre-‐screened to determine responsibility for selecting or influencing relevant products and services.

± Nemertes Research IT executive database, limited to individuals who meet the criteria for the representative population. Individuals from this list represent primarily U.S. companies, but also include companies based elsewhere that have presence in North America. The database includes individuals who have participated in, or who have expressed interest in participating in our research, or with whom Nemertes’ analysts have established a business relationship.

Individuals participated in this project using three methods: ± Web-‐based survey. This is the largest percentage of the respondents. Those

who meet the sample frame randomly received invitations to participate in the survey.

± Visitors to Nemertes’ Web site, and recipients of Nemertes’ blogs and columns in third-‐party media partners’ Web sites. They must meet the criteria to participate.

± Benchmark interviews. This is a smaller percentage of the respondents. Nemertes’ analysts asked numerous detailed qualitative questions to gauge why they rated their service providers the way they did, as well as gathering other information about their usage of communications services.

Benchmark participants spent one to three hours on the phone or in person with a Nemertes analyst discussing issues relating to their use of products and services. The Web-‐based survey participants answered a subset of the benchmark questions that focus on rating the providers, stack-‐ranking important criteria, providing financial data, open-‐ended comments, and demographics.

Planned Sample Size

According to U.S. Census Bureau figures, there are 2,306,070 companies with five or more employees. Our goal was to receive responses from a minimum of 1,000


individuals, which would give us a 95% confidence level and 3% margin of error—if every individual rated every vendor in every technology area rated. We received substantial ratings for each technology category (several hundred per category), but each vendor in each area did not receive a rating from every research participant.

About 4,000 individuals accessed the survey or participated in a benchmark interview. Of those, about 2,000 meet Nemertes’ standards to be considered “valid.” Our survey tool automatically exited individuals employed by IT vendors and providers. Analysts reviewed all other ratings (survey and benchmark) line by line, and categorized as “invalid” those who demonstrated inconsistencies or inaccuracies in their responses as part of Nemertes’ complex qualification methodology.

We achieved validity across the survey and interviews by ensuring the questions we asked were the same and that the interview group and survey group represent discrete samples of the same population. Nemertes achieves survey and interview consistency through the use of pre-‐scripted interview forms and peer review of interview protocols. Analysts also relied upon their own knowledge of the technology areas, natural breakpoints in the data, and interview notes from the survey participants to further validate ratings.

Survey Sub-‐Groups/Stratification

Nemertes’ analysts researched which providers offer products and services in each category and created lists from which participants identified their primary service providers. Participants also were able to select “other,” and identify a service provider they use that may not be included on the explicit list provided.

The challenge is that some providers (Market Leaders) have thousands of business customers and significant market share, while others (Market Challengers) have a few hundred or few thousand customers and smaller market share. We realized some providers would garner a relatively large number of ratings, based on the number of customers they have, while others would have a relatively small number of ratings.

Therefore, we created the two distinct categories for the awards, Market Leaders and Market Challengers, and compared providers within each category. Nemertes placed providers within each category based on its own research and publicly available data. Analysts also examined natural breakpoints in the data. Market Leaders typically have >10% of market share, based on these analyses. Market Challengers typically have smaller market shares. In some categories, there were not enough ratings to issue an award in the Market Challenger category, or the


market is so new that all vendors are considered Challengers. In these cases, Nemertes issues an award only in the appropriate category.

Nemertes reserves the right to address acquisitions occurring during the benchmark and survey period on a case-‐by-‐case basis. Unless otherwise noted, an acquisition merging two companies in the same award category must be complete before the start of the survey and benchmark interview period to be counted as one company in the ratings.


Awards Nemertes is issuing awards in the following categories:

Nemertes PilotHouse Awards, 2011 Award Category Market Leaders Market Challengers

Advanced Communications Services MPLS Services ü ü Carrier Ethernet Services ü ü Internet Access Services ü ü SIP Trunking Services ü No award Managed Router Services ü ü Managed Internet Services ü ü Wireless and Mobility Wireless LANs ü ü Wireless Voice & Data Services ü ü Application Delivery Application Delivery Optimization ü ü Virtual Desktops ü ü Voice Communications IP Telephony ü ü Managed IP Telephony ü ü Hosted Voice Over IP ü No award Data-Center Technologies Servers for Virtualization ü ü Storage for Virtualization ü ü Data-Center Colocation ü ü Unified Communications Unified Communications ü ü IP Contact Centers ü ü Security Managed Firewall/IDS/IPS ü ü Data-Center Firewalls ü ü Small Branch Firewalls ü ü Cloud Software as a Service: Office ü No award


Timing The Web-‐based survey was conducted between March and May 2011. The benchmark research was conducted between January and April 2011.

Incentives to Participate & Time Commitment Participants of the Web-‐based survey received a small incentive for participating in the survey. Participants from Nemertes’ database receive the findings and are invited to participate in a Webcast, in exchange for their time. The Web-‐based survey takes about 15 minutes to complete; the benchmark requires one to three hours of participants’ time.

Future Plans Nemertes plans to conduct its PilotHouse Awards program annually, though it retains the right to cancel the project at any time. About Nemertes Research: Nemertes Research is a research-‐advisory and strategic-‐consulting firm that specializes in analyzing and quantifying the business value of emerging technologies. You can learn more about Nemertes Research at our Website: http://www.nemertes.com