seabeyond 2011 processone - david banes: cleartext microblogging

Enterprise - Microblogging - IM - Email

Enterprise IM, Group Chat, Microblogging & Email

With Integrated Content Filtering, Malware Protection, URL & Image Filtering and Archiving with eDiscovery

David Banes - CEO/Founder

Presentation for

Wednesday, 2 February 2011

Me, product and R&D management in IT security and messaging, (Dr Sols (McAfee AV engine), Symantec(NAV special projects and virus analysis), MessageLabs(SaaS infrastructure, DBCS projects) moving into social platforms in 2005 (Jive Forums, then Clearspace/Jive SBS, hosted/SaaS and on premise.

Going to talk about our microblogging client/server and the decisions we took.


Cleartext Overview

• EIM, Group Chat & Microblogging

• SaaS / on premise hybrid, desktop and mobile

• Web portal for resellers and clients

• Security and compliance designed in

• Standards based XMPP platform (RFC 3920, 3921)

• Extensible e.g. microblogging component


Background - Cleartext Email SaaS, hosting, managed security, business messaging not end user free/freemium etc.CipherIM (1999, moved to Jabber 2001-3)Our platform is ejabberd, custom erlang modules, python microblogging server component, AIR desktop app.


‘Social’ Technologies

Public Private Enterprise

Web Social Facebook Mzinga Jive SBS

Microblogging Twitter Yammer Cleartext


Our view of this technology spacePublic - Open to everyone, who cares who sees who and what is shared? Private - Closed network - Company confidential data etc (Yammer talk about security but they mean SSL)Enterprise - has security and compliance built in


Microblogging

• Main Features

• Timelines, public & direct messages

• Profiles, followers and following lists

• Re-Tweets and mentions

• Hashtags, search (track)

• Enterprise Microblogging

• Malware & URL Filtering

• Content Control

• Archiving & eDiscovery


Key features and how they align with XMPP, so the technology aligns well.Lists and groups also feature which XMPP can handleURL filtering became important a year or so back


Federated Microblogging

• The difference between public and enterprise

• Problems to solve

• Message routing

• Access control

• Message data, address books, search results

• How would a web interface work across domains/servers?

• Is search/track across domains realistic?

• Do we need to solve them for business customers?


Twitter vs, Cleartext, Yammer, OneSocialWeb etc (public services, we want everyone to see, enterprise we don’t, especially those outside the business)Routing is easy - XMPPData access is the real issue (msg data, address books re: follows etc, search)Groups and chat rooms offer some solutionsNo not for public web based microblogging, but yes for XMPP enterprise. (although Yammer use HTTP and XMPP they still silo domains, but allow cross domain groups )


Enterprise Microblogging

• Minimum Features for business use

• Similar to Twitter, tweet, follow, search, DM, RT etc

• ‘Real time’ search results

• Cloud or on premise install*

• Security & compliance modules

• Desktop app to fully integrate with IM.

• Mobile, we use 3rd party eg OneTeam


Key features and how they align with twitter, base line features, i.e. no lists and backward facing timelines not important if got local history and SaaS archive First three are generic, next after talking to customersWe have web app in R&D, it works but not our focusArguably there’s no pressing need for federation yet, customers see microblogging as an internal productivity tool


Enterprise Microblogging Server

• Technology Features

• Inspired by XMPP: The Definitive Guide

• Modelled on ProcessOne Twitter Gateway

• Python XMPP component (XEP-0114)

• ejabberd MySQL user database with modified schema

• NOT federated!

• Announcement! Open Source (Feb ’11)


Cleartext Microblogging platformMessage driven (not IQ’s), just like the processone twitter gateway.Python is easier to work with than erlangRe-use ejabberd schema with small mods to users table, plus a couple more tables for followers etcOpting out of federation ‘solves’ the before mentioned data security problems.We have the component 90% ready for open source, just install docs to tidy up.


Text Based Commands

Help CommandCleartext microblog, version 0.1.11.

Commands:

"me" - shows who you are, your username and jid (Jabber ID)

"ers" - shows your followers

"ing" - shows who you follow

"f username" - follow this user

"u username" - unfollow this user

"d username message text" - send direct message to the user

"@username message text" - mention a user, a public message to a user

"s" - show saved searches

"s word" - save live search term

"us word" - delete live search term

"help" - show this help

Need more help? Go to the online help: http://www.cleartext.com/esm


How does the Cleartext solution work?Just like Process One’s Twitter gateway, the server understands a set of commands


Custom Stanzas

Help Command<message id="153" from="[email protected]/9643007101295859746231576" to="cleartext.cleartext.com" type="chat">

<body>help</body>

<active xmlns="http://jabber.org/protocol/chatstates"/>

<x from="[email protected]/9643007101295859746231576" stamp="2011-01-24T10:46:10Z" xmlns="urn:xmpp:delay"/>

<x xmlns="http://cleartext.net/mblog">

<buddy type="sender">

<displayName>[email protected]</displayName>

<userName>dbanes</userName>

<jid>[email protected]</jid>

<avatar type="hash">56a03afdd9ab286d43c8f1715fac013d4c53f15a</avatar>

<serviceJid>cleartext.cleartext.com</serviceJid>

</buddy>

</x>

</message>


We have a set of custom stanzas that the client and server understands

mailto:[email protected]


http://jabber.org/protocol/chatstates




http://cleartext.net/mblog







Server Response

Cleartext microblog, version 0.1.11.

Commands:

"me" - shows who you are, your username and jid (Jabber ID)

"ers" - shows your followers

"ing" - shows who you follow

"f username" - follow this user

"u username" - unfollow this user

"d username message text" - send direct message to the user

"@username message text" - mention a user, a public message to a user

"s" - show saved searches

"s word" - save live search term

"us word" - delete live search term

"help" - show this help

Need more help? Go to the online help: http://www.cleartext.com/esm



Custom Stanzas

Follow Command<message id="41" from="[email protected]/4732502611296587389793170" to="twitter.cleartext.com" type="chat">

<body>f noornet</body>



</message>

Server Response

<message from='twitter.cleartext.com' to='[email protected]/4732502611296587389793170' type='chat'>

<body>User noornet has been added to your following list</body>

</message>











Custom Stanzas

To Tweet...<message id="156" from="[email protected]/9643007101295859746231576" to="cleartext.cleartext.com" type="chat">

<body>it's time for end of month expenses please</body>



<x xmlns="http://cleartext.net/mblog">

<buddy type="sender">

<displayName>[email protected]</displayName>

<userName>dbanes</userName>


<avatar type="hash">56a03afdd9ab286d43c8f1715fac013d4c53f15a</avatar>


</buddy>

</x>

</message>















Custom Stanzas

Incoming Tweet...<message from='cleartext.cleartext.com' to='[email protected]' type='chat'>

<body>@admin: my expenses will be late</body>

<x xmlns='http://cleartext.net/mblog'>

<buddy type='sender'>

<displayName>Ozzie</displayName>

<userName>admin</userName>


<avatar type='hash'>ed90623da0e6963516abce705dd24bf0e84a03bd</avatar>


<text xmlns='http://cleartext.net/mblog'>my expenses will be late</text>

</buddy>

</x>

</message>




http://cleartext.net/mblog'







Incoming Tweets



XMPP Goodness• Customers can add their own bots e.g. CRM’s can tweet

• DM’s become regular IM sessions (push)

• We can use a users profile (vCard)

• All messages to/from all end points are;

• Archived

• URL scanned

• Image Scanned

• etc...


Simple extensibility that any developer can work with and get results quickly.


CRM’s can Tweet

$conn = new XMPPHP_XMPP('xmpp.service.net', 5222, 'our_crm', 'our_password', 'xmpphp', 'service.net',);

try {

$conn->connect();

$conn->processUntil('session_start');

$conn->presence();

$conn->message('cleartext.service.net', $this->session->userdata('userfullname').' edited customer: '.$_POST['customer_name']);

$conn->disconnect();

} catch(XMPPHP_Exception $e) {

die($e->getMessage());

}


Many libraries that allow customer IT staff to put data feeds together, for example our CRM is built with PHP so it’s as simple as this.


Summary

• PubSub extra work, lack of client support

• XEP-0277 not ready for prime time?

• We don’t think federation is important to business yet

• A federated microblogging platform is more about data security than protocols and routing

• The ACL problem is non-trivial and an admins nightmare

• Join/fork us at Github - a solution that works now.


Does PubSub address data security questions?Microblogging over XMPP - xmpp.org/extensions/xep-0277.htmlLowest common denominator being XMPP messagesWe needed an EIM client anyway Github will have code, docs which include install help and custom stanza details


DEMO



Questions

http://www.cleartext.com/

[email protected]

Twitter @dbanes


seabeyond 2011 processone - david banes: cleartext microblogging

Documents