Multimodal and Voice Solutions

A new approach based on network intelligence

Company information

Solaiemes

Solaiemes

H1 07

H2 07

H1 08

H2 08

H1 09

Feb’09, mencionado en

el informe Frost&Sullivan

MWC 2009 Key

TakeAways Briefing

(slide 8)

• Start-up founded Q4-2006• Focused on mobile services

• Combinatorial services in NGNs, UC, Mobility• Mobile usability

• We develop our own technology• Multimodal voice/web application server platform: AVISG• Media server platform: LiveServe• PTT application server platform: VoxServe

• Objectives become an infrastructure provider of mobile service platforms

PlatformsAVISGTM

• Voice/Web multimodal platform• Simplifies access to information services from mobile devices• Allows integration with most IVR systems• Do not require installing specialized software in the device

LiveServeTM

• Media server compatible with web infrastructures• With support for live video from: mobile devices, IP cameras, webcams, etc.• Interoperates with most CRM/ERP systems to provide multimedia in IT services

VoxServeTM

• Application server for PTT• Allows emulating TETRA services with much lower costs• Allows integration with most IVR systems

Solaiemes voice/web multimodal platform

AVISG 2.0

AVISG 2.0

Mobile Push Platform•Web Contents•Multidevice•Asynchronous•Event Driven

Customer Care

• Multimodal customer care

• Operator assisted browsing

• Rich Media Alerts

Vocal Browsing

• Multimodalization of web portals

Mobile Marketing

• Multimedia push campaigns

AVISG 2.0: Mobile Multimodal Services

MultimodalSystem

Solaiemes AVISG

Seamless Mixsimultaneous

visual and vocal interaction

Leveragesexistent vocal infrastructure

Integrates withexistent IT

systems

AVISG General

Basic functionality• Builds multimodal services combining a web session and a simultaneous plain old phone call• Both, phone call and web session are synchronized• The user can talk and see the results on the device screen

• Web navigation can be commanded by voice commands• Voice dialogs can be switched by keyboard based web navigation• Vocal information and web information are always coordinated

AVISG differentiating ingredients• Service intelligence located at the network side• Most 3G subscribers are potential users

• Works in most 3G devices and in many GPRS/EDGE devices• Do not require installing special software in the device• Works over a wide range of 3G handsets, from low profile like Nokia 6210 to high-end like iPhone

• Ajax support is not required in the mobile browser• The voice session is established through a conventional phone call• The web session is based on standard HTTP/HTML technologies• Can be integrated with existing IVR platforms, no need for special voice service equipments.

AVISG from the user perspective

Best user perception

• User navigates the service using his voice• Or manually through the browser whatever he prefers• Simultaneous vocal and visual information enrich user experience

What the user needs• A mobile phone with multi-rab support (most 3G and many GPRS/EDGE devices comply)• 3G coverage

• AVISG can fallback to a just vocal service in the same phone call, in case of going out of 3G coverage

What the user does• Dial the system phone number (a standard phone call)• Listen the TTS instructions and accept a SMS message• Look at the screen device and see how the web browser opens• Navigate using voice

• The web information shown in the screen will evolve accordingly• Navigate using the keyboard to follow hyperlinks and press buttons

• The TTS information will evolve accordingly

AVISG standards basedWeb content presentation

• HTTP 1.0 and HTTP 1.1 whatever the handset supports

• Also the corresponding secure versions (HTTPS)

• XHTML according to whatever the handset supports

Vocal content presentation

• VoiceXML 2.0 y 2.1, whatever the IVR platform supports

Interconnection with IVR

• Using CCXML 1.0 internally to AVISG

Multimodal coordination

• Coordination among all modes is done using SCXML 0.9 internally

• Web content push to mobile. AVISG is capable of getting web content into the handset browserwithout user intervention. Content is pushed asynchronously to the terminal according topredefined navigations (autonavigations) and externally generated events, like those coming forman IVR. AVISG implements a content push mechanism (patent pending) on top of standard webtechnology that does not requires neither javascript nor AJAX capabilities, enabling the use of anyhandset compatible with WAP 2.0 standard.

NOTICE: Content PUSH does not require on-device applications, AJAX or javascript

• Multidevice. AVISG maintains a device database and is able to identify each web agent. Thisinformation enables the selection of the best suited push mechanism and content formats.

• Fast user perception (just the net delay). One of the most important features in AVISG is the userperception of fast response to vocal commands. The main effect of AVISG push engine is that theonly delay perceived by the user are those caused by network data transfer and IVR operation.

• Without unnecesary traffic consumption. AVISG avoids polling to implement the push mechanismand so it also avoids wasting data traffic .

Multi-device Push Engine

• Simultaneous vocal and visual browsing. AVISG implements a multimodal engine able to synchronize oneWeb mode (channel) with the browser in the handset and a BUS mode with the IVR. The multimodalengine collects events from any mode and propagates proper actions to all of them. Any voice commandor web request will cause both web and grammar change.

• Session based. AVISG is session based. It host a single session with multiple modes. Session can beinitiated either by the Web mode (access to a web page) or by the BUS mode (phone call to an IVR).

• Easy integration with IVR Systems. AVISG supports interaction with IVR carried out by a CCXML &VoiceXML standards. It also implements a bidirectional proprietary BUS to be used with IVRs nosupporting CCXML

• Easy development of services. Multimodal services just consist of:– Dynamic Web contents. They may be templates deployed directly on AVISG or web pages proxied on-line from external Web

portals.

– Dynamic Vocal contents, VoiceXML files representing dialogs associated to each page. They can also be templates deployedon AVISG or can be inferred from the web page presented simultaneously

– Service definition. Web and Vocal contents association, and automatic navigations rules definition. AVISG provides anauthoring tool.

• SaaS Ready. A multimodal AVISG platform on a SaaS configuration allows each final client to self provision

and deploy services.

Multimodal Function

• It allows data integration with external sources. It implements an ESB (Enterprise Service Bus) that eases integrationwith any external IT system. That way web pages (and voiceXML pages) are templates with dynamic data fromexternal systems. It allows implementing transactional services on AVISG.

• Integration with external IT systems. Web content, vocal content and even auto-navigations need not be just staticcontent. AVISG is ready to integrate with heterogeneous external information systems to achieve several goals:

– Introduce dynamic information in contents delivered to the user, both in the web side and in the vocal side

– Capability of initiate transactions from AVISG. A multimodal interaction from a user (vocal o manual) can input data in a form andtrigger a transaction.

– Dynamic web content presentation and auto-navigations. Data got from an external system can be used to determine the navigationof a given user and the information to present to him

• Standards based and SOA ready. AVISG is ready to integrate in a SOA ecosystem, where information sources andtransactions systems are exposed as services. This is achieved by means of an ESB integrated in AVISG. This ESB allowsto directly integrate standard services (Web services, REST services, etc) and to easily develop custom connectors toad-hoc systems

• Corporate environments and SaaS environments. The integrated ESB allows to easily integrate ad-hoc IT systems in acorporate environment and Internet services (REST or WS) in a SaaS service where the IT services are in the clientpremises and are exposed through standard Internet technologies (REST or WS).

• Push services. External services may behave in a request/response way to retrieve data or trigger transactions, but isalso possible that external IT systems trigger interactions or navigations in AVISG. The integrated ESB allows workingmuch alike an Event Driven Architecture.

Integration Data Bus

• Complete solution, no need for external web or vocal content repository. A client of this service does not need to updateseveral systems to publish a new service, all related assets: HTML files, images, CSS files, VoiceXML files and auto-navigationdefinitions are published to a single deployment node. This enhances the availability of multimodal services as is less error-pone than maintaining separate repositories for each type of asset.

• Web access. Content repository can be accessed via Web, thus allowing a remote operator to administer the contents of aservice. This web access is secured and allows different services published on AVISG to be administered by differentadministrators

• Versioning support. Each time a service is updated a version is stored, thus allowing to rollback to previous versions of theservice (that means previous versions of all assests: web files, VoiceXML files, etc.)

• SaaS ready and client self-administration. A single AVISG server can host different services, each of them for different clients.Each client can have one (or several) administrators of his services that will have complete access to all the assets of hisservices, and only to those assets, he won’t be able to see anything of any other different client service.

• Template contents, both vocal and web. Textual contents stored in the repository can be templated. That means that all textualcontent can be fed with dynamical data got from external IT systems or user input. That means Web and VoiceXML content isdynamic.

• Also allows external web portals. In any case is possible to multimodalize existent web portals without storing those webcontents in the AVISG content repository. The vocal contents (VoiceXML) can be inferred from the web content or can also bestored in the AVISG content repository.

Content repository, web and vocal

• Clustering. AVISG is able to work in a cluster mode, that means thatseveral physical AVISG servers can be configured to work as a singleserver. This provides:

– High availability , if one server goes down the other takes care of the service evenongoing sessions are transferred to the server that stays alive.

– High scalability. This makes possible to grow the server capacity without having topurchase bigger and more expensive equipment.

• Web clustering. This is achieved using classic techniquesimplemented by the Java EE container in which we have deployed theAVISG server.

• Event clustering. Regardless of which physical server host a givensession, browsing events for that session will get into the systemusing any of the alive servers in the cluster. The events will beinternally routed to the server that hosts a given session. If the serverthat is getting browsing events for the IVR goes down, the events willbe rerouted to one of the alive servers in the system.

• Highest SLA. Carrier grade system availability

Clustering, scalability and HA

AVISG1

RD

BM

S Clusterable DB

J2

EE

Se

rve

r

AVISG Core servicfes

AVISG Web Push

AVISG2

RD

BM

S

Clusterable DB

J2

EE

Se

rve

r

AVISG Core services

AVISG Web Push

Web traffic

Browsing event bus Browsing event bus

Browsing events

• JMX monitoring. AVISG can be completely monitored administered and operated using JMX, the JavaManagemente standard. Any standard JMX console, even the jconsole included in the JVM can ne used tomonitor the system. The elements managed with JMX include all Java EE Container elements and the specificAVISG elements. All of this elements have:

– JMX attributes that can be read or write to get the configuration or configure that element

– JMX operations that allow to operate on that element

– JMX events, that send asynchronous alerts that some condition has arised (errors, alerts, warnings, etc.)

• SNMP monitoring. All JMX elements can be also exposed using SNMP. They just need to be configured in anincluded JMX-SNMP bridge.

• Web access. The Java EE container chosen include a web JMX console that allows managing the entireplatform from web

• AVISG specific elements monitored:

– Session Manager. Allows managing the AVISG session lifecycle and interaction with the session. Evenbrowsing event s can be sent to that session

– Campaign manager, allows management of the content repository and auto-navigations definitions

– Configuration Manager, allows managing the complete configuration of the AVISG server

Monitoring JMX and SNMP

• AVISG supports two operation modes: self contained web contents and external web portals

• Self Contained Web contents. In this mode all web content is stored in the internal content repository, and dynamic content (templating engine)takes place in AVISG. This mode of operation is most suited to enhance existent vocal services with multimodality capabilities, as it provides:

– A single deployment point of all multimodal service resources (web, content, vocal content, navigation definitions, etc.) with versioningcapabilities to ensure allways consistency of all involved elements (vocal, web, data, etc.)

– It is easier to get web content suited to multimodal browsing, that is without vertical scrolling, and even more suited to several type ofhandsets.

– Most adequate for SaaS enviroments where clients need not have any web infrastructure. A single SaaS offering includes hosting of allelements (vocal and web)

– Allows a seamless transition from vocal-only service to multimodal service and back to vocal-only service, all in the same call. Thus thesystem ensures that a user is allways served with the highest quality of service according to his circumstances. Even in the case of dataconnection loss the user can continue in the same call with at least vocal service.

• External Web Portals (AVISG Proxy). This operation mode can be considered as a multimodal proxy. It allows an existent web portal to becomemultimodal. It takes web requests and relays them to the existent portal. The response web page is modified to implement the push mechanismand navigation can be started both from the web interface or the vocal interface. This mode of operation is most suited to enhance existentportals to give mutlimodal service, as it provides:

– No need to modify the existent web portal

– AVISG can infere the vocal content by parsing the web pages from the portal to ensure consistency between the vocal and web part.Anyway it is also possible to add custom VoiceXML content .

– Most adequate for SaaS offerings where the clients do have their own web infrastructure.

AVISG Operation Modes

Solaiemes application server for the AVISG multimodal platform

AVISG Self contained

AVISG Self contained architecture

3G Network

CCXMLEngine

VXMLBrowser

VXML reqServlet

MultidevicePush

Engine

WebDispatcher

Browsing event busNavigation

eventsVoice->webWeb->voice

Voice call

Web session

Voice eventVoice or call event

Multimodal event (change of VXML dialog)

Star

t/st

op

dia

log

ASR

eve

nt

Voice call

Web session

AVISG 2.0 Server

VoxeoProphecy

MultideviceContent

repository

Integration busESB

Navigationengine

External IT systems

Self contained service development cycle

Configuration

And data integration

Bu

s d

e S

erv

icio

s

Contenidos Navegación

<XML>

<XML><XML>

Personalización

<XML>

<XML><XML>

Integración

<XML>

<XML><XML>

Mu

ltid

isp

os

itiv

o

Bu

s d

e E

ve

nto

s

Templating

Content

Creation

Service

Design

<XML>

<XML>

<XML>

<XML>

<XML>

<XML>

Design Development Deployment

AVISG 2.0

• An AVISG service just consists of Web contents,vocal contents and auto-navigation definitions.Those all are just normal files that just need to bedeployed in the content repository

• Simple model. Content repository is representedby a web folder. Each folder inside it represents anAVISG service. There are no requirements on theinner organizations of each of the AVISG servicefolders. All web and vocal contents are stored inthe AVISG service folder. Navigation definitions arestored in a separate folder.

• Web accessible. Each AVISG service folder can beaccessed using a web interface or a WebDAVinterface. Remote web access is secured to ensureonly the service owner can access his contents.

• SaaS capable. A single repository can be used forseveral clients. Each client will be able to manageonly his own contents, and will be able to self-manage all contents in his service

Self contained web contents: repository model

Web and Vocal

contents

Navigation

definitions

• Embbeded ESB in AVISG is Mule

• State-of the-art and Standards based. It compliesto the most important standards in SOAenvironment and Java Environment (JAX-WS, JBI,WS, REST, etc.)

• In Mule all integration is declared in an XML file

• Mule provides authoring tools for that XML fileembedded in the eclipse development platform.

• There are several connectors to standard systemsavailable

• Connectors to ad-hoc systems can be developed inJava following Mule development guidelines.

Self contained web contents: IT systems integration

Solaiemes web proxy for the AVISG multimodal platform

AVISG Proxy

AVISG Proxy: a brief descriptionWhat is the AVISGTM Proxy?

• It is a proxy module allowing the reuse of external web sites on the AVISG Platform

• It allows using all the AVISG multimodal capabilities

• With the AVISG Proxy you

• … do not need to create any web contents if you want to navigate on an existing site

• … can automatically generate the VXML dialogs from the contents of a web page …

• .. or can create your own VXML dialogs associated to a given web page

What are the requirements of the external web sites

• Must be based on HTML/WML documents (flash or other proprietary formats are not supported)

• Preferably should use correct XHTML

• Must generate appropriate contents for the particular device being used

• The AVISG Proxy does not adapt the contents to the device

• Preferably, the contents of a web page should fit in the device screen to avoid scrolling

• Not special requirements with respect to HTML/WML derived technologies

• Support for: images, forms, CCS, etc.

• Limited support for: javascript/wmlscript

AVISG Proxy Architecture

VXML reqServlet

Web reqServlet

WebDispatcher

Voice reqServlet

VXMLDispatcher

Session and InteractionManager

ProxySCXMLEngine

Sess

ion

var

iab

les

Dia

log

dat

aFo

rm d

ata

Voice event

AVISG Runtime

HTTP ClientUnit

DOMUnit

URL and FormRewriting Unit

DialogUnit

Page Generation

Unit

ProxyUnit

VXML Generator

Unit

External Web Site

Servlet or JSP

Servlet or JSP

Manual generation

AVISG Proxy

AVISG Proxy in actionWhat happens with the multimodal synchronization?• The multimodal stuff is managed by the AVISG Platform• The AVISG Proxy only acts on the web mode of the multimodal session

When the AVISG Proxy mode is active• If a web request arrives to the AVISG Platform

• It is dispatched to the AVISG proxy• The AVISG Proxy maps the requested URI to an external URL• The AVISG Proxy downloads the external URL document• The AVISG Proxy obtains the DOM of the external document• From the DOM, useful information for the automatic generation of the VXML is extracted• All URLs and forms found in the DOM are rewritten (with the Proxy URL)• The final page obtained after this process is send back to the requesting device

• The intelligence on how to coordinate everything is in a SCXML state machine• The VXML dialogs can be generated automatically from the DOM data• The VXML dialogs can be also generated though specific JSPs or Servlets which have access to

the DOM data

Last enhancements and feature roadmap

AVISG Roadmap

X+V Support X+V is a standard endorsed by the W3C initially

developed by Opera and IBM some years ago

to bring vocal functionality to web browsers.

Among its nice features is to allow to embed

voice content (using VXML 2.0) on XHTML

content. This allows to leverage current web

applications architecture to bring also vocal and

multimodal functionality to voice enabled web

browsers.

The Solaiemes approach is to use the AVISG

infrastructure to implement X+V using current

web browsers that do not have any voice

functionality and using current IVR systems.

This allows bringing multimodal (web + vocal)

interaction to users that just have a simple web

browser and a vocal phone (POTS, mobile, SIP,

VoIP, widgets enabling VoIP embeded in a

webpage, etc).

Already implemented

EXAMPLE:

<?xml version="1.0"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+Voice //EN" "xhtml+voice10.dtd">

<html xmlns="http://www.w3.org/1999/xhtml"

xmlns:vxml="http://www.w3.org/2001/voicexml20"

xmlns:ev="http://www.w3.org/2001/xml-events">

<head>

<title>What You See Is What You Can Say</title>

<!—Declare this page as voice aware -->

<meta name=”solaiemes-avisg-voice-embed” content=”yes”/>

<!-- first declare the voice handlers. -->

<vxml:form id=”menu”>

<vxml:prompt>Welcome to your Hotel reservation

application</vxml:prompt>

</vxml:form>

<vxml:form id="voice_city">

<vxml:field name="field_city">

<vxml:grammar src="city.srgf" type="application/x-srgf"/>

<vxml:prompt id="city_prompt">

Please choose a city.

</vxml:prompt>

<vxml:catch event="help nomatch noinput">

For example, say Chicago.

</vxml:catch>

</vxml:field>

</vxml:form>

<vxml:form id="voice_hotel">

<vxml:field name="field_hotel">

……..ask for complete examples

Q4/2009 – Q1/2010 Features

1. Multimodal support of JavaScript

2. Multimodal AJAX controlled engine

3. Fully SaaS/Cloud offer with admin & monitoring tools

4. Interoperability with cloud telephony leader vendors and

cloud IVR providers

5. Desktop multimodal support with internet telephony API’s

6. Enhanced multi-device support

www.solaiemes.com