troubleshooting in the uccx solution · troubleshooting in the uccx solution arunabh bhattacharjee,...
Post on 29-Jul-2018
317 Views
Preview:
TRANSCRIPT
Troubleshooting in the UCCX Solution
Arunabh Bhattacharjee, Software Engineer, Customer Success
Customer Care Business Unit
BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4BRKCCT-1593
What’s most important is ..
Questions ?
Evidence …
Answers !!
• Problem Isolation
• System Monitoring Tools – CLI, CUIC & RTMT
• UCCX Engine - CUCM Interaction
• Cisco Finesse
• Live Data
• Database and Reporting
• HA Failover
• Chat and Email
Agenda
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Troubleshooting
7BRKCCT-1593
Understand the Problem Statement
“Configuration was done a year ago, and
now we are facing ..”
“Agents are unable to login.”
“Appadmin is not responding.”
“Agents are not receiving calls.”
“Database Replication Down !”
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Asking The Right Questions
• Configuration and Script Changes
• Recent Upgrades
• Firewalls, Browsers, ESXi, CUCM, Gateways, MS Exchange
• Patches (COP files)
• Failovers• Could unveil hidden DB replication issues, network issues
• Contact Volume Increases, performance issues
• Holidays, Certain Time of Day, etc…
8BRKCCT-1593
• WHEN? WHAT Changed?
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• WHO is affected by the Problem?
• Are other Sites/Locations affected by the same issue?
• WHAT ELSE in the environment is affected?
• ISOLATION – Breaking the problem down, pinning it to a single problem suspect.
• Identify the EVIDENCE needed to CONFIRM the root cause.
Asking The Right Questions
9BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
RTMT – Real Time Monitoring Tool
• Java based plug-in for UCCX. Installable on Windows or Linux
11BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
RTMT – System Summary
12BRKCCT-1593
Memory | CPU | Disk
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CCX Engine JVM HeapRTMT – Performance Charts
13BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CCX Database StatsRTMT – Performance Charts
14BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Catch Problems Before Users do !RTMT – Alert Central
15BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Several Ways to View LogsRTMT – Log Central
16BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Intermittent Issue?
17BRKCCT-1593
RTMT – Scheduled Trace Collection
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• CLI: show status
• CLI: utils service list
• CLI: utils diagnose test
• CLI: show network cluster
• CLI: show myself
• CLI: show process load [cpu|memory] [cont|num XX]
• CLI: utils uccx ?
• CLI: show uccx ?
• CLI: set uccx ?
19BRKCCT-1593
Command Line Tools – Basic Health Check
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20BRKCCT-1593
Command Line Tools – Tailing Logs• Currently active logs can be tailed at real-time
• CLI: show open files regexp [MIVR|MADM|MCVD|Desktop-webservices]
• CLI: file tail activelog <relative path from /common/log/taos-log location>
• file tail activelog <relative path from /common/log/taos-log location>
CLI: file tail activelog /uccx/log/MIVR/Cisco001MIVR186.log regexp SS_TEL
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Utilization Monitoring – License and Call Handling
• CUIC provides stock reports for this
• License Utilization Hourly report
• Application Performance Analysis report
• Application Summary Report
• Aborted and Rejected Report
22BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23BRKCCT-1593
License Utilization Hourly Grid and Chart Views
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24BRKCCT-1593
License Utilization Hourly Grid and Chart Views
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25BRKCCT-1593
Call Failures – Aborted and Rejected Reports
• Cause Codes :
• Reject - TRIGGER_MAX_SESSION
• Reject - REMOTE_TIMEOUT
• Reject - CHANNELS BUSY
• Aborted – Too many transfer failures
• Aborted – Max Steps Executed (1000)
Aborted Contacts
• Exception occurred in the script workflow – ABORT!
Rejected Contacts
• System reached maximum capacity – REJECT!
Reports give helpful contact details and
exact timestamps!
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
UCCX and CUCM Integration
27BRKCCT-1593
Communications Manager
ClusterFinesse Agent Desktops
PSTN
Router/GWUCCX
AD/LDAP Recording
SIP (5060)
SIP/H323/MGCP
(5060/1720/2427)
LDAP (389 | 3268)
SCCP | SIP (2000 | 5060)
CTI/JTAPI (2748)
AXL (443 | 8443)
CTI (12028)
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28BRKCCT-1593
UCCX Call flow – PSTN call flow
CallManager
ClusterSupervisor
Stations
Agent
Stations
PSTN
Router/GW
UCCX
12
2343
4 6
6
555
1. Inbound PSTN Call to VGW.
2. VoIP leg to CUCM.
3. CUCM routes call to RP registered by UCCX.
4. UCCX responds with REDIRECT request to
available CTI Port.
5. UCCX triggers application, instantiates and
executes script. Call is answered upon Accept
step executing.
6. UCCX provides scripted queuing treatment or
identifies an available resource and CONSULT
TRANSFER the call to the agents device.
Agent ICD Extension: 6000
CTI Route Point 8222
CTI Ports: 9988-9990
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Setting up the Logging Levels
Under Engine UCCX Engine logs (MIVR): There are multiple logging facilities. Each facility is a sub-component.
• SS_CM
• SS_RM
• SS_RMCM
• ICD_CTI
• SS_TEL
• JTAPI plugin logs: JTAPI logging is not enabled by default.
Enable it upto Debugging level
JTAPI logs: Information about the interactions with CUCM
29BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
There is an easier way!
Logging Profiles. Specific logging facilities for specific problem descriptions.
• CCX Serviceability > Trace > Profile
30BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
UCCX Engine in Partial Service?
31BRKCCT-1593
Manager Manager? OR Subsystem Manager? Click to Expand !
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
UCCX Engine in Partial Service?
• If CM Telephony Subsystem is in Partial Service, check the Port Monitor Tool
32BRKCCT-1593
• If Application Manager, then check for an INVALID Application
Appadmin > Tools > Real Time Reporting > Applications
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
STATE LOG DESCRIPTION
RECEIVED Call.received() Call was received at a configured CTI Route Point
ASSOCIATED Call.associated() An available CTI port was assigned.
ACCEPTED Call.accepted() Call was routed to available CTI port
REJECTED Call.rejected() Call failed to be routed to a port
ATTRIBUTED Call.attributed() An application task was invoked to handle this call
ANSWERED Call.answered() Call was answered by the application (Accept Step)
ABANDONED Call.abandoned() Call was dropped by the caller or CTI port during consult
DISCONNECTED Call.disconnected() Call was terminated by the application
TRANSFERRED Call.transferred() Call was successfully transferred by the application
ABORTED Call.aborted() Call was terminated prematurely
Call States - Reference
BRKCCT-1593 34
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
//Call is offered to the CTI RP. cgn=1001 and cn=8222
2872713: Mar 28 09:15:31.264 IST %MIVR-SS_TEL-7-UNK:Route Connection=[8222::1/(P1-abhi_1)
GCID=(1,7057)->ACTIVE]->OFFERED, reason=1, Event= (P1-abhi_1) 7057/1 CallCtlConnOfferedEv
8222::1 [#827] Cause:100 CallCtlCause:100 CiscoCause:0 FeatReason:12, cause=100,
metacode=129, isMaster=true
//CTI Port selected
2872720: Mar 28 09:15:31.266 IST %MIVR-SS_TEL-7-UNK:Route Connection: [8222::1/(P1-
abhi_1) GCID=(1,7057)->ACTIVE]->OFFERED, CTI Port selected:
TP[id=1,implId=9989,state=IN_USE]
//Call disconnected at the CTI RP since it’s REDIRECTED
2872734: Mar 28 09:15:31.299 IST %MIVR-SS_TEL-7-UNK:RP[num=8222], conn=[8222::1/(P1-
abhi_1) GCID=(1,7057)->ACTIVE]->DISCONNECTED, event=(P1-abhi_1) 7057/1
CallCtlConnDisconnectedEv 8222::1 [#842] Cause:100 CallCtlCause:210 CiscoCause:0
FeatReason:6, cause=CAUSE_NORMAL[100], meta=META_CALL_REMOVING_PARTY[131]
//The call for the trigger 8222 is accepted at CTI port 9989
2872739: Mar 28 09:15:31.329 IST %MIVR-SS_TEL-7-UNK:CallID: 13, MediaID: 7057/1 Accepting
call for CTI Route Point: 8222 on CTI Port: 9989, ciscoCause=0
BRKCCT-1593 35
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
//Call is accepted and the call is ringing at the CTI port
2872740: Mar 28 09:15:31.329 IST %MIVR-SS_TEL-7-UNK:Call.accepted()
JTAPICallContact[id=13,implId=7057/1,state=STATE_ACCEPTED_IDX,inbound=true,App
name=app_new,task=null,session=19000000014,seq
num=0,cn=8222,dn=8222,cgn=1001,ani=null,dnis=null,clid=null,atype=DIRECT,lrd=null
,ocn=8222,route=RP[num=8222],OrigProtocolCallRef=0000000000001B9101685E5300000000
,DestProtocolCallRef=null,TP=9989]
2872741: Mar 28 09:15:31.329 IST %MIVR-SS_TEL-7-UNK:CallID:13 MediaId:7057/1,
TerminalConnection to Terminal: renji_9989 is RINGING, [9989::1/(P1-abhi_1)
GCID=(1,7057)->ACTIVE]->ALERTING
//TaskID created: 37000000025
2872744: Mar 28 09:15:31.330 IST %MIVR-SS_TEL-7-UNK:CallID:13 MediaId:7057/1
Task:37000000025 associated with Task ID: 37000000025
//Agent selected and the state is changed to RESERVED
2872829: Mar 28 09:15:32.657 IST %MIVR-SS_CM-7-UNK:RmCm contact 16784273[7057/1]
(12) .addAssociatedResource(abhi)
2872830: Mar 28 09:15:32.657 IST %MIVR-SS_RM-7-UNK:Rsrc: abhi New State:RESERVED
Old State:AVAILABLE Reason code:0
BRKCCT-1593 36
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
//Consult transfer starts
2872835: Mar 28 09:15:32.658 IST %MIVR-SS_TEL-7-UNK:CallID:13 MediaId:7057/1
Task:37000000025, transfer(6000, 12000, ACKNOWLEDGED)
//Consult call is offered at the agent's phone
2872869: Mar 28 09:15:32.690 IST %MIVR-SS_TEL-7-UNK:OrigCall=CallID:13
MediaId:7057/1 Task:37000000025, ConsultEvent= (P1-abhi_1) 7058/1
CallCtlConnOfferedEv 6000::4 [#881] Cause:100 CallCtlCause:100 CiscoCause:0
FeatReason:12
//Consult call ringing on agent's extension
2872893: Mar 28 09:15:33.271 IST %MIVR-SS_TEL-7-UNK:CallID:13 MediaId:7057/1
Task:37000000025, ConsultCallObserver RINGING
//Agent answers the call
2872937: Mar 28 09:15:38.227 IST %MIVR-SS_TEL-7-UNK:CallID:13 MediaId:7057/1
Task:37000000025, ConsultCallObserver TALKING - CallCtlConnEstablished,
[6000::1/(P1-abhi_1) GCID=(1,7058)->ACTIVE]->ESTABLISHED
BRKCCT-1593 37
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Common call failure scenario-1
Partition and CSS issue: it is important to note that the original calling number should be
able to reach BOTH THE CTI ROUTE POINT AND THE CTI PORTS.
In the MIVR logs, the following error message is seen:
302849: Dec 02 23:14:25.566 PST %MIVR-SS_TEL-3-ROUTE_FAILED:Route failed: All
Call
ids=JTAPICallContact[id=10,implId=1481/1,state=STATE_RECEIVED_IDX,inbound=true
,App name=sample_1,task=null,session=1000000011,seq
num=0,cn=7420,dn=7420,cgn=1009,ani=null,dnis=null,clid=null,atype=DIRECT,lrd=n
ull,ocn=7420,route=RP[num=7420],TP=null,List of Active
Connections=[7420::1/(P1-sydney_cti_1) GCID=(1,1481)->ACTIVE]-
>OFFERED,Extension=7410,Exception=com.cisco.jtapi.InvalidPartyExceptionImpl:
Attempt to redirect to an unknown destination,Failure reason= call will be
rejected, CTIERR_REDIRECT_CALL_UNKNOWN_DESTINATION=0x8ccc0034::Attempt to
redirect to an unknown
destination,Contact.Reject.reason=TRIGGER_FAIL,(SelectRouteTime,ObtainingIdleC
hannelTime,RedirectTime=0,0,16)
38BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
99066: Dec 01 09:22:54.039 IST %JTAPI-
PROTOCOL-7-UNK:(P1-10.106.112.196)
[MIVR_SS_TEL_TPG_ROUTE_EXE-44-24-
ROUTE_CALL_EV] sending:
com.cisco.cti.protocol.CallRedirectReq
uest {
sequenceNumber = 148
lineCallManagerID = 2
lineID = 30
callLegCallManagerID = 1
callLegID =
32362287
redirectAddress = 8000
unconditional = true
callingSearchSpace = 0
calledAddressOption = 0
redirectReason = 0
preferredOriginalCalledParty = 8000
modifiedCallingParty = null
99016: Dec 01 09:22:53.644 IST
%JTAPI-PROTOCOL-7-UNK:(P1-
10.106.112.196) received
Event:
com.cisco.cti.protocol.NewCall
Event {
eventSequence
= 395
lineCallManagerID
= 2
lineID
= 30
lineHandleSpecified
= true
deviceName
= ACD
callCallManagerID
= 1
callLegID
= 32362287
globalCallManagerID
= 1
callID
= 1464
callingParty
= 1009
calledParty
= 4000
originalCalledParty
= 4000
99067: Dec 01 09:22:54.211 IST %JTAPI-PROTOCOL-
7-UNK:(P1-10.106.112.196) received Event:
com.cisco.cti.protocol.NewCallEvent {
eventSequence = 397
lineCallManagerID = 2
lineID = 64
lineHandleSpecified = true
deviceName = CCG1_8000
callCallManagerID = 1
callLegID = 32362289
globalCallManagerID = 1
callID = 1464
callingParty = 1009
calledParty = 8000
originalCalledParty = 4000
state = 2
origin = 1
userData = null
auxiliaryData = 1
farEndpointCallManagerID = 1
farEndpointType = 1
reason = 6
callingPartyName =
calledPartyName =
BRKCCT-1593 39
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
The call from the CTI Route point to the CTI
Port is a REDIRECT, therefore the original
calling party should have access to both the
route points and ports.
BRKCCT-1593 40
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41BRKCCT-1593
Remember..
Avoid Unsupported
ActionsShared Lines
MoH
(Non-Multicast)
Steps in Script (Interruptible)
Select Resource
Timeout < RNA on IPCC Line Max Calls (2)
Busy Trigger (1)
2 Second Timeout before Accept (Script)
CUCM Providers Local, CTI
Ports, RPs in separate DPs
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Finesse
Finesse is the only available
Desktop in UCCX 11.6(1) release
-Browser-based
-Co-resident with CCX
- Open REST APIs
- Customizable desktop
- Cisco and third party gadgets
- Integrated administration
- E.164 support
BRKCCT-1593 43
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Finesse health
• Finesse depends on the following services for its normal functioning :
• Cisco Unified CCX Engine Service
• Cisco Unified CCX Notification Service
• Cisco Finesse Tomcat (Tomcat Service exclusive to Finesse)
44BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45BRKCCT-1593
Checking Finesse health
Service status on the GUI shows ‘STARTED’ – does this mean Finesse is usable?
NO. Service state : STARTED = Finesse Tomcat status only
For actual status of Finesse:
https://<UCCX IP>:8445/finesse/api/SystemInfo
http://<UCCX IP>:8082/finesse/api/SystemInfo
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46BRKCCT-1593
Slowness? Check Finesse Health
• Finesse diagnostic portal:
• Tomcat Web Server stats
• CTI Connection stats
• Agent and call stats
https://host.domain.com:8445/finesse-dp/rest/DiagnosticPortal/GetPerformanceInformation
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Finesse Issues – Further Analysis
• From Real-Time Monitoring Tool (RTMT) collect logs for Cisco Finesse, Cisco Unified CCX Engine, Cisco Unified CCX Notification Service
47BRKCCT-1593
Finesse Logs
• Desktop-webservices, client-logs, realm log file for issues related to Finesse
Engine Logs
• MIVR logs,
• LIB_AXL: Axlclientlogs, for login issues
Notification Service Logs
• Needs to be explicitly enabled from CLI.
• For specific login issues, state change issues on Finesse.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failure Scenario – Finesse Monitoring Error
Problem: Monitoring Fails to Start
48BRKCCT-1593
CCX Engine Receives Request to Silent Monitor agent106CCX Engine log:%MIVR-SS_RM-7-UNK:Processing msg: CTIMgrTPCCReqMsg
Socket:Socket[addr=127.0.0.1,port=35064,localport=12028] invokeID:3233 Msg Type =
SUPERVISOR_CALL_REQ Details = length=57 type=SUPERVISOR_CALL_REQ,invokeId=3233
AgentConnectionCallID = -1, SupervisorConnectionCallID = -1, AgentConnectionDeviceIDType = 0,
SupervisorConnectionDeviceIDType = 0, SupervisoryAction = SUPERVISOR_MONITOR,
AgentConnectionDeviceID = 2002, SupervisorConnectionDeviceID = 2001, AgentID = null, AgentDevice
= 2002, SupervisorDevice = 2001
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failure Scenario – Finesse Monitoring Error
Problem: Monitoring Fails to Start
49BRKCCT-1593
CCX Engine detects BiB Is Enabled, Uses JTAPI to Begin Monitoring Call SetupCCX Engine log:
%MIVR-SS_RM-3-Initiating silent monitor request to JTAPI server...:'Initiating silent monitor
request to JTAPI server...‘
%MIVR-SS_RM-7-UNK:processCallCtlConnInitiatedEv featureReason:1257684:
%MIVR-SS_RM-7-UNK:RIMgrAddressCallObserver: CallCtlConnInitiatedEv received for call 16779233
[2017/1] and agent agent1
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failure Scenario – Finesse Monitoring Error
Problem: Monitoring Fails to Start
50BRKCCT-1593
JTAPI Shows the Supervisor’s Monitoring Call State Go Dialtone > Dialing > Disconnected with Error:JTAPI log:
%JTAPI-CTI-7-UNK:(P2-
rmcm){Line:SEP88755651168C:2001:NewYork:(1,100)|Call:[GCID=(2017/1),CID=19750048]}
CallStateChanged (V2) [state=DISCONNECTED cause=UNALLOCATEDNUMBER destType=UNKNOWN destCM=0
fwdDest=: SecStat=1 unCg=2001 unCd= unOrigCd= unLRP= cg=2001 …….
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failure Scenario – Finesse Monitoring Error
Problem: Monitoring Fails to Start
51BRKCCT-1593
CCX Engine Reports this Failure back to Finesse ServerCCX Engine log:
%MIVR-ICD_CTI-7-UNK:OutboundMessageprocessor : sending msg : { length=-1
type=CONTROL_FAILURE_CONF,invokeId=3233,failureCode=CF_INVALID_DESTINATION,errorCode=0,
text=Error from Supervisor Monitor request. Description: ICDJtapiCallControlChannel (monitor);
nested exception is:
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
High Availability
• LAN: Both nodes share AXL, Unified CM Telephony and RMCM Provider list configuration
• WAN: Both nodes have unique AXL, Unified CM Telephony and RMCM Provider list configuration
53BRKCCT-1593
HA over LAN (HAoLAN)
Primary CCX
CM Publisher CM Subscriber
Secondary CCX
HA over WAN (HAoWAN)
Primary CCX
CM Publisher CM Subscriber
Secondary CCX
WAN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54BRKCCT-1593
Failover Issues – Further Analysis
• From Real-Time Monitoring Tool (RTMT) collect logs for
• Cisco Unified CCX Cluster View Daemon (MCVD) logs
Engine Failure
Engine logs – MIVR logs
Database Failure
Online logs – CCX Database
Event Viewer logs
Application logs
System logs
Finesse logs
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55BRKCCT-1593
High Availability
• Primary lost connectivity
• Master Election initiated
• Master Election completed
• Failover completed
Scenario 1: Lost Connectivity
Secondary CCX Server
Engine
Database
Reporting
Finesse
Primary CCX Server
CVDM ?? S M?
!Engine
Database
Reporting
Finesse
CVD
Secondary CVD detects missing keep-alives from Primary CVD
%MCVD-CVD-5-HEARTBEAT_MISSING_HEARTBEAT:
CVD does not receive heartbeat from node for a long period: nodeId=1,dt=2049
.....
%MCVD-CVD-5-HEARTBEAT_MISSING_HEARTBEAT:
CVD does not receive heartbeat from node for a long period: nodeId=1,dt=10245
%MCVD-CVD-4-HEARTBEAT_SUSPECT_NODE_CRASH:CVD suspects node crash:
state=Heartbeat State,nodeInfo=Node id=1 ip=192.168.12.5
%MCVD-CVD-4-HEARTBEAT_SUSPECT_NODE_CRASH:CVD suspects node crash:
state=Convergence State,nodeInfo=Node id=1 ip=192.168.12.5
Master Election initiated on Secondary CVD%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_STARTED, name=Cisco Unified CCX Engine
7631: Apr 28 07:45:13.986 CEST %MCVD-CLUSTER_MGR-7-
UNK:JavaService66:
Cisco Unified CCX Engine on node 1 change master from true to false
7632: Apr 28 07:45:13.986 CEST %MCVD-CLUSTER_MGR-7-UNK:Post Master
Event: MASTER_DROPPED, name=Cisco Unified CCX Engine, node=1
Secondary CVD elects Secondary as Master%MCVD-CLUSTER_MGR-7-UNK:JavaService167: Cisco Unified CCX Engine on
node 2 change master from false to true
%MCVD-CLUSTER_MGR-7-UNK:Post Master Event: MASTER_ELECTED,
name=Cisco Unified CCX Engine, node=2
%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_COMPLETED, name=Cisco Unified CCX Engine
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56BRKCCT-1593
High Availability
• WAN network failure
• Missing Heartbeats detected
• Master Election performed
• Failover completed
Scenario 2: Partitioned Network – ISLAND MODE
Secondary CCX Server
Engine
Database
Reporting
Finesse
Primary CCX Server
Engine
Database
Reporting
Finesse
Primary CVD detects missing Heartbeats
and assumes Primary has failed%MCVD-CVD-5-HEARTBEAT_MISSING_HEARTBEAT:
CVD does not receive heartbeat from node for a long period: nodeId=2,dt=10197
%MCVD-CVD-4-HEARTBEAT_SUSPECT_NODE_CRASH:CVD suspects node
crash:
state=Convergence State,nodeInfo=Node id=2 ip=192.168.13.5
Secondary CVD performs Master Election%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_STARTED, name=Cisco Unified CCX Engine
%MCVD-CLUSTER_MGR-7-UNK:JavaService66:
Cisco Unified CCX Engine on node 1 change master from true to false
%MCVD-CLUSTER_MGR-7-UNK:Post Master
Event: MASTER_DROPPED, name=Cisco Unified CCX Engine, node=1
Secondary CVD elects Secondary as Master%MCVD-CLUSTER_MGR-7-UNK:JavaService167: Cisco Unified CCX Engine on
node 2 change master from false to true
%MCVD-CLUSTER_MGR-7-UNK:Post Master Event: MASTER_ELECTED,
name=Cisco Unified CCX Engine, node=2
%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_COMPLETED, name=Cisco Unified CCX Engine
CVD CVDM M S M?
!
Secondary CVD detects missing Heartbeats
and assumes Primary has failed%MCVD-CVD-5-HEARTBEAT_MISSING_HEARTBEAT:
CVD does not receive heartbeat from node for a long period: nodeId=1,dt=10242
%MCVD-CVD-4-HEARTBEAT_SUSPECT_NODE_CRASH:CVD suspects node crash:
state=Heartbeat State,nodeInfo=Node id=1 ip=192.168.12.5
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57BRKCCT-1593
High Availability
• WAN network restored
• CVDs detect dual Masters
• Master Election performed
• Failover completed
Recovery from Network partition
Secondary CCX Server
Engine
Database
Reporting
Finesse
Primary CCX Server
Engine
Database
Reporting
Finesse
Primary CVD detects Secondary and keeps Master%MCVD-CVD-7-UNK:Split after network partition is detected,new nodeId=2
%MCVD-CVD-7-UNK:Engine bestCandidate runs on nodeId=1
because primaryEngineComputerName=UC85CCXPRI
%MCVD-CVD-7-UNK:Master Cisco Unified CCX Engine conditional-Keep-
LocalMaster
from localServiceCisco Unified CCX Engine on node 1
Secondary CVD detects Primary and drops Master%MCVD-CVD-7-UNK:Split after network partition is detected, new nodeId=1
%MCVD-CVD-7-UNK:Engine bestCandidate runs on nodeId=1
because primaryEngineComputerName=UC85CCXPRI
%MCVD-CVD-7-UNK:Master Cisco Unified CCX Engine DropLocalMaster from
localServiceCisco Unified CCX Engine on node 2, conditional=true
Secondary CVD performs Master election and drops Master%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_STARTED, name=Cisco Unified CCX Engine
%MCVD-CLUSTER_MGR-7-UNK:JavaService167: Cisco Unified CCX Engine
on
node 2 change master from true to false
%MCVD-CLUSTER_MGR-7-UNK:Post Master Event: MASTER_DROPPED,
name=Cisco Unified CCX Engine, node=2
Secondary CVD completes Master election%MCVD-CLUSTER_MGR-7-UNK:Post Convergence Event:
CONVERGENCE_COMPLETED, name=Cisco Unified CCX Engine
CVD CVDM M M S?
!
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58BRKCCT-1593
Finesse Failover Behavior before 11.6
Scenario CCX HA Behavior Finesse Server
HA Behavior
Finesse Client
Behavior
Recovery
CCX Engine Failure Failover to HA node Failover to HA node Failover to HA node Finesse follows
Engine mastership
CCX Notification
Service Failure
No Failover Finesse goes Out of
Service
Sessions closed Finesse unavailable
until Notification
Service comes online
Finesse Tomcat
Failure
No Failover Finesse goes Out of
Service
Sessions closed Finesse unavailable
until Tomcat Service
comes online
Finesse Service OOS No Failover Finesse goes Out of
Service
Sessions closed Finesse unavailable
until issue fixed
Island Mode Both HA nodes
become Master
Both Finesse servers
In Service
Clients connect to
either
Clients reconnect to
Master node
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59BRKCCT-1593
Finesse Failover Enhancements (11.6)
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60BRKCCT-1593
Finesse Failover Enhancements (11.6)
Failure Scenario
Unified CCX HA
Behavior Cisco Finesse - Node 1 Cisco Finesse - Node 2
Unified CCX Engine fails over from
master to Non-Master node
Engine Mastership
changes from master
to non-master.
Return to IN_SERVICE as
soon as it connects to the
new master engine.
Return to IN_SERVICE as
soon as it connects to the
new master engine.
Finesse Service Out of Service
Engine mastership is
not affected OUT_OF_SERVICE
Finesse on Node 2 will
continue to be
in IN_SERVICE
CCX Notification Service Failure
Engine mastership is
not affected OUT_OF_SERVICE
Finesse on Node 2 will
continue to be IN_SERVICE
Island Mode
Both HA nodes
become Master
Finesse on Node 1 will
continue to
be IN_SERVICE and will be
connected to Engine on
Node 1
Finesse goes Out Of Service
and will return
to IN_SERVICE as soon as it
connects to the engine on
Node 2 which is also the
master
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62BRKCCT-1593
Live Data – How it worksLive Reports on Cisco Finesse
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63BRKCCT-1593
Key Flow Points
Engine• Publishes every 3
seconds into JMS
SocketIOConsumer
• Receives through JMS all the messages
• Creates Rooms.
SocketIODispatcher
• Dispatches the updates to subscribed Web Clients (Browsers)
CUIC Live Data
Streaming Source
(ONLINE)
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64BRKCCT-1593
Logging required
• Engine logs (MIVR) – ICD_RTDM, SS_RMCM, upto Xdebugging5
• SocketIO Service logs – DEBUG mode
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
UCCX Engine as Live Data Source
• Engine drafts JSON updates to JMS (ActiveMQ) every 3 seconds
558501: Jan 26 18:57:26.008 EST %MIVR-ICD_RTDM-7-UNK:LiveData:
Sending JMS message :
[{"id":"agent1","operation":"UPDATE","ResourceIAQStats":{"resourceI
d":"agent1","resourceName":"agent1","resourceState":2,"durationInSt
ateMillis":148093,"nHandledContacts":0,"nPresentedContacts":0,"avgT
alkDuration":0,"longestTalkDuration":0,"avgHoldDuration":0,"longest
HoldDurati……… }}]
65BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
UCCX Engine as Live Data Source
• For Event based Rooms, Engine sends empty heartbeats
558509: Jan 26 18:57:26.274 EST %MIVR-SS_RMCM-7-UNK:HeartBeatAction : Sending
JMS message : [] for destinations [topic://ChatAgentStats,
topic://ChatQueueStatistics, topic://AgentCSQStats,
topic://AgentStateDetailStats]
558510: Jan 26 18:57:26.279 EST %MIVR-SS_RMCM-7-UNK:HeartBeatAction : Sending
JMS message : [] SUCCESS
66BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SocketIO debugs
• The Socket IO Server receives messages from the JMS bus for the Topic.
• Here agent=agent1, JMS topic=ResourceIAQStats.
• Room name=ResourceIAQStats*agent1, Status=RECEIVED,
0000001076: 10.106.87.133: Jan 26 2016 18:57:27.007 +0530:
%CCBU_Camel (camel-3) thread #3 - JmsConsumer[ResourceIAQStats]-6-MessageProducer:
%[message={"id":"agent1","operation":"UPDATE","ResourceIAQStats":
{"resourceId":"agent1","resourceName":
"agent1","resourceState":3,.................
..
room_name=ResourceIAQStats*agent1][room_prefix=ResourceIAQStats][status=RECEIVED][topic=jm
s:topic:ResourceIAQStats]: Event Detail Trace
67BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SocketIO debugs
• Socket IO Server processes the messages and maps them to the appropriate rooms.
• Room name=ResourceIAQStats*agent1, Status=ENQUEUED
0000005789: 10.106.87.133: Jan 26 2016 18:57:27.007 +0530:
%CCBU_pool-19-thread-1-6-MessageDispatcher:
%[client_count=-
1][message={"id":"agent1","operation":"UPDATE","ResourceIAQStats":
{"resourceId":"agent1","resourceName":"agent1",".....
..
room_name=ResourceIAQStats*agent1][socket_io_server_type=WS/WSS][status=ENQUEU
ED]: Event Detail Trace
68BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SocketIO debugs
• SocketIO message dispatcher sending the update to clients
• The Message Dispatcher receives the discrete room named messages and transmits them to all the clients that subscribe to the room.
• Operation = UPDATE, Status=SENT
• 0000005791: 10.106.87.133: Jan 26 2016 18:57:27.008 +0530:
%CCBU_pool-19-thread-1-6-MessageDispatcher:
%[client_count=2][message={"id":"agent1","operation":"UPDATE","ResourceIAQSt
ats":{"resourceId":"agent1","resourceName":"agent1",...................
.....
[room_name=ResourceIAQStats*agent1][socket_io_server_type=WSS][status=SENT]:
Event Detail Trace
69BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SocketIO debugs
• 4. On the Browser Side logs (F12) we see the following:
Message received on
socket::{"id":"agent1","operation":"UPDATE","ResourceIAQStats":{"resourceId"
:"agent1","resourceName":"agent1","resourceState":3,"durationInStateMillis":
264720,"nHandledContacts":6,"nPresentedContacts":8,"avgTalkDuration":1165394
,"longestTalkDuration":2266049,"avgHoldDuration":0,"longestHoldDuration":0,"
avgHandleDuration":0,"avgWorkDuration":0,"totalTalkTime":6992368,"totalHoldT
ime":0,"maxReadyTime":10199923,"avgReadyTime":1156099,
The above message can also be tallied with Finesse side Client logs to check browser updates coming in.
70BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71BRKCCT-1593
Live Data – Checklist
• Synchronized
• Resolution Forward and Reverse
• Diagnostic test : utils diagnose test
NTP
DNS
• Resources, NIC = VMXNET3 adapter
• Sizing – Under Limit, Agents per Team!
OVA Profile
Sizing
• Lowercase
• Mismatch, Unsupported Algorithms
Hostname
Certificates
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72BRKCCT-1593
Remember.. Browser Versions
Browser Cache
(Incognito)
Supported Sizing Limits
Disable Compatibility
mode
Field Note Awareness
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Configuration Datastore
Historical Datastore
Repository Datastore
CUIC
UCOSCM DB
RHEL
CCX Application
IBM Informix
Finesse
CCX DB
UCCX DB
Platform DB
UCCX Databases
BRKCCT-1593 74
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Publisher Subscriber
UCCX Replication Types
BRKCCT-1593 75
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
utils dbreplication status – Provides the status of Platform DB replication.
CUIC and Finesse share the platform DB.
utils uccx dbreplication status – Provides the status of the UCCX DB replication.
All the CCX historical, configuration and repository data is stored on this DB.
76BRKCCT-1593
DB Replication States
CLI: utils network connectivity <hostname of secondary>
CLI: utils diagnose test
CLI: utils ntp status
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CCX Database Replication
• View from CCX Admin or CLI
• Connected / Active state shows healthy replication
• Dropped / Timed Out status indicates replication issue
77BRKCCT-1593
Tools-> Datastore Control Center->
Replication Servers
CLI Command:
show uccx dbreplication servers
admin:show uccx dbreplication serversThis operation may take a few minutes to complete. Please wait...
SERVER ID STATE STATUS QUEUE CONNECTION CHANGED-----------------------------------------------------------------------g_uc115ccxpri_uccx 1 Active Local 0g_uc115ccxsec_uccx 2 Active Connected 0 May 9 18:07:55
Command successful.admin:
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78BRKCCT-1593
Command to show the DB Replication Files
CLI: utils uccx dbreplication dump configfiles
File Purpose
/usr/local/cm/db/informix/etc/sqlhosts.uccx Entries for Informix SQLHOSTS for the UCCX Instance
/etc/hosts Entries for the hosts in the cluster
/home/informix/.rhosts Entries for replicate hosts
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CCX Serviceability - UI
Driven
Reset Replication
Enable/Disable
CDS/HDS
Synchronize Data
Command Line Interface
Driven
Reset Replication
forcedatasync
dbserver integrity
79BRKCCT-1593
Database Repair - Checklist
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKCCT-1593
Database Replication Reset
• Replication can be broken if Subscriber is unavailable for too long and send queues buffer is exceeded
• Typically 3-4 days (*can vary with load)
• Alert will be raised
Alert Raised!DBReplicationStopped
Tools-> Datastore Control Center->
Replication Servers
Issuing a Reset Replication causes the following to
occur:
1. Remove database replicationutils uccx dbreplication teardown
2. Setup database replicationutils uccx dbreplication setup
3. Perform data repair processutils uccx dbreplication repair all
80
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tools -> Datastore Control Center ->
Replication Servers
81BRKCCT-1593
Disable/Enable CDS/HDS
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CDS
RDS
HDS
CDS
HDS
Node 1 Node 2
Tools -> Datastore Control Center ->
Replication Servers
RDS
82BRKCCT-1593
Disable/Enable CDS/HDS
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CDS
RDS
HDS
CDS
HDS
Disable CDS/HDS
1. Tear down HDS replication.
2. Deactivate CDS on node 2.
Node 1 Node 2
Tools -> Datastore Control Center ->
Replication Servers
RDS
83BRKCCT-1593
Disable/Enable CDS/HDS
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CDS
RDS
HDS
CDS
HDS
Enable CDS/HDS
1. Setup HDS replication.
2. Merge HDS data.
3. Copy node 1 CDS to node 2.
4. Activate node 2 CDS.
Node 1 Node 2
Tools -> Datastore Control Center ->
Replication Servers
RDS
84BRKCCT-1593
Disable/Enable CDS/HDS
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85BRKCCT-1593
Common Scenario : Configuration IssueMaking Config Changes – Error occurred while saving
• Both DBs must be AVAILABLE!
• WORKAROUND?
• Temporarily Disable CDS on Subscriber
CCX Serviceability Admin -> Tools -> DatastoreControl Center -> Replication Servers
• Disable CDS and HDS
• Enable after connectivity to Subscriber is restored
• Enable CDS and HDS
CCX Administration (MADM) Logs:
%MADM-ADM_CFG-7-UNK:ICDServlet :: Exception occurred
The SUBSCRIBER node of the HA is not available
%MADM-ADM_CFG-3-ADM_EXCEPTION:Unknown ADM Exception:
Exception=java.lang.RuntimeException: The SUBSCRIBER node of the HA is not available
Only for when Subscriber is down.
If Publisher is down, no configuration
changes are possible.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CDS
RDS
HDS
CDS
RDS
HDS
Tools -> Datastore Control Center ->
Datastores
86BRKCCT-1593
Synchronize Data - GUI
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
CDS
RDS
HDS
CDS
RDS
HDS
•Merges a single datastore.
•CDS not eligible (no ER).
Tools -> Datastore Control Center ->
Datastores
87BRKCCT-1593
Synchronize Data - GUI
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Local DB
CDS
RDS
HDS
CDS
RDS
HDS
Remote DB
88BRKCCT-1593
Force Database Sync – CLI Command
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Local DB
CDS
RDS
HDS
CDS
RDS
HDS
Remote DB
Overwrite Target = Local!
89BRKCCT-1593
Force Database Sync – CLI Command
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Local DB
CDS
RDS
HDS
CDS
RDS
HDS
Remote DB
Overwrite Target = Local!
Cluster Reboot + Replication Reset
required!
90BRKCCT-1593
Force Database Sync – CLI Command
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91BRKCCT-1593
Database Integrity
utils uccx database dbserver integrity
• Uses informix “oncheck” to check database for inconsistencies
• System Catalog Tables
• Chunk and extents
• Root reserved pages
• Output logged. CLI: file view activelog /uccx/cli/DbServerIntegrity_xxxx.txt
• Repairs NULL data entries in Config Tables
• Output logged. CLI: file view activelog /uccx/cli/DbConfigIntegrity_xxxx.txt
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92BRKCCT-1593
Reporting Review - Datasources
CUIC Data Sources:
Publisher CUIC uses
Subscriber’s db_cra
Subscriber’s CUIC
Also uses Subscriber’s
db_cra
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Database Issues – Further Analysis
• From Real-Time Monitoring Tool (RTMT) collect logs for
• Cisco Unified CCX Database
93BRKCCT-1593
DB REPLICATION
uccx_repl_output_util.log
file for issues related to
database replication or at a
minimum gather these logs
for TAC
DB
online.uccx.log file for
issues related to database
service
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Reporting Review – User Accounts
• uccxhruser - ”Historical Reporting User”
• CUIC, Wallboards, 3rd Party Reporting Software, Workforce Management
94BRKCCT-1593
uccxwallboard
Wallboard User limited to
RtCSQsSummary and
RtICDStatistics tables
uccxworkforce: Workforce User used by Calabrio WFO.
Large queries on historical tables.
uccxhruser : Historical Reporting User for
CUIC co-resident and custom report access to
all historical tables.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Scenario: Reporting and Database Performance
• Historical Reports slow to execute..
• Overall high server CPU utilization..
• Sluggishness in the system..
• Appadmin is slow..
95BRKCCT-1593
Looking into processes
You’ll find ‘uccxoninit’ as the top process!
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Reporting and Database Performance
• Review the configured CUIC reports and their frequency
• Check the historical tables size
96BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Reporting and Database Performance
• If tables have millions of rows, review purge config:
97BRKCCT-1593
Default of 90 months can be reduced
Individual table purge is new in version 11
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• DBPerf Script:
• DBPerf log contains:
• Timestamps,
• Full CPU snapshot: top,
• Users: onstat –u,
• Top Session information: onstat –g ses 0,
• Memory: onstat –g mem,
• Disk: onstat –d,
• Locks: onstat- k,
Reporting and Database Performance
98BRKCCT-1593
2 parameters - total duration and interval at which to get data
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Internet DMZ CorporateMS Exchange
UCCX
Finesse Client
End User
SocialMiner
UCM
Voice Gateway
Chat and Email Topology
BRKCCT-1593 100
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Exchange-Finesse Email Architecture with Social Miner – note that the exchange can be in the DMZ too.
Cloud Mail
Customer chat
Social Miner
UCCX
ExchangeDMZIntranet
Finesse Client
1
2
3
4
5
From Customer to Agent
BRKCCT-1593 101
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Shared BOSH Connection
Interacts w/ Private REST APIs
HTTP/REST Traffic Proxied via Finesse Tomcat
Reply Template JSP loaded via SocialMiner Tomcat Chat XMPP Service
Interacts w/ Private REST APIs
HTTP/REST Traffic Proxied via Finesse Tomcat
Reply Template JSP HTTP/REST API Traffic
Hosted in SocialMiner
Hosted in UCCX
Server to Server
Communication
Customer Website
Exchange Server
XMPP
(BOSH)XMPPHTTP/S
(Agent)
HTTPS IMAPO
pe
nA
jax
Hub
Gadget to Gadget Communication
Gadget to Gadget Communication via HTML5
UC
CX
Socia
lMin
er
Cisco Tomcat
& Other
Services
SocialMiner
XMPP Service
Cisco Tomcat
& Other
Services
Cisco Unified
CCX
Notification
Service
Cis
co F
inesse T
om
cat
102
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Office 365 and Gmail Support
• Only Socks 5 protocol is supported
• Configure Socks Proxy in SM and enable it.
• There is a Office365 email feed status in SocialMiner:
If the feed is good, it shows green
Else, a red X is seen
Hovering over the X will show the
reason for the state
BRKCCT-1593 103
Object Missing
Object Mismatch
Object Normal
Not Configured: Email Server not configured
Not Applicable: Cisco Finesse not active
SocialMiner incompatible
Email CSQ’s not created
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SocialMinerMS Exchange UCCX Finesse Client
EmailFetch Emails
Notify New ContactNotify New Contact
Load Email UI
Get Email Body and Attachments
Fetch Complete EmailLoad Complete Email
Reserve Contact
Upload/Download Attachments / ReplyUpload/Download
Attachments / ReplyEmail Reply Contact Handled
Contact Handled
104BRKCCT-1593
Finesse Agent Email Message Flow
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Email & Chat Fault Tolerance
• SocialMiner does not support High Availability (HA)
• Fault Tolerance is provided via UCCX HA
• SocialMiner configured to communicate with both UCCX nodes
• When new contact arrives, both CCX nodes notified
• Master node queues the request
• Secondary node ignores request
• If UCCX nodes failover
• Chat: existing sessions terminated, new contacts queued to HA node
• Email: on desk emails requeued, new contacts queued to HA node
SocialMiner
UCCX
M
UCCX
S
BRKCCT-1593 105
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Reinject Email Contacts
UCCX SocialMiner Configuration Page
Email auto routed to Agent
When Agent goes to retrieve email,
network connection between SocialMiner and
Exchange broke
Email contact becomes parked in SocialMiner
Reinject Emails will send the emails back to
the Agent once error corrected
BRKCCT-1593 106
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Finesse Agent Email – Failure Scenario 1
• Denotes email deleted on Exchange Server
• Verify that there are no auto-routing rules that moved the email
• Verify CSQ alias folder has not been deleted or moved
BRKCCT-1593 107
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Finesse Agent Email – Failure Scenario 2
• Denotes a connectivity problem with SocialMiner Tomcat Service
• Verify SocialMiner is reachable
• Verify status of Tomcat Service
• Restart Tomcat Service
BRKCCT-1593 108
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 109BRKCCT-1593
Finesse Web Chat Message Flow
Web ServerSocialMiner
UCCXFinesse Client
Chat RequestNew Contact Notification
Contact Dialog Event
Open Reply TemplateAgent Joined
Reply Template
End SessionSession Complete
Session CompleteContact Dialog Event
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failure Scenario – Non-Voice Gadget
• Cisco Unified CCX Webservices or Non-Voice Subsystem offline
• Agent not configured but gadget displayed
• Controls not enabled – SocialMiner unreachable
BRKCCT-1593 110
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 111BRKCCT-1593
SocialMiner with UCCX - Checklist
Network
• NTP Synchronized.
• DNS - Forward and Reverse resolved
• Network Latency < 250 ms
• utils network host <IP/Hostname>
VM Settings
• ESXi – LRODisabled
• OVA Profile
• NIC (VMXNET3)
Certificates
• Hostnames in Lowercase
• SSL CCX and SocialMiner trust
relationship (tomcat-trust)
Firewall, Proxy
• Open ALL required ports
• Refer Port Utilization Guide
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 112BRKCCT-1593
SocialMiner Logs
• The Main SocialMiner Engine (Runtime)
• RTMT – SocialMiner Runtime
• CCBU-runtime.*.startup.log, Error-runtime.*.startup.log
Runtime Logs
• Main API logs for the SocialMiner REST API
• RTMT – CCP API Logs
• CCBU-ccpapi.*.startup.log, Error-ccpapi.*.startup.log
API Logs
• Public interfacing API (Incoming Chat, Facebook, Twitter poll)
• RTMT – CCP PublicApps Logs
• CCBU-ccppublicapps.*.startup.log, Error-ccppublicapps.*.startup.log
Public App Logs
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 113BRKCCT-1593
What’s most important is ..
Questions ?
Evidence …
Answers !!
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Please complete your Online Session Evaluations after each session
• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt
• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.
Complete Your Online Session Evaluation
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs : LABCCT-2011
• Tech Circle
• Meet the Engineer 1:1 meetings
• Related sessions
116BRKCCT-1593
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Call to action !
• UCCX solution health check (Once a month)
• UCCX EFTs (Early Field Trials) – Spark!
• UCCX Field Notices on Cisco.com
https://www.cisco.com/cisco/support/notifications.html
• Build your own checklists!
117BRKCCT-1593
top related