collecting and analyzing data for valuable decision making...
TRANSCRIPT
![Page 1: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/1.jpg)
Stefano SCAMUZZOENGINEERING Group (www.eng.it)
Technical, Innovation & Research Division
Architectures & Consulting
SPAGOWORLD, Executive Board (www.spagoworld.org)
SPAGOBI, Ecosystem Manager (www.spagobi.com)
Collecting and analyzing data for valuable decision making in a service oriented business scenario
![Page 2: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/2.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Engineering Group
![Page 3: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/3.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Everyone needs services
![Page 4: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/4.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
But … what is a service ?
� The Core concept in a Service Oriented model is the concept of Service which is subject to
interpretations that make its meaning quite ambiguous
� Different points of view are possible:
�The Business Analyst point of view: a service is a functionality, part of a Business
Process
�The Designer point of view: a service is a software module with a precise functional
identity that can be invoked through a well defined and documented public interface
� The Core concept in a Service Oriented model is the concept of Service which is subject to
interpretations that make its meaning quite ambiguous
� Different points of view are possible:
�The Business Analyst point of view: a service is a functionality, part of a Business
Process
�The Designer point of view: a service is a software module with a precise functional
identity that can be invoked through a well defined and documented public interface
� We prefer to talk in terms of processes and components
�The Business Analyst analyzes the processes so as to detect the functional
components he needs to build the process
�The Designer translates the identified components into “services”
� We prefer to talk in terms of processes and components
�The Business Analyst analyzes the processes so as to detect the functional
components he needs to build the process
�The Designer translates the identified components into “services”
![Page 5: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/5.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
What methodology to use ?
Bottom-up
Leverage the existing applications and
start building services as an assembly
of existing components
benefits: time reduction
risks: the developed services have a
shorter lifecycle and demand frequent
maintenance and refactoring
Top-down
Start from business models to identify a
complete plan of the services to be built,
then iterate on their design and
implementation
benefits: tidy and complete architecture
risks: huge initial investment for analysis
an identification of the services plan
Meet-in-the-Middle (Agile Delivery):
Gradual definition of the services plan and parallel implementation of highest priority
services. When the service plan has sufficiently progressed, services can be revised to
make them compliant with the plan.
The goal: to balance the need for a tidy design with the demands of a rapid development
and time to market.
![Page 6: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/6.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Synthesis of the methodology
Enterprise Business Models
Definition
Enterprise Business Models
Definition
TechnologicalArchitecture
Definition
TechnologicalArchitecture
Definition
Service-Oriented Analysis
Service-Oriented Analysis
Service PlanDefinition
Service PlanDefinition
![Page 7: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/7.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
The Universal approach
� Core components (service bundles) for the integration and management of services; for the
definition of business rules and the realization of application modules in SOA architectures.
� By parameterizing each component, the solution gains a considerable modularity, allowing
to meet even the most complex requirements.
� Core components (service bundles) for the integration and management of services; for the
definition of business rules and the realization of application modules in SOA architectures.
� By parameterizing each component, the solution gains a considerable modularity, allowing
to meet even the most complex requirements.
Integration ProcessesProcesses
Innovation / reuse(FI – mobile)
Management& Intelligence
Adaptable Middleware ServicesAdaptable Middleware Services
IT & Businessgoals
Universal MiddlewareUniversal Middleware
IntegrationIntegration
ESBESBCooperationCooperation
ConnectorsConnectors
OrchestrationOrchestration
CollaborationCollaboration
BPMBPM
eFormeForm LifecycleLifecycle
PDFPDFBusiness RulesBusiness Rules
managementmanagementSOA ApplicationSOA Application
(Web (Web –– Mobile)Mobile)
GovernanceGovernance
Design Design
& develop.& develop.
Tools IDETools IDE
Process Process
& services& services
managementmanagement
Business Business
ActivityActivity
monitormonitor
![Page 8: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/8.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case history: a University
Defining the architectural framework to support new projects following a collaborative
development model based on modular software components that interact using consistent,
shared and integrated information.
The development of a service is split into phases: initial integration of the existing systems
followed by a re-engineering process
Defining the architectural framework to support new projects following a collaborative
development model based on modular software components that interact using consistent,
shared and integrated information.
The development of a service is split into phases: initial integration of the existing systems
followed by a re-engineering process
Requirements ?
New components ? Reuse ?
Where is the information ?
Deploying components on infrastructure nodes
New Platform� Unified Information
� Standards
� Methodology
� Application services
� Infrastructural services
� Connectors
� ARIS -> BPMN Monitoring & Management
![Page 9: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/9.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case history: a University
Explicitly expressedExplicitly expressed
� No application silos
� Interoperability
� Scalability
� Flexibility
� Standardization
� No application silos
� Interoperability
� Scalability
� Flexibility
� Standardization
� Workflow
� Document Management
� Unified data model
� SSO & Security/Profiling
� Governance
� Workflow
� Document Management
� Unified data model
� SSO & Security/Profiling
� Governance
� Iterative development - initial integration of existing applications
� ARIS processes modeled by the Customer
� Low organizational impact
� Performance
� Scalability (SLA)
� Availability
� Knowledge gain
Proposed
![Page 10: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/10.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case history: a University
Processes
& services
Unified data
model
Functional areas
![Page 11: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/11.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case history: a University
Config
Publishing rules
Deploy
Implements
Spagic Interface
Jar Library Transformation
OSGi Bundles
Develop new
component
Spagic Bundles
Spagic MetaDB
Registration
Spagic Service
Manager...
New
Requirement
Reusing Jar Library
Supported by Eclipse Visual Tools
R
![Page 12: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/12.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case history: a University
GOVERNANCE SERVICES
SERVICES
GOVERNANCE
INFRASTRUCTURE
GOVERNANCE
SECURITY
GOVERNANCE
KNOWLEDGE
MANAGEMENT
APPLICATION
GOVERNANCE
PROJECT
MANAGEMENT
Services and Contract
Manager
Users
Services/Applications
Analysis and
Development Team
G
o
v
e
r
n
a
n
c
e
Operations
![Page 13: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/13.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Governance requires BI / BAM
DataData
WarehouseWarehouse
MetaDBMetaDB
(real time)
Real time monitor for processes, services and relevant information
Error management and process restart
Support to decision making
Events generated by systems monitor
![Page 14: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/14.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Business Activity Monitoring
BAM systems detect events and make them
meaningful to business users
Complex Event Processing (CEP)
Data Source(s)
Data
DWH STORAGE
Console & Dashboards
Events
Alarms &
Notifications
RT & Historical
AnalysisEVENT DETECTION
EVENT VISUALIZATION
EVENT NOTIFICATION
BUSINESS USERS
![Page 15: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/15.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case History: BAM
BPM System
(TIBCO)
Data Sources:
SAP, CRM, back-
office, technical
DB (events)
Real Time Console & Dashboards
Other SpagoBIEngines
REAL TIME
HISTORICAL ANALYSIS
Web-based authenticated access
Event Manager
![Page 16: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/16.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case History: BAM
![Page 17: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/17.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Case History: BAM
Alarms for critical situations (bottleneck, SLA)
Overview and detail
![Page 18: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/18.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Data as a service
Structured Data
Unstructured Data
Sensor Data
New data types
Transactional
Time phased data
Text, Social
User feedbacks
RFID, GPSQR codes
Video, voice
Images
VOLUMEVOLUME
VELOCITYVELOCITY
VARIABILITYVARIABILITY
V V V V V V V V ……
Big DataBig Data
![Page 19: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/19.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Data as a service
![Page 20: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/20.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
(Big) Data as a service
InfrastructureOn Site
IaaS
Data ManagementCapture
Clean
Load
Store
View and AnalyzeExploration, navigation, presentation
Text analysis
Text mining
ApplicationCloud
SaaS
ETLETL
BIBI
SERVICESSERVICES
![Page 21: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/21.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Big Data Application fields
Brand ReputationBrand Reputation
Sentiment analysisSentiment analysis
Social listeningSocial listening
Social CRMSocial CRM
Complaint AnalysisComplaint Analysis
Competitive intelligenceCompetitive intelligence
![Page 22: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/22.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Big Data Scenarios
![Page 23: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/23.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Big Data approach
![Page 24: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/24.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open Data
Out-of-date
Inhomogeneous
Multi-format
Different detail levels
Dispersed
Growing
Public and private sectors
Structured
Measurable quality
It’s the momentum …
![Page 25: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/25.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open data: case history
![Page 26: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/26.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open data: case history
![Page 27: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/27.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open data: case history
![Page 28: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/28.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open data: case history
![Page 29: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/29.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
Open data: case history
USAGE SCENARIOS
in Services. . .
...t o help companies
per f or m market
r esear ches
in Trade. . .
...t o analyse demands
and emer ging t r ends
in Turism. . .
...t o compar e exist ing
r ange wit h available
of f icial st at ist ics
![Page 30: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/30.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
How open source helps
![Page 31: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/31.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.
New skills requires
![Page 32: Collecting and analyzing data for valuable decision making ...sesar.di.unimi.it/corsi/SOA/materiale/EngineeringSeminar2013.pdf · Stefano SCAMUZZO ENGINEERING Group () Technical,](https://reader030.vdocuments.net/reader030/viewer/2022040901/5e7101c3fb685f00716e0ce6/html5/thumbnails/32.jpg)
Copyright © 2013 Engineering Group, SpagoBI Competency Center. All rights reserved.