we add value to your business - morris & opazoo... · we add value to your business who we are...

www.morrisopazo.com / [email protected] - Temuco - Santiago

We add value to your business

WHO WE ARE

Morris & Opazo is an Advanced AWS Partner, with more than 15 years of experience in Custom Software Development, and 4 years of experience operating with AWS Cloud-based services.

We are a company specialized in providing business solutions in the area of Information Technology

Our goal is to facilitate the adoption of modern technologies,that add value to our clients' business solutions



SKILLS OF OUR LEADERSWe are ready to face any great challenges that lie ahead

Experts in Cloud-based Technologies

Experts in Solutions Architecture

Experts in Agile Methodologies

Experts in Infrastructure

Experts in Data Science

Carnegie Mellon UniversityMaster of Information Systems ManagementBusiness Intelligence and Data Analytics



Big Data & Analytics: Social Networks Insights

SUCCESS STORIES

Big Data & Analytics: Rimac – Data Lake Aguas Altiplano - Data Lake Aguas Araucania – Image and Video Recognition using Machine Learning

Aguas Magallanes – Real-Time Sentiment Analysis in Social Networks

Aguas Chañar – Analyzing Call Center Calls with Machine Learning



● Proof-Of-Concept (POC) implementations.

● Training to learn more about the possibilities AWS Cloud can offer you.

● Optimize your monthly billing.

● Consulting on the many services of AWS with our experienced staff.

● Design and validate your solutions with our Certified Architects.

BENEFITS OF WORKING WITH US



BIG DATA

● What is Big Data?

● Am I in a Big Data scenery?

● How Big Data tools help me?

● What AWS services can I use in a Big Data scenery?



Big data is like teenage sex: everyone talks about it,

nobody really knows how to do it, everyone thinks everyone else is doing it,

so everyone claims they are doing it…

(Dan Ariely, Duke University)



is a broad term to describe such large or complex data sets that traditional tools and solutions are inadequate to process and perform analyzes.

Defining Big Data

Big Data

The Key Features of Big Data: The 3 “V”s

Volume Velocity Variety



The Features of Big Data

The three V’s

Volume Velocity Variety



Volume

Data is increasing at a fast speed

Terabytes of data Petabytes of data

Solutions must work efficiently in distributed systems and must be easily expandable to accommodate peaks in traffic



Velocity

Increased speed of users, devices, applications

75 billion connected devices by 2020

Solutions must be able to manage this speed efficiently, and the processing systems must be able to return results in an acceptable time range

MB/s is normal, GB/s is common

One million transactions per second



Variety

Solutions need to be sophisticated enough to manage all different types of data, and at the same time provide accurate analysis

Various data sets, multiple sources

Most sources are in the Cloud

'Legacy' systems are still present



The Evolution of Data Analysis

Descriptive

Why did "X" happen? The Descriptive Analysis uses data aggregation and data mining techniques to provide insight in the past to provide answers.

Descriptive Predictive Prescriptive




Predictive

What is the probability that "X" will happen? Predictive Analysis uses statistical models and forecasting technologies to understand what might happen in the future.





Prescriptive

What to do if "X" happens? This type of analysis uses optimization and simulation algorithms to advise possible results and answer "What should be done?"




- Tracie Kambies, Nitin Mittal, Paul Roma, Sandeep Kumar SharmaTech Trends 2017, from https://www2.deloitte.com/content/dam/Deloitte/au/Documents/technology/deloitte-au-technology-dark-analytics-061017.pdf

What is Dark Data?

In this age of technology-driven enlightenment, data is our competitive currency.

Buried within raw information generated in mind-boggling volumes by transactionalsystems, social media, search engines, and countless other technologies are criticalstrategic, customer, and operational insights that, once illuminated by analytics, canvalidate or clarify assumptions, inform decision making, and help chart new paths tothe future



Three Types of Risks of Dark Data

Regulatory RiskLeak or loss of sensitive information, latent data and Personal IdentificationInformation (PII)

Intellectual Property RiskFailure to protect Intellectual Property

Opportunity RiskLosing opportunities for improvement



The Value of Big Data

Big Data is not just about data, it's about the value that organizations can get from it and the real-life decisions that can be made based on this data.



Big Data Workflow

Raw data

Ingest / Collect Storage Process / AnalyzeConsume /

Visualization

Answers & Findings



Big Data Simplified Workflow

Ingest / Collect

Storage

Process / Analyze

Consume / Visualization

Big Data Solution



Big Data Workflow: Data Types

Data

Ingest / Collect Storage Process / Analyze


Answers & Findings

File Stream Transactional



Collect Data – Files Data

Data Files

CSV Files Logs Transcripts Pictures Audio Files

Data



Answers & Findings



Collect Data – Streaming Data

Datos

Ingesta / Recolección

Almacenamiento Procesamiento / Análisis

Consumo / Visualización

Respuestas & Hallazgos

Data Stream

Web Applications

Mobile Devices PortableApplications and Services

Industrial Sensors

Data



Answers & Findings



Collect Data – Transactional Data

Datos

Ingesta / Recolección

Almacenamiento Procesamiento / Análisis

Consumo / Visualización

Respuestas & Hallazgos

This type of data are usually managed by database services

Financial Logistic Buy ordersWork-related

dataShipping

informationDeliveries

Data



Answers & Findings



ACCELERATING THE BUILDING OF A DATA LAKE IN AWS

● What is a Data Lake?

● What are the benefits of a Data Lake?

● What AWS services can I use with my Data Lake?



A data lake is not intended to replace existing data warehouses, but rather to complement them. If youare already using a data warehouse, or you are looking to implement one, a data lake can be used as asource for both structured and unstructured data, which can be easily converted to a well-definedschema before being consumed in the data warehouse

A data lake is an architectural approach that allows you to store massive amounts of data in a centrallocation, so that they are easily available to be categorized, processed, analyzed and consumed byvarious groups within an organization.

Since data - structured and unstructured - can be stored as they are, there is no need to convert them toa predefined schema and you no longer need to know in advance what questions are going to be askedabout the data.

What is a Data Lake?

Decouple of Data Storage and Data Processing



A Data Lake should support the following capabilities

• Collect and store any type of data, at any scale and at low cost

• Secure and protect all data stored in the central repository

• Search and find the relevant data in the central repository

• Administration frameworks to govern data, including moving, transforming and cataloging data

• Quickly and easily perform new types of data analysis on data sets

• Advanced engines to consult and analyze data; and build, test and execute models in a variety of ways, including Machine Learning and Artificial Intelligence.



Benefits of a Data Lake on AWS

• Data Storage at Low Cost

• Security and Compliance

• Easy Collection and Data Ingestion

• Categorize and Manage Your Data

• Built for Analytics

• Artificial intelligence



Parquet is a file format for storing data in a columnar and compressed form, designed to query large amounts of data,independent of the processing platform, the data model, or the programming language. Compared with traditional non-refined formats such as CSV, JSON or TXT, Parquet can reduce the storage space required, improve the performance ofqueries significantly, and immensely reduce the costs of queries for AWS services, which are charged for the amount ofdata scanned.

Amazon tests comparing the CSV and Parquet formats using 1 TB of log data stored in CSV format against the Parquetformat showed the following:

● Space savings of 87% with Parquet (1 TB of log data stored in compressed CSV format against 130 GB with Parquet)

● A response time for a representative query in Athena was 34 times faster with Parquet (237 seconds for CSV vs. 5.13seconds for Parquet), and the amount of data scanned for that Athena query was 99% lower (1.15TB scanned for CSV).against 2.69GB for Parquet)

● The cost to run that Athena query was 99.7% lower ($ 5.75 for CSV versus $ 0.013 for Parquet)

Parquet has the additional benefit of being an open data format that can be used by multiple query tools and analytics ina data lake based on Amazon S3, particularly Amazon Athena, Amazon EMR, Amazon Redshift, and Amazon RedshiftSpectrum.

Cost and Performance Optimization



Cloud Architecture Best Practices

• Think in parallel.

• Losing coupling frees you.

• Deployment Automation.

• Automate the Infrastructure.

• Embrace hardware restrictions.

• Design for failure and Nothing will fail.

• Implement Elasticity

• Design stateless applications.

• Take advantage of different storage options.

• Build security in each layer.



Big Data Use Cases

Media / Advertising

• Processing of images and video.

• Digital advertising / advertising offers.

• Customer Support

Financial Services

• Portfolio / Trade Analysis

• Fraud Detection

• Risk Analysis

Oil / Gas

• Gas meters

• Pipe sensors

Consumer's Health

• Bio-sensors

• Clinical data analytics

Retail

• Recommendations

• Transaction analysis

Social Networks

• Demographics

• Usage Analysis

• Metrics in-game



Big Data Services Ecosystem



Building a Big Data Solution

DEMOReal-Time Sentiment Analysis in Social

Networks



Real-Time Sentiment Analysis in Social Networks



Data Source: Social Networks Stream

● A Stream is a flow of Data that, in a manner analogous to a television or radio signal,transmits constantly data packets allowing a continuous diffusion and reading of thecontent, without interruptions

● It is usually used for audio and / or video broadcasting, but can be used for any type ofcontent

● The data is transmitted only once, and only the clients connected at that moment receivethe transmission

● Popular social networks such as Facebook and Twitter provide an excellent opportunityto test the concept of Data Stream, as they are constantly producing new data, whenevera user writes a new post or message

● A client (program) connected to the Stream to capture the data and redirect it to anotherdestination (for example Kinesis) is known as Data Producer



Data Ingestion: Amazon Kinesis

● Amazon Kinesis allows to collect, process, and analyze in real time data from a DataStream

● It is the ideal mechanism for ingesting data that requires a quick reaction by the user ofthe information

● Examples of use for Kinesis are video, audio, application logs, website clickstreams, IoTtelemetry, among others.

● It allows operating in real-time working modes (Kinesis Streams) over a predefined timewindow (eg: 24 hours), and also batch processing (Kinesis Firehose) that can be definedaccording to the time and / or size of each lot



Serverless Processing: Lambda

● Lambda is a serverless computing service that allows executing code implemented in avariety of languages in response to AWS platform events

● An event indicates the occurrence of activity of some kind, which can be attended by theLambda function to achieve some objective in relation to the event (eg: someone uploadsa file to S3, and Lambda reacts by sending an alert to the administrator through a systemof notifications like SNS)

● Automatically manages the computing resources necessary to execute the implementedlogic

● Ideal to respond to real-time processing scenarios with variable workload, and whosetasks can be executed in a short time



● With Amazon Comprehend it is possible to analyze the content of text written by humans, for humans, and extract metadata indicating factors such as

○ Language○ Keyphrases○ Places

● Through the Comprehend APIs it is possible to perform this processing quickly, associated with processes in real time or near-real time

● Multiple languages supported○ German○ English○ Spanish

Natural Language Processing:

Amazon Comprehend

○ People○ Trademarks○ Events

○ Feeling○ Topic○ Etc...

○ French○ Italian○ Portuguese



Storage: S3

● Unlimited storage of objects, which allows storing, retrieving, consulting and analyzingunlimited amounts of data

● Designed to provide the highest durability and availability of data, while incorporatingthe most extensive list of compliance and security certifications, able to meet the mostdemanding requirements and regulations

● It offers different levels of storage, to operate according to the temperature of the data,thus achieving significant cost savings for cold data storage scenarios

● Widely supported by solution providers around the world● In the AWS ecosystem, it represents the cornerstone of the Big Data scenario, since it is

able to interact with all the data processing and analysis services, managing tocompletely decouple the storage of the computation. This service ultimately hosts theData Lake



BI Visualization: QuickSight

● Consumption and visualization service for Data Analytics and Business Intelligence● It facilitates the creation of graphics and panels that allow to deliver visualizations of the

information● Excellent integration with AWS data storage services, as well as traditional Databases and

Archives● Serverless platform, able to scale automatically to adapt to the level of use and activity



Analyzing Call Center Calls with Machine Learning

Image and Video Recognition using Machine Learning

EXTRA CONTENTS



Analyzing Call Center Calls with Machine Learning



Image and Video Recognition using Machine Learning

[email protected]

USA - CHILE

Find out more at

¡Visit our Web Site!