deep learning and its practical applicationsdeep learning and its practical applications dr. r....

Deep Learning and its Practical Applications

Dr. R. Scott Starsman [email protected]

757-232-7043

An ISO 9001:2015 Registered,

CMMI Maturity Level 3 Company

Agenda

CONFIDENTIAL AND PROPRIETARY. Copyright 2018 Avineon, Inc. All rights reserved.

• Introduction to Deep Learning

• Deep Learning Implementation

• Opportunities in Deep Learning

2

Introduction to Deep Learning


• What is deep learning?

• What kind of problems can it solve and is solving today?

• Why is it growing so rapidly now?

3

An implementation of machine intelligence capable of learning input/output problems (mappings)

4

Unknown System

Inputs Outputs

• This discussion is focused on neural network implementations

• Often requires computing power and large input/output data sets

• Supervised learning requires these data sets to be properly ‘tagged’

Train model to emulate this system

What is Deep Learning?


Unknown System Inputs Outputs

Autonomous Car Video of road ahead, ultrasonic

sensors, maps, destination, etc.

Steering and speed control

commands

Medical Diagnosis Vital signs, symptoms, recent travel

history

Diagnosis and treatment

recommendations

Facial Recognition Image containing faces Identification of people matching

those faces

Natural Language Translation Text from Language 1 Translation into Language 2

Unknown System Inputs Outputs

Typical Problems Being Addressed

CONFIDENTIAL AND PROPRIETARY. Copyright 2018 Avineon, Inc. All rights reserved. 5

• Automated review of cybersecurity data and detection of intrusion attempts

o Outsider and insider threat

• Detection of drones from video

• Analysis of satellite imagery and identification of scenes of interest

• Facial recognition

o Matching surveillance to persons of interest

Potential C4ISR Utilization


• Availability of massive computing power

o Moore’s Law

o Graphics Processing Units (GPUs)

• Availability of massive data storage

o Large drives

o Large RAM

• Availability of large volumes of input and output data

o Digitization

o Internet

o Internet of Things

Growth Factors

7 CONFIDENTIAL AND PROPRIETARY. Copyright 2018 Avineon, Inc. All rights reserved.

• Neural network architecture

• Making neural networks “deep”

• Convolutional Neural Nets

• Recurrent Neural Nets

Deep Learning Implementation


• Inspired by biological brains

Dendrite

Nucleus

Soma

Myelin sheath

Schwann cell

Node of

Ranvier

Axon Terminal

Axon

Inputs Outputs

Neural Network Architecture (part 1)


• Perceptrons (connectionist approach)

…

W1

W2

WN

+

• First described in 1957 with the peak of popularity in the late 1960s

• Unable to solve simple classes of problems such as the XOR damped interest in these type of solutions

Inputs Outputs



• In the mid-1970s, modifications to the Perceptron was proposed that addressed its deficiencies

• This is the classic neural network

Inputs Outputs

Hidden Layer

The hidden layer allowed neural networks to solve arbitrarily complex problems



• Training a neural net requires many computations

o Using a trained neural network is much less demanding

• Neural networks can theoretically solve arbitrarily complex non-linear problems

• As problems grew more difficult, several issues arose

o Challenge 1 : Requires massive computing power

o Challenge 2 : Solving big problems requires large amounts of ‘tagged’ training data

• Moore’s law and the digitization of data have largely addressed these challenges

o Some algorithmic improvements were also made

• Deep networks may have 10, 20, or more layers

Making Neural Networks “Deep”


• Determine whether a series of packets are an attack or normal traffic

• 41 network connection features analyzed including:

Data from: NSL-KDD dataset from UNB

— Duration

— Protocol

— Bytes (In & Out)

— # Logins Attempted

— SU Attempted

— Various Errors (In & Out)

• Successfully classified 99% of network traffic

Cyber Events Confusion Matrix

Ac

tua

l

Predicted

Normal Attack

No

rm

al

Att

ac

k

Deep Learning Case Study


• Utilizes a convolution operation rather than the more traditional activation functions

• Particularly effective at consuming vast quantities of information and extracting features from it

• Often used in conjunction with traditional neural networks to process information and reduce it to a manageable set of features

• Frequently used to process images and videos

Convolutional Neural Networks (part 1)


… … Inputs Outputs

~ 50 layers

Basic Features Facial Features Faces

Convolutional Neural Networks (part 2)


• Automated Imagery Analysis

CNN Case Study


• Designed to handle sequence information

o Time signals (EKGs, seismographs, etc.)

o Sentences (sequences of words)

o Sounds

• Successfully used to perform functions such as

o Machine translation

o Medical monitoring analysis (EKG, EEG, etc)

o Music composition

• These networks tend to be very processor hungry

Recurrent Neural Networks (RNN)


• Insider Threat Detection

• Feature vectors encode normal user behavior

• RNNs detect anomalies

RNN Case Study

Diagram from: Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams


File Email Http

User 1 RNN

User 2 RNN

User 3 RNN

User N RNN

Batcher / Dispatcher

Feature Extractor

…

…

…

…

Raw “Events”

Anomaly Scores

Feature Vectors

Http Email

• Identifying opportunities

• Types of problems ripe for deep learning solution

• Deep learning resources

Opportunities in Deep Learning


• Typical opportunities include those with:

o Processes that require human intuition/knowledge

o Processes involving massive amounts of data

o Processes involving complex relationships between system inputs and outputs

• Input and output data can be:

o Numbers, text, images, sounds, video, graphs, charts, etc.

• Network design is impacted by the type of data but shouldn’t constrain the possibilities

Identifying Opportunities


• Cyber security log analysis

o Input logs and packet captures and detect intrusion/exfiltration attempts

• Automated text parsing

o Input requirements document and produce architectural artifacts

• Law enforcement and/or military intelligence

o Collect data and build connections between contacts of interest

• Insider threat detection

o Monitor multi-dimensional behavior patterns and provide alerts

Potential Opportunities


Deep Learning Resources


• DeepLearning.ai • Five courses on Coursera

• https://www.deeplearning.ai

• Machine Learning Mastery Blog and eBooks

• https://machinelearningmastery.com/

• Fast.ai • Online courses and discussion

• https://www.fast.si/

• Deep Learning with Python • https://www.manning.com/books/deep-learning-with-python

• Kaggle • Data science and deep learning community and competitions

• https://www.kaggle.com

• GAO AI Assessment • https://www.gao.gov/assets/700/690910.pdf

https://www.gao.gov/assets/700/690910.pdf

Deep Learning and its Practical Applications

Dr. R. Scott Starsman [email protected]

757-232-7043

An ISO 9001:2015 Registered,

CMMI Maturity Level 3 Company

Questions/Discussion

deep learning and its practical applicationsdeep learning and its practical applications dr. r....

Documents