machine learning in the field: an end-to-end architecture
TRANSCRIPT
Machine Learning in the field: An end-to-end architecture for real-time monitoring for remotely deployed personnel
Richard Collins – Head of Product - Bodytrak
Gabriel Nepomuceno – Software Engineer - Microsoft
01/11/2018
Gabriel Nepomuceno
@gbico
“A computer in every desk and in every home”
1980
“Every desk as a computer”
2018
The ML Pipeline
Prepare Data Build & Train Deploy
Step 3: Deploy!
Docker Containers
Azure Kubernetes Service (AKS)
Azure Batch
Azure IoT Edge
Any other container host…
Azure Machine Learning
Databricks
Custom Infra
The Azure ML Deployment Pipeline
The AI lifecycle
11
Bodytrak Precise Physiological Monitoring
Remote monitoring
station
Cloud
Bodytrak in-ear vital signs
sensor
Physiological data
Health marker
intelligence & alerts
2 way communications
Data modelling and analytics
Confidential
Proprietary algorithms (embedded and cloud)
• Core Body Temperature (CBT)• Non contact non invasive continuous temperature monitoring• Heat stress alert
• Heart Rate Monitoring• Photoplethysmography PPG signal processing• Fatigue level monitoring based on HRV• Level of consciousness
• Physiological Strain Index (PSI)• Standard measure of physiological strain using CBT and HR
• Motion and actimetry• Man down/fall detection and alert• Inactivity monitoring• VO2 monitoring for a general indication of fitness
• Noise level metering• Decibel dB(A) measurement for indicating and alerting for excessive noise
Alerting for signs of heat stress or effects from heat exposure
Monitor body posture abnormal events such as a
heavy fall or prolonged inactivity
Monitor physiological strain and alert when dangerous
levels of strain are detected
Measure levels of environmental noise at the ear
for protection against NIHL
Future potential for fatigue monitoring and level of
consciousness
User privacy
• Bodytrak does not own any person’s physiological data
• No algorithm input data can be traced back to an individual record
• Full transparency is provided for every user
• Any algorithm or “marker” can be removed from a customer solution
• Full GDPR compliance
DemonstrationReal time physiological monitoring
Temperature sensor
Ear tip
Heart rate module
Speaker driver
Ear bud (S/M/L)
External noise metering
Bodytrak earpiece sensors and features
Cable tucked behind the ear
No external protrusion
from the ear
Cable line tucked under tunic
Configuration
Advanced posture monitoring & fall detection system
6 axis motion
sensor array
9 axis motion
sensor array
Head position
Sudden fall
detection
Inactivity
monitoring
Environmental
noise metering
Body position
A machine learning approach to fall detection
Hardware sensors Feature extraction Artificial Neural Network Fuzzy logic
Environmental
Motion
Amplitude
Phase
Energy
…
Neuron 1
Neuron 2
Neuron 3
Neuron n
w1b1
w2b 2
0-100%
92%
75%
84%
YES
Fall detected
Inactivity
Historic events
Fall confirmed
Cloud access options
• Android apk• Displays vitals and alerts direct to an Android smart device
• Web app• Remote access to the Bodytrak hosted web application server
• Web portal• Dedicated web access from a customer’s central command centre
• Cloud REST API• Direct integration to a 3rd party network
• Container solutions• Custom solution for hosting on customer cloud
• Air gapped cloud solutions
Cloud based real time monitoring and analytics
Machine Learning algorithmsNear real time biometricsData/analytics/reportingAPI
Gateway
Time critical data
LTE-M/NB-
IoT/LoRa
Post-operative patient
monitoring
Lone worker/Industrial workforce well-being
and protection
Long distance driver well-being
Soldier acclimatisation and well-being
Fire services life preservation
Pro sports fitness training
API
Gateway
Real-time display from Bodytrak Cloud
BLE -
LoRA
BLE -
LTE
WiFi -
Satellite
Real-time biometrics on Bodytrak application
Time critical data
BLE -
LTE
BLE -
DMR
BLE -
LTETime
critical data
Time critical data
Bodytrak Cloud
Cloud integration trial example for risk management
Machine Learning AlgorithmsReal time physiologyAnalytics Reporting
Neural network analysis
Bodytrak Cloud
Lone worker well-being and protection
Construction worker safety monitoring
Marine environment safety
Oil & gas offshore safety management
Wind farm maintenance
Risk management practice
Risk analysis and
assessment
Sector safety monitoring
and reporting
Professional services in insurance
Real time physiological monitoring
Customer reportingData & analyticsREST APICustomer
API
Customer Cloud
Bodytrak human vital signs monitoring customer use case
Machine Learning AlgorithmsReal time physiology
Analytics Reporting REST APIsNeural network analysis
Bodytrak Cloud LTE Base Station
(Private subnet)
UAV
Customer Cloud(Air gapped)
Real time vital signs monitoring
Real time vital signs monitoring
Bodytrak container
Core body tempHeart rate analysisPhysical Strain Index (PSI)VO2 monitoringFall detection alertReporting
Remote monitoring station
Mobile device
A Bodytrak scenario for real time algorithm processing
• SARS epidemic in the centre of London
• Monitor medical personnel in the field for early signs of the infection
• Use Bodytrak data to overlay a live heat map of the area where mission personnel are operating
• Use incoming data on infected individuals and location to assess immediate risk of the personnel
• Use the cloud to tune a classifier (or marker) in real time to predict the onset of infection before it becomes critical for the individual
A cloud based algorithm to predict infection
• 2 stage alert system• Core body vitals monitoring for any abnormal change (on device)
• A personalised indicator tuned to recognise the earliest signs of infection (cloud)
• Standard algorithm trained to understand the effects of CBT with heavy protective clothing
• Cloud algorithm using real time motion metrics, body temperature, respiratory rate and cardiac derived sensor data
Cloud based processing for real time algorithms
API Gateway
Postgres DB
ML data (input)
N Series GPU
Ingress web data
Web Host
Memory Cache
NGINX(API manager)
NTP Server
Events Hub(AMQP system)
Air gapped
networks
Resource Manager
Artificial Neural Networks
Algorithm functions
Tensorflow
Algorithm data
(output)
N
Archive DB
Live heat map of real time infection monitoring
Normal
Low
Current state
Predicted riskNormal
Low
Current state
Predicted risk
Current state
Predicted riskLow
Current state
Predicted risk
Normal
High
Normal
Low
Current state
Predicted risk
Normal
Session Feedback
Please rate this session in the Future Decoded app!
Microsoft UK AI Research Report
Download the AI Report at http://aka.ms/UKAIreport
Visit our Hands-on Labs on Level 3
Try technology out with on-demand labs and expert help
Go deep with Documentation
http://docs.microsoft.com
Things to do next
Thank You
www.bodytrak.co