smart-log: an automated, predictive nutrition …...overview of iot based smart-log system box-1...

27
Smart - Log: An Automated, Predictive Nutrition Monitoring System for Infants through IoT P. Sundaravadivel 1 , K. Kesavan 2 , L. Kesavan 3 , S. P. Mohanty 4 , E. Kougianos 5 , and M. K. Ganapathiraju 6 University of North Texas, Denton, TX 76203, USA. 1,4,5 University of Texas, Arlington, TX , USA. 2 Independent Researcher, Dallas, TX, USA. 3 University of Pittsburgh, Pittsburgh, PA, USA. 6 Email: [email protected] 1 , [email protected] 2 , [email protected] 3 , [email protected] 4 , [email protected] 5 , [email protected] 6 12th Jan 2018 Smart-Log Talk, ICCE 2018 1

Upload: others

Post on 22-Aug-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Smart-Log: An Automated, Predictive Nutrition Monitoring System for Infants

through IoT

P. Sundaravadivel1, K. Kesavan2, L. Kesavan3, S. P. Mohanty4,

E. Kougianos5, and M. K. Ganapathiraju6

University of North Texas, Denton, TX 76203, USA.1,4,5

University of Texas, Arlington, TX , USA.2

Independent Researcher, Dallas, TX, USA.3

University of Pittsburgh, Pittsburgh, PA, USA.6

Email: [email protected], [email protected],

[email protected], [email protected],

[email protected], [email protected]

12th Jan 2018 Smart-Log Talk, ICCE 2018 1

Page 2: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

2

Introduction

Novel Contributions

System level Modeling of Smart-Log System

Implementation and Validation

Conclusions and Future Research

Outline of the talk

12th Jan 2018 Smart-Log Talk, ICCE 2018

Page 3: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 3

Introduction

Nutrition imbalances in infants

Significance of Nutrition Monitoring System

IoT in Smart Healthcare

Page 4: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Nutrition Imbalances

• Imbalances in nutrition can occur either due to

undernourishment or over nourishment.

• Malnourished children tend to have weakened

immune system.

12th Jan 2018 Smart-Log Talk, ICCE 2018 4

Page 5: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Significance of Nutrition Monitoring System

• Healthy lifestyle starts from having a well balanced

diet.

• Many children tend to be fussy eaters where

aversion of certain food stays forever.

• In such scenarios, finding potential replacement of

the concerned food item is important to maintain a

well balanced diet.

12th Jan 2018 Smart-Log Talk, ICCE 2018 5

Page 6: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Internet of Things (IoT)

• The Internet of Things helps in connecting real

world sensor data to cloud based solutions.

• The IoT acts as a virtual brain

of wireless sensor networks

which can be realized as mixed

signal systems.

12th Jan 2018 Smart-Log Talk, ICCE 2018 6

Page 7: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Novel Contributions

• An automated solution which keeps track of the

amount of food consumed along with the nutrition

facts.

• A prediction based method to analyze nutrient

values of future meals.

• Feedback is taken from the user each time to

customize the user’s need.

12th Jan 2018 Smart-Log Talk, ICCE 2018 7

Page 8: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 8

System Level Modeling of Smart-Log System

Smart Sensor Board

Data Acquisition

Future Meal Predictions

Page 9: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 9

Overview of IoT based Smart-Log

System

Box-1 Box-2 Box-3

Box-4 Box-5 Box-6

Box-7 Box-8 Box-9

Piezo-sensor

Food

Product

Data logged into the cloud

Feedback

to the user

Camera to acquire

nutrient values

Page 10: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Smart Sensor board

• The proposed sensor system consists of

piezoelectric sensors paired with a micro-controller.

• Piezoelectric sensors generate equivalent voltage

signals for the applied weight or mechanical force.

• This equivalent voltage value can be read with help

of micro-controller which saves the obtained value

along with the timestamp.

System level Modelling of Smart-Log System

12th Jan 2018 Smart-Log Talk, ICCE 2018 10

Page 11: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

System-Level Modelling of Smart-Log System

Smart Sensor board

• Micro-controller helps in scheduling

• It helps in easier integration into wireless modules,

which helps in connecting the prototypes to the

internet for IoT based applications.

12th Jan 2018 Smart-Log Talk, ICCE 2018 11

Page 12: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Data Acquisition in Smart Log

System

Optical Character Recognition

Application Program Interface (API)

Database Approach

12th Jan 2018 Smart-Log Talk, ICCE 2018 12

Page 13: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

User takes a picture of the Nutrition

Facts using Smart Phone

Use Optical Character Recognition

(OCR) to convert images to text

Nutrition facts obtained through

OCR

User scans the barcode of the product

Using Open Application Program

Interface (API)’s and Database

approach, the nutrition facts are

acquired from Central database

Nutrient facts obtained through

API’s

Weight and Time information obtained

through Sensing Board

Calculate Nutrient Value of the meal

Save the Nutrient value, Weight, Time

of each meal for future predictions

Data Acquisition in Smart-Log System

12th Jan 2018 Smart-Log Talk, ICCE 2018 13

Page 14: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

System level Modelling of Smart-Log System

Future Meal Predictions

• The main concern in infants is the amount of food

wasted after each meal.

• The purpose of each meal may vary from person

to person.

• By considering the weight of the wasted food, the

corresponding nutrient value is calculated.

12th Jan 2018 Smart-Log Talk, ICCE 2018 14

Page 15: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Calculate weight of the left over food

Calculate the left over nutrient value

Ask the User regarding Purpose

of the meal

1. Rich in Carbohydrates

2. Rich in Protein

3. Rich in Fiber

4. Well-balanced Diet

Goal Achieved ?

Give feedback and Predictions for the

next meal

Calculated

Nutrient values for

the meal

Predictions for the Next Meal

12th Jan 2018 Smart-Log Talk, ICCE 2018 15

Page 16: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 16

Implementation & Validation

Sensor Design

Classifier evaluation using WEKA

Page 17: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

• The sensor system was implemented using Proteus

and the data analysis to build an efficient

prediction based system is done using WEKA.

• With help of a JAVA program, the input and output

of the classifier was displayed in a webpage.

Implementation of Smart-Log System

12th Jan 2018 Smart-Log Talk, ICCE 2018 17

Page 18: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

The sensor system is required to possess the

following functionalities:

• Calculate the total weight of the product placed on

the sensor board.

• Calculate weight of the product after the user’s

consumption.

• Log the weight each time in order to give better

predictions.

Implementation of Smart-Log System

12th Jan 2018 Smart-Log Talk, ICCE 2018 18

Page 19: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 19

ARDUINO based Sensor Module

Page 20: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 20

ARDUINO based Sensor Module

• ARDUINO board helps in faster prototyping and

can be easily programmed using C and C++.

• A LCD module was used in order to read the

variations in voltage value in accordance to the

weight of the elements.

• Data logging was achieved through the USB port.

Page 21: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 21

Data for the Sensor System

• A database of 8791 instances consisting of both

readily available and raw ingredients for baby food

was considered.

• This SR8 database is available through the US

Department of Agriculture website, and can also

be accessed using API.

• The unique NDB number is used to access the food

item.

Page 22: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 22

• Waikato Environment for Knowledge Analysis

(WEKA) helps in evaluating the system using

numerous algorithms.

• Whenever a user makes a data entry, an Attribute-

Relation File Format (.arff) is created, which serves

as input for WEKA.

Classifier evaluation using WEKA

Page 23: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Classifier evaluation using WEKA

Classifiers Mean

Absolute

error

RMS error Relative

absolute

error (%)

Root

Relative

Squared

error (%)

Naïve Bayes 0.0106 0.0179 2.65 3.98

Bayes Net 0.0052 0.0055 1.3108 1.2236

Naïve Bayes Updatable 0.0106 0.0179 2.65 3.98

Multilayer Perceptron 0.0149 0.0179 3.72 3.99

Decision Table 0.0322 0.0363 9.06 8.63

Simple Logistic 0.1192 0.1192 33.55 22.28

Naïve Bayes Multinomial Text 0.0354 0.0419 4.78 5.06

12th Jan 2018 Smart-Log Talk, ICCE 2018 23

Page 24: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Performance Comparison of Food recognition System

Research Works Food recognition method Efficiency

(%)

Chen et al. [23] Image based feature extraction 90.9

Maiti et al. [19] Image based feature extraction 96.2

Joutout et al [12] Image based classification

using multiple kernel learning

61.34

This paper Mapping nutrition facts to a

database

98.4

12th Jan 2018 Smart-Log Talk, ICCE 2018 24

Page 25: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Characterization Table for the Proposed

System

Characteristics

Sensor System ARDUINO UNO, Piezoelectric sensor and LCD module

Data acquisition OCR, API and Database approach

Data Analysis Tool WEKA

Input Dataset 8172 instances

Classifier Naïve Bayes

Accuracy (Worst case) 98.4%

12th Jan 2018 Smart-Log Talk, ICCE 2018 25

Page 26: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

Conclusion and Future Research

• This paper presents an autonomous food data logging

system.

• Two approaches for data acquisition have been evaluated

using WEKA in order to build a better prediction model.

• As a cost effective sensor system, with seamless data

logging method, this system can be an essential consumer

electronic device used in child care.

12th Jan 2018 Smart-Log Talk, ICCE 2018 26

Page 27: Smart-Log: An Automated, Predictive Nutrition …...Overview of IoT based Smart-Log System Box-1 Box-2 Box-3 Box-4 Box-5 Box-6 Box-7 Box-8 Box-9 Piezo-sensor Food Product Data logged

12th Jan 2018 Smart-Log Talk, ICCE 2018 27

Thank You !!!Slides Available at: http://www.smohanty.org