executive summary - instantcoke.files.wordpress.com€¦ · web viewmarketing and cost...

Image Recognition Algorithm for Instant Inventory of Coca-Cola Freestyle Cartridges

ECE4011 Senior Design Project

Coca-Cola Freestyle Instant Inventory Project Advisor: Linda Milor Project Sponsor: Riyon Harding

Peter Lyons Forrest Mercier Curtis Norris JingJing Pan

Quentin Samuels Ben Tran

Submitted April 14, 2017

Coca-Cola Freestyle Instant Inventory ii

Table of Contents

Executive Summary iii

1. Introduction 1

1.1 Objective 11.2 Motivation 11.3 Background 2

2. Project Description and Goals 2

3. Technical Specification 3

4. Design Approach and Details 7

4.1 Design Approach 74.2 Codes and Standards 84.3 Constraints, Alternatives, and Tradeoffs 9

5. Schedule, Tasks, and Milestones 11

6. Project Demonstration 12

7. Marketing and Cost Analysis 14

7.1 Marketing Analysis 147.2 Cost Analysis 14

8. Current Status 15

9. Leadership Roles 15

10. References 17

Coca-Cola Freestyle Instant Inventory iii

Executive Summary

The basic idea of this project is as follows: An algorithm will be developed using computer

vision and deep learning techniques in order to train a model to recognize various brand cartridges for

the Coca-Cola Freestyle drink machines. This model will then be integrated into an application for

consumer use in which they will take a photo of a rack or box of their current inventory. This picture

will then become the input for the model and the model will recognize each cartridge and count how

many of each brand of cartridge is in the photo. This inventory will then be displayed on screen for the

user to see. Once the inventory is complete, the user will then have the option to order more cartridges

based on the inventory that is displayed. This project should be done because of the current struggles

that Freestyle machine owners have in keeping a good inventory of their current products, leading to

over-ordering and spending too much money for cartridges at a time that may hurt them as a business

in the long run. The cost of this project is estimated to be up to $1000. The expected outcome of this

project is a trained model with a robust underlying algorithm that will recognize and document

cartridges based on their individual colors and logos. This model will then be used for an aesthetically

pleasing, user facing application to potentially deploy to Freestyle machine owners everywhere.

Coca-Cola Freestyle Instant 1

Recognition Algorithm for Instant Inventory of Coca-Cola Freestyle Cartridges

1. Introduction

The Coke Instant Inventory team is requesting up to $1000 of funding to to develop the recognition

algorithm for Coca-Cola Freestyle Cartridges.

1.1 Objective

Our objective is to build an object recognition system in order to automatically create inventory lists

from photos of Coca-Cola syrup cartridges on the rack. The syrup will be filled into Coca-Cola

Freestyle Dispensers to serve soft drink of a variety of tastes in the restaurants. Currently the inventory

is manually checked by Coca-Cola staffs on a daily basis, which brings high manual cost. Our

automatic recognition system will make such process much more efficient.

1.2 Motivation

As mentioned above, our sponsor from Coca-Cola expects this automatic inventory system to save

their human cost of updating inventory list. There is not a similar automatic system inside the

company, so they want us to initiate the project.


1.3 Background

Although there is no similar recognition system for syrup cartridges in used by Coca-Cola, outside the

company there are companies such as IBM [1] and NEC working on general product recognition

system, which can help retailers to track the numbers and positions of different products on the shelves

for inventory management. Although some researches are conducted in the academia [2], this

technology is more actively developed in the industry because of its bring profit. Both companies use

traditional object recognition approaches instead of deep learning. While deep learning approach

should be able to obtain higher accuracy in product recognition like it does in other tasks, the reason

behind the choices of both companies could be that the robustness of deep learning was proved in 2012

[3], while the commercial systems have been developed for a longer period and deep learning was not

a reliable solution back then.

2. Project Description and Goals

The fundamental of goal of the Coca-Cola Freestyle Instant Inventory team is to design and test an

mobile application whereby a user can take a device camera picture of a back room inventory rack

where Coca-Cola Freestyle cartridges are stored and translate the image into an inventory listing that

can be directly submitted to the central cartridge ordering location. The application will utilize a

phone camera, and contains image recognition software, and code translating the image into an

inventory listing in .xml format. Via their phone cameras, users can take a picture of an entire rack of

Coca-Cola Freestyle cartridge and upload the picture to the Instant Inventory application. Image

recognition software will recognize individual Coca-Cola Freestyle cartridges from unique

characteristics such as color and product font/logo. From the image, code formulated within the

Instant Inventory application will tally the number of each type of cartridge and translate raw data into


an


inventory listing in .xml format. The features of the applications include:

● Seamless upload of Freestyle rack images

● Sophisticated image recognition software from several angles

● Conversion of raw image data to an inventory listing in .xml format

● In-application option for sending inventory listing to supplier via email

3. Technical Specifications

Figure 1. Layout of various Freestyle Products and Labels.


Table 1. Specifications of Coca-Cola Freestyle Labels

Label Specification

Double Cartridge 4 x 3.5 cm

Single Cartridge 4 x 1.5 cm

Table 2. Color of Double Cartridge Coca-Cola Freestyle Labels

Label Specification

Coca Cola Red and White

Coca Cola Zero Black, Red, and White

Diet Coke Grey, Red and White

Caffeine Free Diet Coke Tan, Red and White

Dr. Pepper & Diet Dr. Pepper Dark Red and White

Barq’s & Diet Barq’s Root Beer Ice Blue and White

Sprite and Sprite Zero Dark Green, White, and Grey

Minute Maid Lemonade Light Yellow and Black

Seagram’s and Diet Seagram’s Ginger Ale Green and Grey


Mello Yello and Mello Yello Zero Yellow, Green, Red, and Black

Table 3. Color of Single Cartridge Coca-Cola Freestyle Labels

Label Specification

Ingredient Solution CC Lavender

HiC Flavors Purple, Yellow, Red, and Orange

Powerade Dark Blue and Black

Vitamin Water Purple and White

Raspberry (Coca-Cola Freestyle) Plum

Cherry (Coca-Cola Freestyle) Red

Peach (Coca-Cola Freestyle) Light Orange and Black

Strawberry (Coca-Cola Freestyle) Rose

Fruit Punch (Coca-Cola Freestyle) Light Red

Grape (Coca-Cola Freestyle) Dark Purple

Fanta & Fanta Zero Orange, Dark Blue, and Green

Dasani Blue, Tan, Gold, and Green


Lime (Coca-Cola Freestyle) Lime

Vanilla (Coca-Cola Freestyle) Light Yellow

Lemon (Coca-Cola Freestyle) Gold

Orange (Coca-Cola Freestyle) Orange and White

Table 4. Specifications of Interface for Mobile App

Device Specification

Apple iOS 10

Android Android 7 Nougat

Table 5. Platform Specifications

Standard Code

XML Python/MATLAB/C++


4. Design Approach and Details

4.1 Design Approach

The Design Approach we are taking for the implementation of this project is as follows: A

mobile application for both iOS and Android operating systems will be developed. This will be an

extremely simple user interface simply involving a screen where a client will take a photo of the

cartridge inventory using the camera on their phone, and from there the trained computer vision model

will do its work in analyzing the picture, generating a formatted list in both JSON and XML of all of

the products, their serial numbers, and corresponding quantities counted, which it will then display to

the user in an aesthetically pleasing manner. From there, the user will have the option to send this list

to the vendor so that the vendor can ship the correct number of cartridges to the user. A UML sequence

diagram of the process has been provided below. The model will be trained on another computer with

more processing power than a phone and then uploaded to the application for use.


Figure 2. Sequence Diagram for Mobile Application.

4.2 Codes and Standards

Due to the extremely software-centric nature of this project, no industry codes apply to this

project. One standard that applies to this project is the JSON standard [4]. A figure showing an

example of a JSON entry for the Coca-Cola Classic cartridge can be found below.


Figure 2. JSON Example of Cartridge Entry.

The second standard that will apply to this project is the Extensible Markup Language,

otherwise referred to XML [5]. A figure showing an example of the same entry above in XML can be

found below.

Figure 3. XML Example of Cartridge Entry

4.3 Constraints, Alternatives, and Tradeoffs

One of the biggest discussions that was had was whether or not to have the model be trained on

the device as more pictures are taken or have it be trained on a separate, more powerful device and

then have the model simply be used on the application. In the end it was decided to have the model be

trained off the device and then uploaded onto the application. This decision was mainly due to the fact

that there are a varying amount of cellphones around the market that all have varying amounts of

memory and processing power, so having the model be trained on the device would potentially crash


or slow down user’s phones, causing undue stress on the user. Another discussion that was had was to

format the results in either JSON or XML formats, but it was an easy choice to just have the option

for both due to the ease of converting between the two.


5. Schedule, Tasks, and Milestones

Figure 5. GANTT Chart showing project timeline.


Figure 6. PERT Chart showing project timeline.

6. Project Demonstration

The system will be designed to be portable and can be used in any room that stores Coca-Cola

Freestyle cartridges. The formal demonstration will consist of one person doing the following:

1. The user will open the Coca Cola Freestyle tracker application on his or her mobile device

and position the camera towards the items to be inventoried.


2. With the camera aimed at the inventory rack, the user will ensure that each item they want to be

inventoried can be clearly seen on the display of the device. If there is not enough space in the

storage room to stand far back enough to capture the entire cartridge rack, panorama mode may

be used to sweep capture a row at a time.

3. The inventory mode will then be activated to begin scanning each cartridge and sending

the data to an excel spreadsheet once the scan is complete.

In order to demonstrate the specifications, a mobile device running on Apple iOS and Android 7

Nougat will be used for image recognition and transferring data to an excel spreadsheet.

The specifications will be demonstrated as follows:

■ Cartridge Size Distinction: The image recognition algorithm will determine the width

between each case based on standard size dimensions provided by Coca Cola. The

cartridges will be housed in either single or double cartridges. Only specific flavors come in

double cartridges, which will be added to the algorithm for error detection.

■ Cartridge Color Distinction: Again, the image recognition algorithm will determine the color

of each cartridge and provide inventory data based off this information. Each flavor represents

a unique color or series of colors that may be used to identify the specific product.

■ XML Data Transfer: The inventory data will be formatted into an XML data file through the

application and can be transferred to other devices.


Figure 6. Coca Cola Instant Inventory Process.

7. Marketing and Cost Analysis

7.1 Marketing Analysis

Replacing traditional inventory methods of counting inventory, the Coca-Cola Instant Inventory

System will save the company both the time and manpower used in manually tracking backroom

inventory. The system will also reduce the amount of errors produced keeping track of the inventory,

therefore saving the company the potential cost of having incorrect amounts of product in its

dispensers each week.

7.2 Cost Analysis

Because the system will require little to no hardware, the expected cost to develop a prototype of the

Instant-Inventory System is very low. A graphics processing unit (GPU) will potentially be used in the

experiment to minimize the amount of time taken to perform experiments and trials. The expected cost

of one GPU is up to $1000. The main cost of developing the system would be due to the labor, which

includes having group meetings, coding the software, debugging the code, creating the apps, and

preparing the demonstration.


8. Current Status

As of the writing of this proposal, we have conducted the appropriate research on the approach

implementation of our algorithm and the following application. A concrete implementation is the next

step.

9. Leadership Roles

Application Architect: (Peter Lyons) The application architect will design, build, and test an application

whereby a user can seamlessly take a picture of a rack of Coca-Cola Freestyle cartridges and translate the

image into an inventory listing (.xml file) that can be directly submitted to an email address. Once the

application is built, the application architect is to monitor the application, understand all interactions

between components on a given level of abstraction, and define limitations and requirements for source

code.

Algorithm Scientist: (Forrest Mercier/JingJing Pan) The algorithm scientist will research and design

image recognition algorithms, or sequences of instructions that perform different types of tasks. The

algorithm scientist will devote time to determining methods in which less sophisticated, faster-running

algorithms can be substituted for more complicated algorithms.

Data Processing Analyst: (Ben Tran) The data processing analyst will interpret large quantities of

complex image data and identify particular trends or patterns within the data using statistical techniques.

The data processing analyst will provide ongoing reports as to new findings or suggested strategies.


Documentation: (Curtis Norris) The documentation chair will oversee the research, content planning, and

creation and maintenance of project documentation. The documentation chair will also ensure all

documentation satisfies technical specifications, and is completed and submitted in a timely manner.

External Relations: (Quentin Samuels) External relations will maintain consistent, professional, and

punctual communication with both project advisors, external project sponsors, and teaching assistants

(TAs). External relations will inform team members of the details of any project-related correspondence.

Project Coordinator: (Forrest Mercier) The project coordinator will gather and dispense needed

information and updates and coordinate the work throughout the project life cycle. The project

coordinator will organize team meetings, develop project strategies, monitor project progress, and oversee

the successful completion of the project.

Webmaster: (Quentin Samuels) The webmaster will create and maintain electronic tools/media used for

the project, including Google Drive, Google Docs, Google Slides, and T-Square. The webmaster will also

monitor the content and organization of any electronic media used.


10. References

[1] M. Marder, "Using image analytics to monitor retail store shelves," IBM Journal ofResearch and Development, vol. 59, no. 2/3, pp. 3:1 - 3:11, 2015.[2] M. Floerkemeier, "Recognizing Products: A Per-exemplar Multi-label Image Classification Approach," in European Conference on Computer Vision, 2014, 2014.[3] A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012.[4] JSON.org, Introducing JSON, [Online] Available: http://www.json.org/ [Accessed April 14 2017][5] W3, Extensible Markup Language, [Online] Available: https://www.w3.org/TR/REC-xml/ [Accessed April 14 2017]

executive summary - instantcoke.files.wordpress.com€¦ · web viewmarketing and cost...

Documents