Project based Courses in Control Cyber Physical SystemCo-Design

Youngsoo KimDept. of Electrical Engineering

San Jose State UniversitySan Jose, CA

youngsoo.kim@sjsu.edu

Alan ChenDept. of Electrical Engineering

San Jose State UniversitySan Jose, CA

alan.chen01@sjsu.edu

Shrikant JadhavDept. of Electrical andComputer Engineering

North Carolina A&T UniversityGreensboro, NC

ssjadhav@aggies.ncat.eduClay S. Gloster

Dept. of Computer SystemsTechnology

North Carolina A&T UniversityGreensboro, NC

cgloster@ncat.edu

Thuy LeDept. of Electrical Engineering

San Jose State UniversitySan Jose, CA

thuy.le@sjsu.edu

Ping HsuDept. of Electrical Engineering

San Jose State UniversitySan Jose, CA

ping.hsu@sjsu.edu

ABSTRACTToday’s cyber-physical (CP) electronic system design in-cludes hardware and software with intelligent sensing andcontrols. Those CP future applications use high definitionimages, videos and context information with interactionsto the physical world through sensors and actuators. Se-nior level courses including embedded controls, FPGA de-sign, and server side programming have been developed andtested in senior and undergraduate courses and senior de-sign projects. The project based courses were designed andmotivated by the Urban Information Systems, project foremerging Smart Cities. This paper reviews specific curric-ula, lab organization, hardware and software platforms, andpreliminary results of offered courses in 2015-2016.

1. INTRODUCTIONTeaching embedded systems and its application to Cyber

Physical System (CPS) to Electrical Engineering undergrad-uates is challenging since this involves HW/SW co-designconcepts, interfaces with physical systems including sensorsand actuators, and cyber systems developments. In the lasttwo years, senior level courses were developed to build theseembedded system sequences for cyber physical systems atSan Jose State University and North Carolina A&T StateUniversity.An example CPS system is presented in Fig. 1. To in-

terface and receive the signal from the end node, the pe-ripherals are connected to micro-controllers and transmitted

1This work is partially supported by the U.S. Army grantsW911NF-13-1-0135 and W911NF-16-1-0217.

ACM ISBN 978-1-4503-2138-9.DOI: 10.1145/1235

to IoT routers and back-end servers accelerated by FPGAs.Communication protocols such as Zigbee and Bluetooth LowPower Energy are used to ensure that data are collected andprocessed in the backend server. In this example CPS designis motivated by smart city applications including Chicagoand Glasgow case studies. We are developing Smart cityCPS systems in the city of San Jose, CA by installing hun-dreds of sensor boxes on the city’s street light poles. Thesensor boxes are used to monitor air quality and are plannedto expand their use for measuring temperature, pedestrianflow, and noise. They will include camera systems to makeadditional informed decisions about allocating resources toSan Jose citywide (e.g. place new bus stops, detect trucksidling in one spot, and pedestrian count for social scientists).Previously, the project based curricular have been proposedin relevant papers especially FPGA platforms [2, 1, 3, 4]. Wecomplement those laboratories and project based offering bybuilding embedded sequences by providing application basedcourses sequences for Smart city CPS system.

This paper is organized as follows. Section 2 discussesa physical system education course which teaches sensors,interface and user input control using a traditional micro-controller hardware platform. It describes the detailed con-tents of our laboratory assignments design. Section 3 intro-duces the proposed cyber systems based project using Javabased acceleration on an FPGA platform. Section 4 presentsexperimental results and students responses compares themwith conventional methods. We conclude in Section 4.

2. PHYSICAL SYSTEMSThe course EE138 Embedded Control System has two

components consisting of lectures, and hands-on laboratoryassignments. The class was designed in flip class style sothat students can use open lab hours to complete hands-onlab assignments. Course topics are summarized in Table 1.

There are five laboratory assignments for the EmbeddedControl System course. Each assignment requires studentsto understand the hardware provided and to develop soft-ware to meet the objective of the lab. In each lab, studentsare paired in groups of two and are given a time constraint

Table 1: Course LaboratoriesBasic concepts Assigned labs

Program Design and Spec. Finite State Machine labSensor interface ADC, and GPIO labsFeedback System PID controller, actuator labs

Filtering digital filter labCommunication Serial comm. lab

for assignment completion. Before each lab assignment, stu-dents will be given lectures about the lab concept and a labmanual and sample code. The lab is split into five sections,allowing students to follow a time table for each section cul-minating with the expected completion date. At the end ofeach lab, the groups are expected to present their completedassignments along with any extra features they may add tothe assignment.

2.1 Hardware PlatformsWe developed an extension board that can be used with

ATMEL SAMD20 micro-controller board. The extensionboard has multiple components students can learn to useperipherals including various actuators input, and analogsensors as in Fig. 2. The focus is end system node develop-ments which can function as a sensor node, or an end systeminput control node in the CPS systems.

Figure 2: Custom Extension Board for CPS end-node design

The various types of components are provided includingmultiplexed seven-segment display, 4x4 keypad, potentiome-ter, speaker, LCD display, motor driver, two-phase light en-coder, and an H-bridge motor. The students will use theperipherals supported by the micro-controller to manipulate

the components using C programs. Peripherals includingADC or DAC can utilize the speaker and potentiometer todevelop a volume controlled speaker. The GPIO peripher-als are used to display values or symbols on the multiplexedseven-segment display or to detect a user’s input responsefrom the 4x4 keypad essential to an end-node user inter-face design. The extension board also provides multiple testpoints for debugging. By using an oscilloscope, multi-meter,or function generator, students have the ability to observe orinfluence any important port location using the test points.In the next section, the description of each laboratory as-signment is provided:

2.1.1 General purpose input/output labThe first lab introduces students the basic manipulation

of micro-controller I/O. With a general understanding ofGPIO, each group will be expected to develop a calculatorthat can compute the basic requirement of addition, subtrac-tion, negative and backspaced values using a keypad and amultiplexed four digit seven-segment LED display. Sincethe hardware circuit is multiplexed, students will need todevelop a program to detect a keypad response from a sin-gle row while powering and displaying a seven-segment digitdisplay. Once the code is setup to communicate to the hard-ware, the next step of the lab will be to display four uniquesymbols on each of the seven-segment digit displays as wellas to design a de-bouncing finite state machine for the key-pad before completing a software calculator. Upon comple-tion, each student should have a strong understanding of thehardware provided and how to communicate to the micro-controller through IDE, the integrated developer platformfor developing and debugging micro-controller code.

2.1.2 Analog interfacing labsIn the second lab, students learn to configure and enable

the ADC and DAC peripheral of the micro-controller. Thefirst section starts with the ADC. Each group will need toreceive and read a 12 bit voltage value within a range of 0to 3.3 volts. The voltage is supplied by the micro-controllerand will be changed by a potentiometer. Students will needto configure the gain, offset, reference voltage, samples, av-eraging and other register options. Once the peripheral con-nection is set up to read within the voltage range, the 12bit value must be displayed on the seven-segment displayin Volts. The challenging part of the lab is to accuratelydisplay and stabilize the voltage reading up to the thirddecimal place, when compared to a voltage meter. In thesecond section, the DAC peripheral is utilized to create asinusoidal frequency. The sinusoidal frequency is outputted

Figure 1: Example of CPS sytem

from the micro-controller to a speaker that can emit up to20 kHz. Students will generate a requested frequency fromthe instructor using Matlab. The frequency can be heard bythe speaker and observed with an oscilloscope to check foraccuracy.

2.1.3 Pulse width modulation labFor the third lab, students utilize the PWM peripheral to

accomplish two tasks. The first task of the lab is to gener-ate a sinusoidal waveform with a frequency, specified by theinstructor, using a low pass filter circuit. As a single PWMdigital signal is sent through the low pass filter, the capaci-tor in the filter charges and discharges for a longer or shortertime based on the duty cycle of the PWM. This phenomenonallows students to generate a sinusoidal waveform by vary-ing the PWM’s duty cycle from 0% to 100%, then back to0%. The second task is to create an open loop motor controlusing an H-bridge motor driver and potentiometer. The H-bridge motor drive circuit is controlled by two PWM digitalsignals, which will determine the direction and speed of themotor. Using the ADC and PWM peripheral, students areexpected to control the motor’s speed and direction usingthe potentiometer from the previous ADC lab and duty cy-cle of the PWM digital signal. An oscilloscope is utilizedto observe the difference in the two PWM signals in orderto correlate with the behavior of the motor. The PWM labteaches students the concept of PWM and how to controland move a motor.

2.1.4 Timer and interrupt labsThe fourth lab introduces the concept of interrupts to stu-

dents. The first interrupt is a timing interrupt, used to cre-ate a digital filter. A function generator is used to senda signal to the micro-controller while an oscilloscope helpswith the observation of the filtered results. Students are as-signed a specific low pass and notch filter frequency. Withthe frequency given, Matlab is used to help students converta continuous time function into discrete function. Given thediscrete time function, a segment of code can be generatedto create a digital filter. When the timing interrupt setupis complete, each group will used the ADC peripheral to re-ceive a signal, the digital filter equation to filter a specifiedfrequency, and the DAC peripheral to emit the filtered signalat a sampling timed interval. The second interrupt is the ex-ternal interrupt, used with an optical position encoder fromthe H-bridge motor used in lab 3. Students are expected toconfigure how an external interrupt is triggered and to incre-ment or decrement a count. The count symbolizes the posi-tion of the motor as it moves clockwise or counter-clockwise.With the ability to determine position and time using theexternal and timing interrupt respectively, students are ableto deduce how fast the motor is moving in terms of rotationper minute (RPM). This lab teaches students about impor-tance and the use of interrupts.

2.1.5 Motor controller labLab 5 requires the knowledge and experience of all pre-

viously discussed labs. Students are to develop a PID con-troller for the H-bridge motor used in lab 3.With the feedback of the motor completed in the previous

lab, the focus will be on the calibration of the PID equationas well as the functionality of the motor and keypad userinterface. The motor is mounted on a stand and is equipped

Figure 3: Motor controller load

with a load to measure whether the PID control exists andis well calibrated during final examination. Students areadvised of that each constant for the PID. There are fourlevels of grading for this lab. Each level represents a gradingtier. With the first tier representing a passing grade, everysequential tier would represent a sequentially incrementinggrade. In order to code and organize the different featuresof the motor’s and seven-segment display’s functionality, astate machine system will be implemented. The first tier hasfour states consisting of idle, acceleration, deceleration, andspeed control. The speed control state represents when themotor has reached the desired speed. The first tier repre-sents speed control at a single desired speed. In the secondtier, position control is added in order to create a five statemachine system. The third tier involves a better user inter-face and keypad command, allowing users to determine whatdirection, speed, or angle the motor should be moving at.The last tier can be considered a bonus tier for exceptionalstudents. Students who have reach this tier is suggested todo several functions such as implementing error detection,the position or measured speed can be outputted into ananalog signal, and the ability to set the speed and positioncontrol using the keypad or pot. In this lab, each group isprovided with the experience of calibrating the constants ofthe PID control, introducing students to the concept of con-trol theory. With many lines of code implemented into thefinal lab, students are exposed to the difficulties of codingorganization and efficiency, emphasizing the need for a statemachine system.

2.2 Students’ Background and ResponsesAll students attending this course should be an Electrical

Engineering student because the students are expected tounderstand basic circuit analysis and theory for the exten-sion board connected to the SAMD20J18 micro-controller.The C/C++ and microcontroller programming experiencecan help students communicate to the SAMD20J18 micro-controller, develop finite state machines, and create simplelogical bit-wise commands and algorithms for each labora-tory assignment. In the past, San Jose State University stu-dents, who have taken the EE138 course, typically have anlimited to intermediate mastery of the programmable lan-guage in C/C++, assembly, and microcontroller program-

ming. Since the course grading focuses on laboratory assign-ments, students react positively to the course material andlaboratory teachings. The course is a nice transition for stu-dents looking to further their knowledge of micro-controllerperipherals and embedded control system design. Overallstudents come out of the class with a satisfactory knowl-edge on micro-controllers and embedded system design fortheir future senior project.

3. CYBER SYSTEMSIn order to implement cyber systems for Smart Cities sens-

ing, students will have continued to work for a project basedsenior design after the physical systems course in section2. Summer research programs at San Jose State Universityand North Carolina A & T State University in the year of2015 were performed by faculty members. Undergraduatestudents collaborated to develop streaming video image ca-pabilities for the Smart City project. They implemented astreaming client/server program pair to display and processvideo in near real-time. The client and server communi-cate using the Real-Time Streaming Protocol (RTSP) andthe Real-Time Transfer Protocol (RTP). The activities ofthis project that were performed by students include: plan-ning, creating, testing, and deploying an FPGA Java Re-mote Method Invocation (RMI) system and Java Native In-terface (JNI) system. The students produced references thatcite any documents governing specifications or required pro-cesses.Additionally, students developed an FPGA server frame-

work which includes pre-synthesized coprocessor librariesthat store FPGA binaries for the application acceleration.User request specific coprocessor functions on a server sideFPGA through Java RMI for speed-up. The server executedthe specific functions on the hardware side and returns theresult value to the user. This version employed a byte-by-byte streaming I/O process to fill the sockets and transfer re-sult to its destination. The transport of big data sets includ-ing Ultra-High Definition (UHD) image pixels and charac-ter strings across the aforementioned client/server systems isimportant. Big data transfer utilities have been developed tomeet these demands in bioinformatics and other image pro-cessing research disciplines based on a client/FPGA serverenvironment for Smart cities.

Figure 4: The interaction between Smart CityServer and an FPGA board

First, students are required to go through an overview ofcore aspects of the Java programming language. Studentsare expected to learn key aspects of Java such as variabletypes, arrays, input/output using the scanner, conditionalstatements, etc. The students are given increasingly com-plex Java assignments for example implementation of theprogram that will provide the user a list of colors. The userwould then type in one of the colors given, and the programwould determine if the user had typed in a match. Next,

students are introduced to the idea of Java RMI. Java RMIis a feature of the language that essentially allows users tocall a function from a remote sever. The idea is implementedin four steps. First, a working architecture had to be set upso that a mathematical function could be called from onecomputer to another. The next three steps, which all tiedtogether, were to successfully send a text file, send an imagefile, and then stream a video/image file, respectively. In or-der to successfully accomplish these steps, students neededto build a library that allowed the user to implement a filetransfer framework easily, because since there was limitedtime to understand and begin testing.

A literature survey was conducted on big data transferprotocols to be used in conjunction with Java RMI. Theseprotocols are essentially libraries/programs that facilitatethe set-up process of Java RMI, along with additional fea-tures. The three main protocols researched were Java Re-mote Method Invocation Input/output (RMIIO), Fast DataTransfer (FDT), and Grid File Transfer Protocol (GridFTP),Pros and cons of each data protocol were analyzed based onour research, in order to determine the protocol to use. FDThad very clear instructions, worked on the first few attempts,and even offered various documentation so that it could beeasily modified to accommodate the level at which studentswere already working.

4. CONCLUSIONSThis paper presented a cyber physical systems course de-

velopment focused on control systems, and its flexible hard-ware implementation on an FPGA.We attempted our coursesequences can provide options for predominantly Undergrad-uate institutions leading to the efficient embedded systemscourse offering. This sequence can also be utilized for Re-search for Undergraduate students in summer semesters.Additionally, we show that our approach can complementexisting tools and methodologies and help instructors makea rapid and efficient cyber systems implementation for theirspecific use cases. This course sequence can be used directlyfor undergraduate colleges in electrical and computer engi-neering, information technologies disciplines.

5. REFERENCES[1] V. Bonato, M. M. Fernandes, J. M. P. Cardoso, and

E. Marques. Practical education fostered by researchprojects in an embedded systems course. Int. J.Reconfig. Comput., 2014:7:7–7:7, Jan. 2014.

[2] A. Kumar, S. Fernando, and R. C. Panicker.Project-based learning in embedded systems educationusing an fpga platform. IEEE Transactions onEducation, 56(4):407–415, Nov 2013.

[3] Y. Ugurlu and T. Nagano. Project-based learning usinglabview and embedded hardware. In System Integration(SII), 2011 IEEE/SICE International Symposium on,pages 561–566, Dec 2011.

[4] S. Wong, L. Carro, M. Rutzig, D. M. Matos, R. Giorgi,N. Puzovic, S. Kaxiras, M. Cintra, G. Desoli, P. Gai,S. A. Mckee, and A. Zaks. ERA – EmbeddedReconfigurable Architectures, pages 239–259. SpringerNew York, New York, NY, 2011.