Model Predictive ControlFinal Report

Spring 2019

Sponsor:UTC Aerospace Systems

James Fanjunqiang.fan@carrier.utc.com

Contents

1 Summary 21.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Neuromorphic Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 32.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 MPC on Microcontrollers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 MPC on FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4 Neurobiology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.4.1 Hodgkin-Huxley Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4.2 Leaky Integrate and Fire Model . . . . . . . . . . . . . . . . . . . . . . . . . 62.4.3 Rate Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.5 Neuromorphic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.6 Neuromorphic Chips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.6.1 Memristor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Fall Semester Solution 113.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Initial Conceptual Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.1.2 Preliminary Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2 Neuromorphic Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.1 Conceptual Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.2 MATLAB Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1

3.2.3 Model Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2.5 Circuit Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Spring Semester Solution 214.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.1.1 Hardware and Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.1.2 System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.1.3 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5 Timeline 285.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6 Budget and Costs 296.1 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

7 Team Contribution 30

1 Summary

1.1 Model Predictive Control

Proportional-Integral-Derivative (PID) controllers have dominated the industry for a long timedue to being easy to understand and computationally cheap to implement and it is likely to staythat way for a while. One issue with PID control is that it can only respond to current eventsrather than future ones as it does not have what is called a prediction horizon, or how far a processlooks into the future. For example, a car with a PID controller would only begin to stop afterregistering that a stop sign has been crossed and not preemptively before. Another issue, or rathersomething that other control methods do better, is control of multivariate systems with coupledvariables. PID also cannot optimally control constrained systems [1].

Model Predictive Control (MPC) is a systematic approach to solving real-world problems.MPC is valued due to its ability to handle physical constraints and multivariate systems whilestill providing optimal control. The application of MPC is widespread and the industry hasimplemented it in engineering solutions for decades. MPC is known to provide improved controlperformance and is superior for controlling processes with many variables. For the task at hand, thegoal is to research MPC and its implementations on embedded microcontroller systems. Simulationand modeling will be done using tools such as MATLAB, more specifically the Model PredictiveControl Toolbox, in cohesion with Simulink. With proper models in place, the MPC system willbe realized in C programming and tested. Once simulation and modeling has been completed, thesolution will be implemented and tested on the acquired hardware.

2

1.2 Neuromorphic Computing

As the end of Moore’s Law is approaching, scientists have been looking for a better way to makecomputers faster and more energy efficient. Neuromorphic computing is the answer to this. Itis the idea of having a brain-like computer which mimics the way brains process and store in-formation. Nowadays, companies such as IBM and Intel have built neuromorphic chips. Thereare also memristor chips from KNOWM on the market that can be bought individually to buildneuromorphic systems. Our goal is to design a neuromorphic chip using the KNOWM memristorsto (1) control an electric motor directly and (2) to calculate the gains of a PID contoller to tunethem to there optimal condition.

In the following report we discuss the background research to understand neuromorphic com-puting. We then move to solutions section where we discuss the simulation conducted for eachobjective that we are trying to accomplish. We also talk about the potential circuit design for themodel. We then talk about the timeline that we are following to accomplish this project as well asthe budget that we are proposing. So far we have done intensive background research on neuronand synapse models, neural network models, as well as preliminary modeling of neural networks.Our next steps are to design and simulate our conceptual design and to look into subsystem designand component selection for the system. We are on track we our time line and these next goalsshould be achieved as expected.

1.3 Pivoting

As we began the spring semester, concerns about the project’s time constraints arose. The initialdecision to split the team in two and work separately on two different control system designswas hindering progress. After much consideration we decided to reconvene and focus our effortstowards the MPC branch of the project, as this was the approach specified by the sponsor. Whilethere was substantial work done on the neuromorphic design, the immense amount of researchrequired to properly develop the system would have taken the team over our allotted time. Oncethe path ahead was clear, the team chose to create a system that would control the temperature ofa modified toaster oven. We failed to gain access to a chiller, thus leading the team towards thisconcept for the project. The report will therefore consist of the MPC and neuromorphic progressprior to the pivot, as well as the finalized design and results from the new MPC system.

2 Background

2.1 Model Predictive Control

MPC is a well-established control technique across a variety of process industries ranging fromchemical plants to printing shops. Its success is due to its ability to handle multivariate systemswith inputs, outputs, and/or control restraints [2]. The core of MPC is to solve an optimizationproblem in real time while considering these restraints. It predicts future behavior of a process andsolves for the optimal control sequence to minimize an objective function, or cost function, undera certain horizon into the future [3]. MPC can be formulated as a quadratic programming (QP)problem as an optimization control problem that is solved at each time step. In the instance of a

3

linear, time-invariant, discrete-time system with affine input and state constraints and a quadraticcost function, the optimization control problem is equivalent to a QP [4]. The cost function forany given MPC problem can be separated into four main elements:

J(zk) = Jy(zk) + Ju(zk) + J∆y(zk) + Jε(zk) (1)

where zk is the sequence of state vectors from sample k to k + p − 1, Jy is the cost function forreference tracking, Ju is the cost function for manipulated variable tracking, J∆y is the cost functionfor change in manipulated variables, and Jε is the cost function for constraint violations[5]. Onemethod used in MPC for modeling the controlled plant is in the form

x[k + 1] = Ax[k] +Bu[k] (2)

y[k] = Cx[k] (3)

where x[k], y[k], and u[k] are the system input, output, and internal states respectively[5]. A,B, and C are the dynamics, control, and output matrices corresponding to the modeled plant.

2.2 MPC on Microcontrollers

Microcontrollers can be considered to be on-chip computers, as they contain all the necessarycomponents to warrant them as a computational system. These components include a centralprocessor, both volatile and non-volatile memories, input and output ports, analog-to-digital con-version, and even signal processing [6]. These devices are present in numerous semiconductorproducts across various disciplines. Their industrial applications include pharmacy, aerospace,automobiles, transportation systems, home appliances, and much more [6]. The technologicalprogress of these devices, coupled with their speed and functionalities, make them ideal for a widevariety of control applications. Low power consumption and high speed are usually traits deemednecessary for embedded systems, and microcontrollers have much to offer when it comes to thesechip attributes.

Previous research done using MPC on microcontrollers include micro aerial vehicle control,thermal process control, speed control of a DC motor, and auto code generation [7]. However,these topics barely scratch the surface of the limitless utilizations of embedded MPC. These papersbelieve the STM32 make of MCU to be a great fit for MPC implementation [2], [6], [7]. The STM32-F103RB model includes an ARM Cortex-M3 32-bit RISC core that operates at a maximum of72Hz. It also brings 128 KBs of flash and 20 KBs of SRAM with low power consumption andhigh performance. Success of past implementations, affordability, and high specifications make theSTM32 a plausible option for this experiment. An alternative that arose during the later stages ofthe project was the Arduino MEGA, but this idea was quickly disposed as the memory requiredfor the compiled CVXGEN code was too large for this board. The nRF52382 Development Kit isa board more suited to our purposes, with increased computational power and memory it becamethe clear choice for an affordable and powerful microcontroller.

4

2.3 MPC on FPGA

FPGAs are an example of reconfigurable hardware, and they are a promising platform for deploy-ing embedded MPC solutions to applications which require flexibility and short design cycle [8].While other works have been completed without the use of FPGA parallel computing capabilities,some researchers believe that there is opportunity for improvement if the parallel method canbe correctly integrated. FPGAs are good alternatives to application specific integrated circuits(ASICs) for high speed embedded MPC applications since they offer much reduced low-volumecost, greater flexibility, and a shorter design cycle, reducing the risk while still maintaining deter-ministic execution time and a high power efficiency.

Several papers implementing MPC on FPGA boards have used Xilinx’s Spartan series [9][3][8].One of the advanced models is the Spartan 6 FPGA SP605, which is a high performance boardwith fast speeds and ample memory storage. However, for the purposes of this project, explorationinto cheaper alternatives is preferred. This brings us to the Nexys 3 Spartan-6 Trainer Board. ThisFPGA board is significantly cheaper than the SP605, but still brings high performance with a clockspeed of around 500 MHz. This cost-efficient board will provide all the necessary specificationsfor MPC implementation while keeping the budget in check. FPGA-based systems have beenapplied in applications ranging from signal processing, image processing, to network processorsand robotics, and this is only naming a few of the many roles FPGA can fill [10]. However, giventhe time constraints after the pivot took place we only sought an MPC design on a microcontroller.But it is no doubt that FPGAs are viable options when it comes to MPC on embedded systems,and with more time a solution using such a device could have been deployed.

2.4 Neurobiology

Neurons, as depicted in Figure 1 below, are biological cells that have long “tentacle-like” projec-tions that allow neighboring neurons to interconnect and form networks. There are two types of“tentacles.” The first type is the axon which spreads outwards to connect to other neurons. Thesecond type is the dendrite which is shorter but acts as the interconnect to the axons of other neu-rons. Each neuron receives current pulses through its dendrites from the axons of firing neuronsin the previous layer. Once enough charge has accumulated for the membrane potential to reacha certain threshold, the neuron fires and continues the process to the next layer. In other words,axons are the outputs and dendrites are the inputs.

There are several popular models of neurons. The first model is the Hodgkin-Huxley model[13], [14]. This is the most accurate and biologically plausible model as it accounts for the specificionic currents flowing through the neuron as well as the gate dynamics controlling said currents.This is also the most computationally taxing method since it involves the most system dynamics.The second model is the Leaky Integrate and Fire Model [13]–[15]. This model only accounts forleakage current and is only modeled until the membrane potential of the neuron reaches a certainthreshold, after which the potential resets to a given value and the neuron is said to “fire.” Mostneuromorphic systems are based on this model due to its simplicity yet surprisingly good accuracy[13], [14]. There are also many other models with varying biological detail that will not be coveredwhich are discussed in [14].

5

Figure 1: A diagram of the neuron on the left, taken from [11], and a graph of the membranepotential of the neuron during a ”fire” on the right, taken from [12].

2.4.1 Hodgkin-Huxley Model

The Hodgkin-Huxley (HH) model of the neuron is composed of a capacitor and several seriesresistance-voltage source branches in parallel as shown in Figure 2 below. The capacitor representsthe capacitance of the cell membrane and each resistor-voltage source branch represents the sodium,potassium, and leakage pumps that control the potential of the cell membrane through ioniccurrents. The equation representing the HH model is

u(t) = Cm ∗dVmdt

+1

RL

∗ (Vm − VL) +1

RNa

∗ (Vm − VNa) +1

RK

∗ (Vm − VK) (4)

where ist is the stimulating current, Cm is the membrane capacitance, Vm is the membranepotential, GNa, GK , GL are the ion-pump conductances, and ENa, EK , EL are the membrane ionpotentials. The ion-pump conductances are also time variant and are defined as being betweentotally closed (0) and totally open (1). The equation representing the dynamics of each gate is

dm

dt= αm ∗ (1−m)− βm ∗m (5)

where m represents how “open” each ion gate is and αm and βm are constants governinggate behavior. This leads to the non-linearity in the neuron’s response which causes it to spike.Unfortunately, this level of detail also makes it difficult to simulate in software and emulate inhardware which is why most neuromorphic systems are designed around the leaky integrate andfire model.

2.4.2 Leaky Integrate and Fire Model

The leaky integrate and fire (LIF) model is a simplified HH model but with only the leakage currentas shown in Figure 3 below. The membrane capacitance from the HH model is kept but the threeresistor-voltage source branches are replaced with a single branch with a leakage resistance anda voltage source representing the resting potential of the membrane which sits around -70 mV.

6

Figure 2: The Hodgkin-Huxley model of the neuron.

As the name suggests, the model is simulated until the membrane potential reaches a certainthreshold, about -55 mV, after which the neuron is said to “fire.” The membrane potential spikesconsiderably and current pulses are sent to post-synaptic neurons, or neurons further down at thenext layer of the network. The equation defining the neuron potential is

ist = Cm ∗dVmdt

+1

R∗ (Vm − VR) (6)

where ist, Cm, and Vm are the stimulus current, membrane capacitance, and membrane potentialrespectively. R is the membrane’s leakage resistance and VR is the resting potential of the neuron.The second equation that defines the synapse current is

Isyn = gsyn ∗ e− tτsyn ∗ (Vm − Vsyn) (7)

where Isyn is the current that flows through each synapse, gsyn is the conductance-weight ofthe synapse, τsyn is the time constant of the conductance spikes in the synapse, and Vsyn is theresponse voltage of the synapse after a fire.

2.4.3 Rate Equations

The rate equations between two layers of a network is

τsyn ∗d~V

dt= −~V +W ~U +~iext (8)

where τsyn is the time constant of the neurons, ~U and ~V are the firing rates of the presynaptic

and postsynaptic layers respectively,~iext is the external stimulus if applicable, and W is the weightmatrix representing all of the synapses in between two layers. Note that ~U would be zero for thefirst layer since no layer precedes it and that ~U and ~V not absolute names for layers in the network,

7

Figure 3: The Leaky-Integrate-and-Fire model of the neuron.

i.e. Equation 8 above describes the behavior of a layer based on the layer preceding it. Equation8 above turns into the difference equation

~V [k + 1] =

(1− Ts

τsyn

)∗ ~V [k] +

Tsτsyn∗W ~U [k] +

Tsτsyn∗~iext[k] (9)

where Ts is the sampling period and ~V , ~U , and ~iext are discretized. ~U in this equation wouldbe zero for the first layer and the firing rate of the previous layer for the other layers.

2.5 Neuromorphic Control

Neural networks have been used for control systems and while the main research topic of thisgroup is not on artificial neural networks (ANNs), the supporting frameworks should be the sameas in [16] by Saiful and Sigeru, for example. They created an adaptive PID controller by using aneural network to self-tune a separate PID controller, which differs from our main goal of havinga neuromorphic system replace the entire controller but it is a step in the right direction. Ananalog-like spike-based PID controller was created on an FPGA where all inputs and outputs areencoded as spikes, acting to mimic neurological systems [17], [18]. Another team managed to createa controller for a robotic vehicle with obstacle avoidance and target acquisition using a ROLLSchip which is a mixed-signal neuromorphic chip with 256 integrate and fire neurons [19], [20].

2.6 Neuromorphic Chips

2.6.1 Memristor

Neuromorphic chips are integrated circuits developed to simulate the behavior of biological neuronsand synapses. Some have created chips that use fundamental VLSI blocks based on transistors andgates to model integrate-and-fire neurons with Hebbian learning [21] and Intel has produced it’sLoihi processor that models spiking neural networks [22]. IBM has a million-neuron neuromorphic

8

chip called Truenorth [23]. There are also a few independent fabricators producing neuromorphicchips such as the ROLLS chip [20].

There are also individual chips offered by KNOWM [24], a startup company selling memristordevices, which come as several configurations in DIP packages. The internal schematics of thechips KNOWM offers are in Figure 4 below. These chips are the most promising for this projectas they allow maximum customizability. KNOWM presents its chips for Anti-Hebbian/Hebbianmachine learning processes and provides simulation tools [25] but we plan on using the chips tophysically prove neuromorphic control.

Figure 4: Internal schematics of a few KNOWM chips, taken from [24].

A memristor is a device that changes its resistance as current flows through it. It is a two ter-minal device made out of semi-conductor materal sandwich between two ends [26]. The KNOWMMemristor is the main component of this design. It will be used for the Anti-Hebbian and Hebbianlearning of Spiking Neural Network (SNN). In the fall we thought about using a stochastic spikingneuron, which would use memristors as well, however it was different then the LIF Model. TheLIF model inputs are accumulated overtime until it reaches a threshold, while the stochastic spik-ing neuron ”incorporates a probability function that observes the spiking activities at the inputfrom the pre-layer neurons during a time step, and produce a spike with a certain probabilitythat depends on the weighted summation of these activities [27].” The following image shows thestochastic memristor neuron as well as a single time step.

Figure 5: (a) LIF Neuron Model with Unijunction Transistor.

9

Figure 6: (a) Representation of a Memristor Neuron (B) a single time step of the model [27].

The fact that memristors can be used to store memory it is possible for it to memorize thesequence at the write level time step, and the read time steps lets us know if there has been achange in the memristor activity and lastly the reset time step resents the memristor to its highestresistance at the off state. After careful consideration and more research, it was suggested to usean LIF model with a unijunction transistor (UJT) 5. The reason for that is that it would useless components then the stochastic spiking neuron, as the UJT could simulate the spike whenthe voltage reaches the specified threshold. To implement the neural network, we use a cross bardesign that was suggested in [27].

Figure 7: (a) Feedforward neural network representation of a cross bar design (B) Actual Crossbardesign that we will be mimicing [27].

The crossbar design is very simple to implement. We decide how many neurons we would likein the input layer, and output layer, connect each memristor to an input as well as an output.From there the feed forward design is implemented.

10

3 Fall Semester Solution

3.1 Model Predictive Control

3.1.1 Initial Conceptual Design

Earlier in the report the general structure of a MPC system is described. The conceptual design forthe desired solution will follow this fundamental idea. Inputs will be given to the model and outputswill be predicted. These outputs are then compared with a reference trajectory for the data, withthese results the system can then predict future errors. This information is consulted when theobjective function is considered during the optimization procedure. The physical constraints arealso applied to the data during this time, and when computations have finished the results arereturned as input for future iterations. A simplistic model of this process can be seen in Figure 8below [28].

In the case of our project, the variables we would need to track are heat, pressure, and mass forinputs, and temperature and level for outputs. We seek to track these variables in order to producea safe and efficient system that can correctly regulate control in a chilling device. Based off ofthe simple model previously mentioned, a design for the desired implementation was constructedusing the MATLAB Model Predictive Control Toolbox and Simulink. This system is more fleshedout and makes use of MIMO (Multi-Input Multi-Output) structure to appropriately handle thenecessary variables and constraints. Figure 9 shows the conceptual design for the system. It canbe noted that the inputs and outputs use step functions as reference trajectories as they simulatethe chiller activity. Overall the system makes use of trivial functions for optimization and thesewill be further refined as more simulation is done.

Figure 8: A simple model that describes the basic structure and processes of Model PredictiveControl.

11

Figure 9: The model constructed to represent the chilling regulation system.

3.1.2 Preliminary Simulation Results

Simulation of the previously mentioned model for the chiller system has been conducted. Thisanalysis consists of the tracking of the input and output variables as they are iterated throughthe MPC system. Here we look for stabilization and regulation of these variables as the modelpredicts future inputs and outputs while accounting for error and optimization. On Figures 10and 11 below, the data is compared with the reference trajectory signals. We can observe that thevariables roughly follow the step functions as they are refined by the MPC model. Certain errorsare visible and can be accounted for by the simplistic function that is directing the system. Oncethis algorithm is refined, the errors will be reduced and the data shall become more precise.

Again, the variables being tracked are heat, pressure, and mass for inputs, and temperatureand level for the output. It can be noted that as more time passes, the variables will be furtherassimilated and more accurately reflect the trajectory of the reference. However, for this simulationit was desirable to analyze the behavior of the system over a short period of time to observe theinitial response. Ideally the system should be able to nearly instantly accommodate for alterationsto the environment, thus regulation must be examined within close range. Additionally, higherfrequency ranges are being experimented with to check the system for robustness and speed. Asa whole, this model is a good starting place to begin further testing and changes as the projectcontinues.

12

Figure 10: The simulation results done in MATLAB. This chart displays the input variables asthey undergo iterative operations through the MPC model.

Figure 11: Simulation results of tracked output variables of the MATLAB model. As time pro-gresses, the outputs more closely reflect the reference which they are compared to.

3.2 Neuromorphic Computing

3.2.1 Conceptual Design

An LIF neural network (LIFNN) model is being used to simulate neuromorphic control in Simulink.The model is currently designed as a feed-forward network as shown in Figure 12 below. This model13

is called a feed-forward neural network as information propagates forwards as synapses fire, excitingsuccessive neurons further into the network. The inputs to the LIFNN model would generally bethe target trajectory speed, position, etc. as well as feedback from sensors measuring the plantstate. The network output could either directly drive the plant as shown in Figure 13 below orcontrol other variables in a system such as the PID gains as in Figure 14 below. The direct controlmodel has reference and feedback inputs and one output that goes to the plant. The goal is tohave direct neuromorphic control but PID tuning is also being explored to compare with Saifuland Sigeru’s results with artificial neural networks in [16]. The PID tuning model differs in thatit has three outputs corresponding to the PID gains KP , KI , and KD.

Figure 12: Common representation of a neural network. The colored circles are LIF neurons whereeach column represents a layer in the network and each connecting line a synapse. The first layertakes in the input stimulus and the last layer outputs its fire count.

Figure 13: System block diagram where a neuromorphic controller is used for direct control.

14

Figure 14: System block diagram where a neuromorphic controller is used to tune the gains of anembedded PID system.

3.2.2 MATLAB Models

The LIFNN model was implemented using MATLAB System Blocks in Simulink which allowsdirect application of MATLAB code. This was important as the LIFNN model was originallydesigned with MATLAB code to arbitrarily scale in size and the spike and reset logic was trivialin code. The code can be given on request. The LIFNN model takes current as input as it consistsof LIF neurons and outputs the current fire count of the last layer. The fire-count outputs of theLIFNN network are conditioned with a lowpass filter to reduce noise and the firing rate is foundby taking the derivative.

The MATLAB implementation of the system described in Figure 13 above is shown in Figure15 below. A motor-based plant was chosen for simulation as it is sufficiently fast to test and provethe speed of neuromorphic control and cheap to physically implement. The block labeled “IFModelML System” is the LIFNN controller which is immediately followed by the filter/derivative blockdiscussed in the previous paragraph for the fire-count to fire-rate conversion. Figure 16 belowshows the interconnected LIF neural network for the direct-control LIFNN controller. Note thatthe the “Hidden Layers” are arbitrary at this point as a proper analysis for implementing PID orMPC specifically on neuromorphic hardware has not yet been done.

Figure 15: Simulunk model of direct control using LIFNN model.

15

Figure 16: Neural network used for direct control LIFNN model.

The MATLAB implementation of the system described in Figure 14 above is shown in Figure17 below. Again, this system uses neuromorphic hardware to tune the gains of a traditionalPID controller. Figure 18 below shows the interconnected LIF neural network for the PID-tuningLIFNN controller. Also note that, just as in the case with direct control, the “Hidden Layers” forPID tuning are arbitrary at this point.

Figure 17: Simulink model of PID tuning using LIFNN model.

16

Figure 18: Neural network used for PID-tuning LIFNN model.

3.2.3 Model Parameters

Tables 1 and 2 below show the parameters used for simulating the LIFNN and motor model. TheLIFNN model values are more or less standard while the motor values are temporary until thechosen motor has been characterized.

Table 1: Parameters used in LIFNN model.Parameter Value Unit

Membrane Capacitance 250 [pF]Leakage Resistance 40 [MΩ]

Membrane Time Constant 10 [ms]Membrane Resting Potential -70 [mV]

Threshold Potential -55 [mV]Reset Potential -70 [mV]Refractory Time 2 [ms]

Base Synaptic Conductance 10 [µS]∆ Synaptic Conductance 0.01 [%]Synaptic Time Constant 50 [ms]

Table 2: Parameters used in first-order motor model.Parameter Value Unit

Motor Steady-state Gain 1.2 [ radsV

]Motor Time Constant 60 [ms]

17

3.2.4 Simulation Results

Figures 19 and 20 below show the pre-learning LIFNN 5 layer-by-5 neuron system response toan external sinusoidal stimulus on the first layer. The graphs show a clear connection betweenthe stimulus current and firing rates of the neurons and the simulation response to a step inputmatches the model.

Figure 19: MATLAB simulation results over 1 second of a 5 layer, 5 neuron stack neural networkbased on the LIF model without learning.

Figures 21 and 22 below show the preliminary modeling results of the MATLAB Simulinkmodels shown above. The results are from pre-learning models and are only static examples. Thatbeing said, the direct control system shows some level of tracking. The next steps from here onfor the LIFNN model is to add learning to the model such as Hebbian learning to match theneuromorphic hardware being designed.

18

Figure 20: MATLAB simulation results over 0.1 second of a 5 layer, 5 neuron stack neural networkbased on the LIF model without learning.

Figure 21: Simulink simulation results of direct control. The yellow line is the reference signal,the blue line the LIFNN controller output after processing (filter + derivative), and the red linethe motor speed.

19

Figure 22: Simulink simulation results of PID tuning control. System step response on the leftand PID gains on the right.

3.2.5 Circuit Design

We believe that we could build a Neuromorphic circuit using only Capacitors, resistors as wellas memristors as the main components. The circuit design mimic the simulation design wherewe have 2 input neuron, one representing the reference, the other the feedback from the outputneuron. Then 3 neurons in the hidden layer which would combine to one output neuron withthe proper firing rate. Between all of the neurons are memristors, which represents the synapsesweights in a neural network. The resistance of the memristors are tuned offline before we placethem on the circuit, and when we start testing the circuit the resistance of the memristors willadjust to an optimal state. From there the current from the memristors is summed up to enterthe neuron on the next layer. Unfortunately the circuit was not tested to know exactly if suchdesign would work, however it gives a general overview of how such a circuit could be design. Thefollowing figure 23 represents our preliminary design of our PCB circuit board.

Figure 23: Neuromorphic Chip Preliminary Circuit Design.

20

4 Spring Semester Solution

4.1 Model Predictive Control

4.1.1 Hardware and Software

The plant in our control system is a modified Oster toaster oven. It has been stripped away of it’soriginal temperature controls in favor of solid state relays that will handle the heating elements ofthe oven. A thermocouple have been added to keep track of the unit’s temperature and will beused to measure the current state of the system. Much of the outer shell has been removed andcovered in fiberglass strips to help preserve the internal heat. The inside has also been layeredwith reflective tape that will serve to retain the internal temperature. Figure 24 below showsthe modifed toaster oven. This alternative to a chiller was selected as it is in the same vein oftemperature control.

Figure 24: Frontal view of modified toaster oven showing reflective inner lining.

CVXGEN was the tool used to auto generate the code from solving QP-representable convexoperation problems. It provides a high-level description language that is used as the template forgenerating C code. The high-level abstraction allows easy redefinition of the MPC problem interms of state-vector size, prediction horizon, etc. The problem-description used in CVXGEN forthis project is shown in Figure 25 below.

21

Figure 25: Example problem description used for

22

The microcontroller that was initially used for implementation was an Arduino MEGA. How-ever, the code generation tool we used, CVXGEN, produced files that exceeded the program storageand/or the limitations of the MEGA. A more powerful microcontroller had to be used to reducethese limitations. The nRF52832 from Nordic Semiconductor was chosen as it provided 512kB offlash memory storage, double that of the Arduino MEGA, and 64kB of ram, or 8 times as much asthe Arduino MEGA. The nRF52832 also uses a more powerful ARM Cortex M4 processor clockedat 64MHz with a floating point unit (FPU) and digital signal processing (DSP) instructions thatallow for faster floating-point calculations, increasing calculation speed for the CVXGEN code.

4.1.2 System Identification

Without getting too far into thermodynamics, the model of a chamber separated from an externalenvironment by some sort of interface can be represented as a first order system. The chamber hasa heat capacity related to the occupying material (mostly air) which stores energy. The insulatedwalls of the chamber have a thermal resistance as the boundary between the chamber and theexternal environment. A direct analogy can be made to a standard RC circuit with a capacitorand a parallel leakage resistor. The capacitor stores charge while the resistor provides a leakagepath for charge to dissipate and the capacitor voltage decays. In this analogy the heat capacityof the chamber is the capacitor and the leakage resistor is the thermal resistance of the insulatedwalls. Knowing this, we are able to model the oven using the equation [29]

H(s) =K

τs+ 1e−sT (10)

where H(s) is the transfer function of the plant, K is the gain, τ is the time constant, and T is adelay. Figure 26 shows the constants as they relate to the step response. A delay is included in themodel to represent other unmeasured factors such the time it takes for the heating elements to heatup and the distance between the heating elements and the thermocouple. The heating elementshave are also nonlinear due to temperature coefficients that leads to varying power input dependingon their temperature. The constants in Equation 10 above can be found by mathematical analysisfrom a step input according to [29] by the equations below

K = y(t =∞) (11)

τ = 2(t2 − t1) (12)

T = 2t1 − t2 (13)

where y(t) is the temperature at time t, t2 is the time where y(t2) = 0.63K, and t1 is the time wherey(t1) = 0.39K. Figure 27 below shows the points corresponding to y(t2) and y(t1). We found thatour time constant τ was much larger than the time delay T which allowed us to approximate thetransfer function in Equation 10 above to just a first-order transfer function without a delay. Ourresults are shown in a later section.

23

Figure 26: Step response of a first order system with time delay.

Figure 27: Step response of a first order system with time delay. Times t1 and t2 and system statesy1 and y2 have been marked according to [29].

24

4.1.3 Design

Figure 28: Physical system block diagram.

Figure 29: Flow chart for software running on microcontroller.

4.1.4 Results

25

Figure 30: Measured step input system response of plant.

26

Figure 31: Step response of model using calculated constants overlaid on measured plant stepresponse.

Figure 32: MPC control with sampling period Ts = 4s, prediction horizon N = 41, for the giventemperature profile.

27

5 Timeline

5.1 Model Predictive Control

The timeline over the course of two semesters has changed drastically during the run of this project.The goals of the first semester were met but the later deadlines were altered to appropriatelyaccommodate for the pivot. In September the group was split into two teams to handle bothMPC on embedded systems and neuromorphic control. An introduction including backgroundwas completed for each subject during this time. October brought about system requirements anddesign for each form of control, and November saw modeling and simulation as well as subsystemdesign. In December the team collaborated on both a presentation and report for the Fall semester.

Figure 33: The year-long timeline for the MPC project.

During the beginning of the Spring semester the team came to the conclusion that more researchwas being done to design the systems than to actually realize them. Given the time constraintsplaced upon the project we found it necessary to pull together and focus on MPC for embeddedsystems. This shift led to the concept of control for a modified toaster oven which was developedin January with initial MATLAB simulations taking place in February. In March we acquiredour hardware and began configuring MPC onto the microcontroller. Finalization of the designand testing was done in April along with a final presentation of the project. A final report anddemonstration will be conducted on May 3rd.

28

6 Budget and Costs

6.1 Model Predictive Control

The budget for our MPC design is fairly conservative, many of the materials acquired were cheapand easily accessible. The shield wrap tape used to insulate the oven cost $13.28 and the externalthermal heat wrap covering the exterior cost $12.78. Both of these design choices were made tomaintain temperature within the oven and prevent heat from escaping to ensure accurate readingsfrom the the thermocouple. The two solid state relays used to control the heating elements cost$23.96 total and operate from 0 to 1. The Oster toaster oven costs $30.00 and serves as the base forour modifications and is the primary system being manipulated by our design. While we originallyplanned on using an Arduino MEGA to house the control system, we ended up opting for a morepowerful system with more storage. The result was the purchase of an nRF52832 Development Kitwhich came to a total of $39.00, it was necessary due to the large file size of the compiled CVXGENcost function optimization code. The K-Type Thermocouple is the device used to measure thetemperature of the oven, it cost $18.99. CVXGEN and MATLAB and both platforms that arefree for academic use, and therefore came at no additional cost. Although we came to the decisionto no longer pursue the neuromorphic side of the project, there were outstanding purchases madethat must be accounted for. Those being the memristors that were designated to be used in theneuromorphic circuit design, a total of twenty units were bought coming to a cost of $627.00.However, these will be seized by the university to do with as they please. Table 3 below illustratesthe total budget and costs for the project.

Table 3: Finalized Budget and Costs [MPC]Model Predictive Control Budget

Component Amount Cost Total CostShield Wrap Tape 1 $13.28 $13.28Thermal Heat Wrap 1 $12.78 $12.78Solid State Relays 2 $11.98 $23.96Toaster Oven 1 $30.00 $30.00nRF52832 Dev Kit 1 $39.00 $39.00K-Type Thermocouple 1 $18.99 $18.99CVXGEN 1 Student License $0MATLAB 1 Student License $0Memristor 20 $42.00 $627.00

Total $765.01

29

Advisor:Professor Abhishek Duttaabhishek.dutta@uconn.edu

Team:Indrid Xhuti [EE]

indrid.xhuti@uconn.edu

Vincent Turnier [EE]vincent.turnier@uconn.edu

Brendan Formanski [CmpE]brendan.formanski@uconn.edu

7 Team Contribution

Table 4: Team Contribution by sectionMember Contribution by section

Section Members NameSummary: MPC Brendan FormanskiSummary: Neuromorphic Computing Vincent TurnierBackground: MPC Brendan FormanskiBackground: MPC on Microcontroller Brendan FormanskiBackground: MPC on FPGA Brendan FormanskiBackground: Neurobiology Indrid XhutiBackground: Neuromorphic Control Indrid XhutiBackground: Neuromorphic Chips Vincent TurnierSolutions: MPC Indrid Xhuti, Brendan FormanskiSolutions: Neuromorphic Computing Simulation Indrid XhutiSolutions: Neuromorphic Computing Hardware Design Vincent TurnierTimeline Brendan Formanski, Vincent TurnierBudget Brendan Formanski , Vincent Turnier

References

[1] A. Dutta, C. Ionescu, M. Loccufier, and R. De Keyser, “Robust penalty adaptive modelpredictive control (pampc) of constrained, underdamped, noncollocated systems,” Journalof Vibration and Control, vol. 22, no. 2, pp. 549–558, 2016.

[2] A. K. Abbes, F. Bouani, and M. Ksouri, “A microcontroller implementation of constrainedmodel predictive control,” World Academy of Science, Engineering and Technology, vol. 80,pp. 731–738, 2011.

30

[3] C. Ekaputri and A. Syaichu-Rohman, “Model predictive control (mpc) design and implemen-tation using algorithm-3 on board spartan 6 fpga sp605 evaluation kit,” in InstrumentationControl and Automation (ICA), 2013 3rd International Conference on, IEEE, 2013, pp. 115–120.

[4] P. Zometa, M. Kogel, T. Faulwasser, and R. Findeisen, “Implementation aspects of modelpredictive control for embedded systems,” in American Control Conference (ACC), 2012,IEEE, 2012, pp. 1205–1210.

[5] N. Khaled and B. Pattel, Practical Design and Application of Model Predictive Control, MPCfor MATLAB and Simulink Users. Butterworth-Heinemann, 2018.

[6] A. A. Kheriji, F. Bouani, M. Ksouri, and M. B. Ahmed, “A microcontroller implementation ofmodel predictive control,” World Academy of Science, Engineering and Technology, vol. 53,pp. 68–74, 2011.

[7] I. Malouche, A. K. Abbes, and F. Bouani, “Automatic model predictive control implemen-tation in a high-performance microcontroller,” in Systems, Signals & Devices (SSD), 201512th International Multi-Conference on, IEEE, 2015, pp. 1–6.

[8] K.-V. Ling, B. F. Wu, and J. Maciejowski, “Embedded model predictive control (mpc) usinga fpga,” IFAC Proceedings Volumes, vol. 41, no. 2, pp. 15 250–15 255, 2008.

[9] M. S. Lau, S. Yue, K. Ling, and J. Maciejowski, “A comparison of interior point and activeset methods for fpga implementation of model predictive control,” in Control Conference(ECC), 2009 European, IEEE, 2009, pp. 156–161.

[10] M. He and K.-V. Ling, “Model predictive control on a chip,” in Control and Automation,2005. ICCA’05. International Conference on, IEEE, vol. 1, 2005, pp. 528–532.

[11] S. X. staff, Why are neuron axons long and spindly? study shows they’re optimizing sig-naling efficiency, [Accessed 20 October 2018], Jul. 2018. [Online]. Available: https : / /

medicalxpress.com/news/2018-07-neuron-axons-spindly-theyre-optimizing.html.

[12] Kurzweil, Kurzweilaccelerating intelligence, [Accessed 20 October 2018]. [Online]. Available:http://www.kurzweilai.net/ibm-scientists-emulate-neurons-with-phase-change-

technology.

[13] P. Dayan and L. Abbott, Theoretical Neuroscience: Computational and Mathematical Mod-eling of Neural Systems. MIT Press, 2014.

[14] C. D. Schuman, T. E. Potok, R. M. Patton, J. D. Birdwell, M. E. Dean, G. S. Rose, andJ. S. Plank, “A survey of neuromorphic computing and neural networks in hardware,” arXivpreprint arXiv:1705.06963, 2017.

[15] A. N. Burkitt, “A review of the integrate-and-fire neuron model: I. homogeneous synapticinput,” Biological cybernetics, vol. 95, no. 1, pp. 1–19, 2006.

[16] S. Akhyar and S. Omatu, “Neuromorphic self-tuning pid controller,” in Neural Networks,1993., IEEE International Conference on, IEEE, 1993, pp. 552–557.

[17] A. Jimenez-Fernandez, G. Jimenez-Moreno, A. Linares-Barranco, M. J. Dominguez-Morales,R. Paz-Vicente, and A. Civit-Balcells, “A neuro-inspired spike-based pid motor controllerfor multi-motor robots with low cost fpgas,” Sensors, vol. 12, no. 4, pp. 3831–3856, 2012.

31

[18] J. Liu, J. Harkin, M. McElholm, L. McDaid, A. Jimenez-Fernandez, and A. Linares-Barranco,“Case study: Bio-inspired self-adaptive strategy for spike-based pid controller,” in Circuitsand Systems (ISCAS), 2015 IEEE International Symposium on, IEEE, 2015, pp. 2700–2703.

[19] H. Blum, A. Dietmuller, M. B. Milde, J. Conradt, G. Indiveri, and Y. Sandamirskaya, “Aneuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor.,” inRobotics: Science and Systems, 2017.

[20] N. Qiao, H. Mostafa, F. Corradi, M. Osswald, F. Stefanini, D. Sumislawska, and G. Indiveri,“A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neuronsand 128k synapses,” Frontiers in neuroscience, vol. 9, p. 141, 2015.

[21] P. Camilleri, M. Giulioni, V. Dante, D. Badoni, G. Indiveri, B. Michaelis, J. Braun, andP. Del Giudice, “A neuromorphic avlsi network chip with configurable plastic synapses,” inHybrid Intelligent Systems, 2007. HIS 2007. 7th International Conference on, IEEE, 2007,pp. 296–301.

[22] M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi,N. Imam, S. Jain, et al., “Loihi: A neuromorphic manycore processor with on-chip learning,”IEEE Micro, vol. 38, no. 1, pp. 82–99, 2018.

[23] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L.Jackson, N. Imam, C. Guo, Y. Nakamura, et al., “A million spiking-neuron integrated circuitwith a scalable communication network and interface,” Science, vol. 345, no. 6197, pp. 668–673, 2014.

[24] Shop, https://knowm.org/product/, Accessed: 2018-10-1.

[25] A. Nugent and T. Molter, “From memristors to compositional machine learning: Explor-ing neuromemristive algorithmic abstractions with the knowm api,” in Proceedings of theNeuromorphic Computing Symposium, ACM, 2017, p. 4.

[26] N. TaheriNejad, P. S. Manoj, and A. Jantsch, “Memristors’ potential for multi-bit storageand pattern learning,” in Modelling Symposium (EMS), 2015 IEEE European, IEEE, 2015,pp. 450–455.

[27] P. Wijesinghe, A. Ankit, A. Sengupta, and K. Roy, “An all-memristor deep spiking neuralcomputing system: A step toward realizing the low-power stochastic brain,” IEEE Transac-tions on Emerging Topics in Computational Intelligence, vol. 2, no. 5, pp. 345–358, 2018.

[28] A simplified and integrated approach to model predictive control implementation. [Online].Available: https://www.plantautomation.com/doc/a-simplified-and-integrated-approach-to-model-0001.

[29] R. Zhang, A. Xue, and F. Gao, “Temperature control of industrial coke furnace using novelstate space model predictive control,” IEEE transactions on industrial informatics, vol. 10,no. 4, pp. 2084–2092, 2014.

32