Dynamic Partial Reconfiguration in Embedded Systems for Intelligent Environments

Javier Echanobe, Inés del Campo, Raul Finker Department of Electricity and Electronics

University of the Basque Country Leioa, Spain

e-mail: javi@we.lc.ehu.es

Koldo Basterretxea Department of Electronics and Telecomunications

University of the Basque Country Leioa, Spain

e-mail: koldo.basterretxea@ehu.es

Abstract—In this paper we propose to apply the Dynamic Partial Reconfiguration (DPR) technology to embedded systems intended for Intelligent Environments. To reach this goal, we have developed a system based on a Field Programmable Gate Array (FPGA) in which high performance hardware modules can be reconfigured on-line according to the necessities of the system at each moment. Two different implementations have been carried out to measure the time required to reconfigure each module and also to measure the FPGA resources that can be saved if we keep configured only the modules that are required at each time. The Obtained results show how this technique offers advantages in cost, size and power when applied to embedded systems for intelligent environments.

Keywords-component; Embedded Systems, Dynamical Partial Reconfiguration, Intelligent Environments, FPGA, NeuroFuzzy Systems

I. INTRODUCTION In recent years much effort has been devoted to research in

the field of Intelligent Environments. This recent paradigm which is also known as "Ambient Intelligence (AmI)", proposes environments (e.g., public or private halls, rooms or spaces) endowed with a set of electronic systems, that are able to adapt to the preferences and necessities of the people existing in them in order to make their daily activities more easy and comfortable [1][2][3][4]. To cope with these special skills, these systems usually have used computational intelligence paradigms such as fuzzy systems, neural networks or genetic algorithms [5].

One of the requirements for achieving this goal in its broadest sense, is certainly, the availability of small-size, low-cost and low-power electronic devices with also high processing speed as they act in a scenario that requires real-time response. However, such features are quite compatible with the high computational requirements of the above-mentioned intelligent paradigms. As a result, most existing solutions are basically PC-based [6][7][8], because they focused more on the feasibility of the models, rather than on their physical implementations. An embedded system for Intelligent Environments has been proposed by our group in [9] as a solution for achieving efficient implementations of these systems. The system was implemented in a Field Programable Gate Array (FPGA) which is an integrated circuit designed to be configured by the customer or designer after manufacturing.

In order to obtain even smaller, cheaper and lower power systems, we propose here to provide these systems with the technology of Dynamic Partial Reconfiguration [10].

Dynamical Partial Reconfiguration (DPR) is a feature that allows to modify dynamically a part of the logic of an FPGA while the rest continues operating without interruption. This ability makes it possible to mutiplex many different functions in time, similar to the feature of a microprocessor to switch tasks in a multi-task single-processor system. Thus, a functional block (block of logic) is implemented or deleted in the FPGA according to the needs of the system. The benefits of this recent technology are very noticeable [10]: 1) size reduction of the FPGA device required to implement a given function and therefore reductions also in cost and power consumption. 2) More flexibility to select among different algorithms for a given application. 3) Improving FPGA fault tolerance. 4) Accelerating configurable computing.

In this paper, we propose an embedded system intended for Intelligence Environments, which incorporates DPR technology for the purpose of reducing size, cost and power. The intelligent capabilities of the system are addressed by means of a number of ANFIS-like Neurofuzzy systems [11] which are previously trained with input-output data collections. In particular, an FPGA-based SoPC (System on Programmable Chip) has developed out, in which the ANFIS systems are implemented as reconfigurable hardware modules. Thus, the system contains only the modules required in each moment with the consequent saving in power and size of the FPGA. The whole system has been implemented in 2 different Xilinx FPGAs (Virtex-5 and Virtex-6) so that a comparative between devices can be established.

The rest of the paper is organised as follows. Section 2 presents the Neuro-Fuzzy system used here which in fact is a modified version of the well known ANFIS model that has been called PWM-ANFIS by the authors because of its Piece-Wise Multilinear (PWM) behaviour. In Section 3 the proposed FPGA-based architecture with reconfigurable ANFIS modules is described in detail. Section 4 presents the particular implementation details: that is, the FPGAs used, the required times for reconfiguring each module, the size of those modules and the device resources utilization. Also we discuss the benefits of using this technology. Finally the main conclusions of the work are explained in Section 5.

2012 Eighth International Conference on Intelligent Environments

978-0-7695-4741-1/12 $26.00 © 2012 IEEE

DOI 10.1109/IE.2012.35

109

2012 Eighth International Conference on Intelligent Environments

978-0-7695-4741-1/12 $26.00 © 2012 IEEE

DOI 10.1109/IE.2012.35

109

2012 Eighth International Conference on Intelligent Environments

978-0-7695-4741-1/12 $26.00 © 2012 IEEE

DOI 10.1109/IE.2012.35

109

II. ANFIS MODEL FOR INTELLIGENT ENVIRONMENTS As we have stated in the Introduction, the intelligent

behavior of the proposed AmI system is addressed by means of an ANFIS-like System. An ANFIS system [11] is a Fuzzy Inference System whose parameters -membership functions and consequents- are trained by means of Neural Network algorithms. In fact, such a system can be viewed as a particular Neural Network that is functionally equivalent to a Fuzzy Inference System. Hence, it exhibits both the linguistic knowledge representation of fuzzy systems and the learning abilities of Neural Networks. To understand the structure of an ANFIS system we must pay attention first to the fuzzy system it represents:

Consider a rule based n-input fuzzy system with m antecedent functions per dimension. In general, we have a total of mn rules where the j-th rule can be expressed as:

R j : IF x1 is M j1

1 and ... and xn is M jn

n THEN y is c j1 j2 .. jn,

where: x=(x1, x2, ..., xn), x∈ Rn is the input vector, M j1

1 ...M jn

n

are linguistic labels associated with the membership functions μiji (xi) (i.e., antecedents of the rules; normally gaussian functions), y is the output variable and c j1 j2 .. jn

∈ R is a crisp consequent (i.e. a singleton).

If the center of gravity defuzzification method is adopted, the output of the system is given by

y =

w jj=1

mn

∑ c j

w jj=1

mn

∑

, (1)

where

w j = M j1

1 (x1)⋅ M j1

1 (x1)⋅ ... ⋅ M jn

n (xn ) . (2)

The computation of the output of this system -(1)- for a given input (i.e., the inference mechanism) can also be performed by a Neural Network like the one shown in Fig. (1). - The first layer computes the membership functions for all the antecedents: M ji

i (xi) = μiji (xi). - The second layer computes the values w j (1≤ j ≤ mn ) as the product w j = M j1

1 (x1)⋅ M j1

1 (x1)⋅ ... ⋅ M jn

n (xn )These values are also called the activation of the rules.

- The third layer normalizes these values by dividing each

one by the sum of all of them: w j = w j / wii=1

mn

∑ .

- The fourth layer multiplies each term by the consequent: w jc j

.

Figure 1. ANFIS Network Structure.

- Finally the output of the network is provided by the fifth layer which aggregates the overall output as the summation

y = w jc jj=1

mn

∑ . (3)

As can be shown, this network performs the same function as the previous fuzzy system.

To train the above network, a hybrid algorithm has been proposed [11]. The algorithm is composed of a Least Squares Estimator (LSE) process, followed by a back propagation (BP) algorithm. LSE computes the consequents and BP adjusts the parameters of the antecedents. This is possible due to the fact that the output of the network is linear in the consequent parameters. This hybrid algorithm is executed iteratively from a collection of training data (i.e., input/output pairs) until all the parameters are adjusted.

Due to the huge amount of information and the large number of parameters that are involved in an Intelligent Environment, we have used a modified model called PWM-ANFIS (PieceWise Multilinear ANFIS) which has been proposed by the authors [9][12] to reduce the computational requirements in high dimensionality systems. In particular, we have shown how, by introducing some restrictions on the membership functions (i.e., antecedents), a much more simplified system can be obtained with hardly any loss of the learning and approximation capabilities. These restrictions basically involve the use of normalized triangular membership functions overlapped by pairs. As a result, only two antecedents per input have non zero values for every new incoming vector and hence, only 2n rules become active each time (instead of mn in the unrestricted ANFIS). Also, the divisor in (1) (or in the 3th layer computation) becomes now 1 and therefore the division is no longer necessary. The computation of the output is now given by

y = w jc jj=1

2n

∑ . (4)

M11

M1m

Mn1

Mnm

x1

xn

Π

Π

Π

Nw1

w2

wmn

w1

w2

wmn

N

N

c1

c2

mnc

Σ

w1 c1

w2 c2

wmn mnc

y

110110110

To perform an inference we have now to sum 2n terms wjcj where each term wj is computed by multiplying n membership functions (2). As the antecedents are now triangles, every membership function evaluation is also simplified because it is obtained by solely multiplying the slope of a triangle by the input (to be exact, the offset of the input with respect to the triangle's corner). In addition, because the membership functions are normalized, the two active antecedents have complementary values and therefore the computation of one of them gives directly the other one only by performing a logical complement. As we can see, the inference process is computed by means of only a few sums and products so it is greatly simplified.

In [9][12] we showed the satisfactory modelling and learning abilities of the proposed PWM-ANFIS system in dealing with AmI environments. The systems were trained with a data subset from a real experiment at the intelligent dormitory (iDorm), which is a real ubiquitous computing test bed from Essex University [6]. The data related to seven input sensors and to four controlled actuators. The input sensors were: internal light level, external light level, internal temperature, external temperature, chair pressure, bed pressure and time. The actuators were four variable-intensity spot lights. After exhaustive experimentation, we could conclude that the system shows quite good learning and approximation capabilities. In addition, the results are not far from those obtained with the ANFIS model without restrictions.

III. SYSTEM ARCHITECTURE Taking into account the above issues, we have developed a

SoPC-based embedded system in which the learning/adapting and reasoning features are achieved by means of a collection of PWM-ANFIS systems. In particular, each PWM-ANFIS system is intended for a particular output (i.e., action) like those showed in the previous Section (actuators for variable-intensity spot lights). The SoPC architecture proposed here is based on a Microblaze Softcore together with ad-hoc reconfigurable hardware modules to compute each PWM-ANFIS output, which is given by (3). These PWM-ANFIS modules are reconfigured (added, deleted or changed) according the needs of every moment. That is, since each output (action) of the system has associated an PWM-ANFIS network, we keep implemented only those PWM-ANFIS modules whose outputs are required in each moment. This strategy is quite similar to that used in the management of cache memory on a computer, (i.e., keeping in fast memory - onchip memory - that part of the program/data memory that is needed in every moment). Similarly, the reconfiguration of one or more modules is performed whenever the context in the environment is modified to some extent: the user moves to another area and activates the sensors located there; the ambient parameters change significantly; the user changes of activity; a different user comes into the environment, etc.

The proposed architecture is depicted in Fig. 2 and has the following components:

- Microblaze. This Xilinx "Softcore" is the main component of the system. It deals with the global operation and also with the input-output handling. It is also the master of three buses to which the other modules are connected: 1) Local Memory Bus

(LMB) which is a Xilinx fast, local bus for connecting the MicroBlaze processor instruction and data ports to high-speed peripherals, mainly on-chip block RAM (BRAM). 2) Peripheral Local Bus (PLB) which is a Xilinx 128-bit Processor Local Bus for connecting an optional number of PLB masters and slaves. 3) Fast Simple Link (FSL) which are uni-directional point-to-point communication channel buses used to perform fast communication. The μBlaze executes a program stored in an internal RAM block (BRAM) that is accessed through the bus LMB.

- External Flash Memory. This memory module is external to the FPGA and contains the bitstreams of the ANFIS modules to be reconfigured. It is accessed by the Microblaze via the Xilinx's External Memory Controller (EMC). This controller is connected to the PLB bus.

- General Purpose Input-Output (GPIO). This module receives the system inputs coming from the environment sensors and sends the outputs out to the actuators.

- HW Internal Configuration Access Port (HWICAP). This hardware block -attached to the PLB bus - is the functional module that enables the FPGA to be reconfigured from itself. It receives from the MicroBlaze a bitstream - loaded from Flash Memory - and reconfigures the related ANFIS module.

- Reconfigurable ANFIS Modules. These ad-hoc reconfigurable hardware blocks represent one of the main parts of our work. Its design has been carried out by exploiting the parallel nature of the FPGA resources. In this way, an efficient and high performance architecture has been achieved. Each module or block performs the calculations of one PWM-ANFIS network. They act as co-processors of the Microblaze in the sense that each block calculates its output whenever the Microblaze sends their corresponding inputs. We have developed three different types of blocks: 2, 3 and 4 inputs blocks. In addition, each block can be dynamically reconfigured; that is, it can be deleted, implemented or replaced with a new ANFIS core according to the needs of the system as explained above.

Figure 2. Architecture of the system.

uBlaze

FPGA FLASHMEMORY

LMB

PLB

BRAMEMC

ICAP

Reconf.Anfis

Block 1

anfisblocks

bitstreams

main program

externalmodule

controller

Reconf.Anfis

Block n

GPIO I/O

FSL1 FSLi FSLn

111111111

Each one of these cores is connected to the MicroBlaze by means of an FSL bus and is composed of the following three main modules or units (Fig. 3):

First, we have a local ROM block to store the parameters needed for evaluating the output (3): the slopes and corners of the triangular antecedents and the consequents of the rules.

Second, the module called "Preprocessing unit" receives the input data from the Microblaze and determines which are the respective active rules by comparing the inputs with the corner of the triangular antecedents. As a result, it loads the involved parameters from the ROM (slopes and consequents) and also computes the input offsets.

Finally, the third module is called the "Inference Processing Unit". It takes the offsets together with the loaded parameters and computes (3).

IV. GLOBAL FUNCTIONALITY The operation of the entire system is governed by the

Microblaze: Initially the FPGA is configured with various ANFIS cores depending on each specific application. The Microblaze starts reading input values provided by one or more input peripherals, which are connected to different sensors. Then, it makes a data conditioning so they can be better processed; e.g., scaling, normalization and truncating to a fixed length. Next, the data are sent to the PWM-ANFIS cores to perform the respective neural computation and the results are returned to the Microblaze, which sends them out through the I/O peripherals. The system remains in this mode until one or more ANFIS cores need to be reconfigured. This may be due to any of the reasons mentioned above (i.e., an event occurrence) and has been implemented as a finite state automata where the transistions from one state to another takes place with one of these events. The microblaze reconfigures the required ANFIS cores by taking the related bitstreams from the flash memory and sending them to the ICAP core. Then, it returns to the previous mode and continues its operation normally.

V. RESULTS AND DISCUSSION In this section we analyze the results obtained in two different implementations of the proposed architecture. The aim is to show by concrete implementations, the viability of the solution, based on the use of DPR. In particular, the architecture has been implemented in the Xilinx's Virtex5 and Virtex6 FPGAs [13]. These FPGAs together with Virtex4, are those in which DPR is currently supported by the design tool used: "ISE Design Suite 12.4" [14]. More precisely, we have used the devices XC6VLX240T and XC5VSX50T which are medium-sized devices within their respective families. The parameters that are to be analyzed are the time required to reconfigure the ANFIS modules and the percentage of resources of the FPGA that these modules require.

The reconfiguration time is a crucial parameter because the system operates virtually in real time and therefore cannot be halted beyond a very short time. This time value directly depends on the size of the bitstream as well as the operating frequency of the ICAP module. Therefore, for a given implementation -with a fixed clock frequency- the reconfiguration time is higher for larger size bitstreams.

Figure 3. Internal Architecture of the PWM-ANFIS Cores.

The amount of resources used by each module is also important information because we can estimate the power that can be saved every time a module is not implemented (even if the module is standby, there is a static power consumption due to the leakage currents). In addition, this information is helpful to analyze the possibility of using smaller devices with reduction in cost and consumption.

Tables I and II show the main results of the implementations. As we can see, the time to reconfigure the modules is of miliseconds: It ranges from 16 miliseconds for a 2-input module to 126 miliseconds for a 4-input module in the Virtex-6 device and from 16 miliseconds to 234 miliseconds in the Virtex-5 device. Although we deal with a real-time system, the operating speed of an intelligent environment need not be especially high, since the interaction between the user and the system is about seconds. Hence, the times required to reconfigure our modules - at most 234ms. - is quite appropriate. In addition, we can verify how the reconfiguration time is directly proportional to the size of the bitstream as we have already explained.

Related to the required FPGA resources of the modules, we can see that 2-input modules require less than 1% of the available resources. Therefore the saving in size and power by using DPR, would have sense only if the system had a lot of these modules. Instead, 3-input and 4-input modules are quite larger, especially the latter. In these cases the potential savings can be significant. For example, in the Virtex-5 we can save more than 15% of the FPGA slices whenever a 4-input module is not being implemented.

VI. CONCLUSION In this work we present some results of applying the

technology of Dynamical Partial Reconfiguration (DPR) in embedded systems intended for Intelligent Environments. The aim has been to analyze the benefits that can be obtained by using this technology in these systems. To reach this goal, we have designed an FPGA-based embedded system that contains, among others, a collection of ad-hoc hardware modules, which can be dynamically reconfigured according to the system

PRE-PROCESSING

ROM

Antecedentsand

Consequents

INFERENCE PROCESSING UNIT

PWM-ANFIS MODULE

FSL

FSL

112112112

requirements. Each one of these hardware modules - called PWM-ANFIS modules - computes the response of an ANFIS-like neurofuzzy system.

The system has been implemented in two different Xilinx FPGAs in order to compare performances. From the results obtained, we draw the following conclusions:

On the one hand, the reconfiguration time is about some ten of miliseconds which are sufficient for Intelligent Environments where the operation time - defined by the user-machine interaction - is of the order of seconds. This allows to remove/add dynamically the modules from/to the FPGA, based on the system requirements, without affecting the normal operation. On the other hand, we have shown that in many cases, the savings in FPGA resources by applying DPR can lead to a significant power and size reduction. The reduction in size means the possibility of using smaller devices and thus lower cost.

In view of these results, we conclude that DPR can bring benefits into the embedded systems applied to the field of Intelligent Environments.

TABLE I. VIRTEX-6 (XC6VLX240T) IMPLEMENTATION

ANFIS 2v ANFIS 3v ANFIS 4v

Reconf. time 16 ms 63.8 ms 126 ms

Bitstream 22K 88K 174K

% FPGA Slices 0.2% 0.9% 2.4%

TABLE II. VIRTEX-5 (XC5VSX50T) IMPLEMENTATION

ANFIS 2v ANFIS 3v ANFIS 4v

Reconf. time 16 ms 66.8 ms 233.7 ms

Bitstream 22K 92K 322K

% FPGA Slices 0.76% 2% 15.4%

ACKNOWLEDGMENT This work was supported in part by the Spanish Ministry of

Science and Innovation and European FEDER funds under Grant TEC2010-15388, and by the Basque Country Government under Grants IT419-10, S-PC10UN09 and S-PC11UN012.

REFERENCES

[1] European Commission IST Advisory Group, Scenarios for

Ambient Intelligence in 2010, Final Report, Feb. 2001. [2] European Commission IST Advisory Group, Ambient

Intelligence: From Vision to Reality, 2003. [3] T. Basten, M. Geilen, and H. de Groot, Eds., Ambient

Intelligence: Impact on Embedded System Design. Boston: Kluwer Academic Publishers, 2003, Part I.

[4] F. Sadri. Ambient Intelligence: A Survey. ACM Computing Surveys, Vol.43, No. 4, October 2011.

[5] H. Hagras, I. Packham, Y. Vanderstockt, N. McNulty, A. Vadher, and F. Doctor, An Intelligent Agent Based Approach for Energy Management in Comercial Buildings, Proc. IEEE International Conference on Fuzzy Systems, pp. 156-162, Hong Kong, 2008.

[6] F. Doctor, H. Hagras, and V. Callahan, A Fuzzy Embedded Agent-Based Approach for Realizing Ambient Intelligence in Intelligent Inhabited Environments, IEEE Transactions on System, Man, and Cybernetics-Part A, vol. 35, no. 1, pp. 55-65, Jan. 2005.

[7] A. Vainio, M. Valtonen, and J. Vanhala, Proactive Fuzzy Control and Adaptation Methods for Smart Homes, IEEE Intelligent Systems, vol. 23, no. 2, pp. 42-49, Mar./Apr. 2008.

[8] P. Rashidi, and D.J. Cook, Keeping the Resident in the Loop: Adapting the Smart Home to the User, IEEE Transactions on System, Man, and Cybernetics-Part A, vol. 39, no. 5, pp. 949-959, Sep. 2009.

[9] I. del Campo, I., J. Echanobe, G. Bosque, and J.M.Tarela, “Efficient Hardware/Software Implementation of an Adaptive Neuro-Fuzzy System,” IEEE Transactions on Fuzzy Systems, vol. 16, no. 3, pp. 761-778, Jun. 2008.

[10] Partial Reconfiguration User Guide, www.xilinx.com. UG702 (v12.3) October 5, 2010 . Xilinx.

[11] J.-S. Jang, ANFIS: Adaptive-network-based fuzzy inference system, IEEE Trans. System, Man and Cybernetics, vol. 23, pp. 665-685, june 1993.

[12] J. Echanobe, I. del Campo, G. Bosque, and J.M. Tarela, An Adaptive Neuro-Fuzzy System for efficient implementations, Information Sciences, vol. 178, pp. 2150-2162, 2008.

[13] FPGA Family Overview, Xilinx Inc., San Jose, CA. www.xilinx.com/products/silicon-devices/fpga/index.htm.

[14] ISE Design Suite 12.4, Xilinx Inc., San Jose, CA, www.xilinx.com/support/documentation/dt\_ise12-4.htm

113113113