ISSC 2010, UCC, Cork, June 23–24

Room Occupancy Measurement UsingLow-Resolution Infrared Cameras

Martin Berger†, Alistair Armitage∗

School of ComputingEdinburgh Napier University,

UK

E-mail: †martinberger85@gmx.de ∗a.armitage@napier.ac.uk

Abstract — Measuring room occupancy has always been a desirable endeavour. Thiscould be for security reasons, to save energy or simply for statistical reasons. Recentlythe use of thermal imaging systems in this area has increased as these systems becamecheaper and therefore more attractive. This paper investigates room occupancy mea-surement using a low resolution (16x16 pixel) infrared camera. After giving a shortoverview of application areas, the chosen people detection algorithm is outlined. Thisincludes the introduction of a novel method for blob classification, using morphologicalarea adjustment.

Keywords — People Detection, Occupancy Measurement, Thermal Imagery, Blob Detec-tion, Low-Resolution, Infrared

I Introduction

Knowing how many people occupy a certain roomor building is a key factor for its energy and se-curity management. The detection of people isincreasingly done by thermal imaging systems asthey are independent of light and therefore chang-ing illumination. Also, people are normally hot-ter than their background; this simplifies detectionmethods in thermal imagery.

A lot of research has been done for detection ofpeople in high-resolution thermal imagery [1, 2, 3].Most of this research aims towards pedestrian de-tection in larger, outdoor areas, e.g. streets orparks. Moreover, high resolution infrared camerasare still quite expensive. On the other hand, stud-ies have been done for simple motion detectionusing very-low-resolution infrared sensors [4, 5].These sensors are mainly passive infrared (PIR)sensors; they are not able to determine still ob-jects. Also, individual PIR detectors are not suffi-ciently effective: several have to be combined intoan array to cover an adequate area.

A small amount of work has been done usinglow-resolution (16x16 pixel) PIR sensors to detectpeople’s trajectories [6]. However, for this work

Fig. 1: Example image for low-resolution infrared

we used a 16x16 pixel infrared camera whichobtains absolute temperatures using a chopperwheel. This camera provides significantly moreinformation than individual PIR sensors, yet costsa fraction of high resolution infrared cameras.Figure 1 shows an example image of a group ofpeople in a hallway viewed from above.

This paper first presents an overview of the pro-posed application areas in section II. Section IIIdescribes the camera used. In section IV we in-troduce the proposed algorithm and give a short

overview of it. Section V describes the appliedbackground subtraction approach, whereas the ac-tual person detection is outlined in section VI. Fi-nally we present our results and a conclusion insections VII and VIII.

II Application areas

Most intelligent building systems use motion de-tectors to obtain occupancy information, generallyfor energy control. By using low-resolution cam-eras we are no longer dependent on the movementof a person as they measure the absolute temper-atures of the observed scene. Areas where move-ment is infrequent or little, e.g. libraries or com-puter centres, could be monitored more accurately.In addition, simple motion detectors, also knownas burglar detectors, can not count people as theydo not provide position information.

For security applications, visible CCTV cam-eras are still very common. However, by using lowresolution IR imagery, we do not have to worryabout privacy issues because people are not dis-tinguishable by their gender, age or other personalattributes. Also, people detection in low-resolutioninfrared imagery is less complex than in visible im-agery [7].

Another area of application is in the health caresector, where elderly people in assisted living ac-commodation can be monitored, searching for un-usual behaviour [5]. For example, an alarm mightbe given when the observed person is detected inthe middle of a room and does not move for a des-ignated time.

In our investigations, we focused on energy sav-ing applications for indoor areas. For example, anintelligent lighting system can use the occupancydata to automate lighting control. Heating con-trol is also conceivable as we can measure absolutetemperatures rather than temperature differences.The occupancy information can then contribute toa demand-oriented heating control. However, thesystem is just as suitable for security and surveil-lance applications as the sensitivity can be ad-justed accordingly.

III Camera properties and limitations

Our recordings have been done using an IRI 1011thermal imager from Irisys1. The key part of thisis the 16 by 16 focal plane array (FPA) detector,which delivers images with a frame rate of 8 fps.An accuracy was given with ±5◦C, and a spectralresponse range between 8 to 14 micrometers, witha temperature detection range of −10◦C to 300◦C.

The major limitation for our purpose was thenarrow field of view, with a lens of only 20◦. Forobserving broader scenes, which is necessary foroccupancy measurement, we had to do recordings

1Irisys: http://www.irisys.co.uk

from large distances. For example, a distance of10m observes a plane area of 3.5m by 3.5m.

For this prototype study, we decided to keepthe camera in a fixed position, which drasticallydecreases the background subtraction’s complex-ity. Moreover, the angle between scene and cam-era was kept near to a top view position, i.e. closeto 90◦.

IV People detection approach

For people detection in high resolution images,pattern recognition can be used to detect partsof the human body. However, in low-resolutionthermal imagery we can not see detailed charac-teristics of people. People appear to have a round“blob” shape. Thus, we decided to use blob detec-tion methods for people detection.

A lot of work has been done to detect blobsin thermal or visual imagery [8, 9]. Area, centrepoint or direction are only some attributes of blobswhich can help to classify them as persons. In ourwork we use the centre point to obtain peoples’location, whereas the area is used for classifica-tion. Figure 2 shows an overview of our approach.

Fig. 2: People detection algorithm

The low-resolution input image needs to be pre-processed before we actually apply blob detection.Thus, the first step in our algorithm is a rescal-ing operation which increases the size and quality.For rescaling we used bicubic interpolation, whichincreased the image quality significantly, by reduc-ing the effects of aliasing. Rescaling is followed bythe background subtraction explained in section

V. Afterwards, the Laplacian of Gaussian blob de-tection method is applied, which is then followedby our area adjustment algorithm (see section VIand VII). Finally we extract the detection resultsand superimpose them on the rescaled image; theblob attributes are also available. Moreover, wemeasure the occupancy in different image regionsby dividing the image into areas of interest andcounting people in each area. The algorithm andits testing environment has been implemented us-ing Matlab2 with its image processing toolbox.

V Background subtraction

To define people or objects in images one has todetermine whether a pixel belongs to the back-ground or to the foreground. In this algorithmwe can choose between two different methods toidentify the background. The first is simply bychoosing background frames manually. The sec-ond method is a background estimation using mor-phological operations. Having estimated the back-ground we then subtract it from the input image.The output is then given by two images, one con-taining objects warmer then the background andanother one consisting of objects colder than thebackground.

a) Method for choosing frames

Determining the background manually by choos-ing frames is a fairly easy and fast way to estimatethe background. Because of image noise and alsochanges of the background itself, an average overa range of frames Bk(x, y) should be used to esti-mate the background B(x, y). Equation 1 showsthe calculation of the estimated background usingN manually chosen background frames.

B(x, y) =

N∑k=1

Bk(x, y)

N(1)

However, this method requires a scene where ob-ject free frames are obtainable. This might causeproblems in constantly occupied scenes. Also,the background changes from time to time, e.g.when the room temperature drops or computersget switched on or off. Therefore, a more dynamicmethod is introduced in the next section.

b) Morphological background subtraction

This approach uses morphological opening to builda background model. Thus, it uses the geometricalshape informations of persons. Seen from an angleclose to top view, this shape appears to have around, blob shaped boundary, especially in low-resolution images.

In general, morphological opening is often usedto eliminate objects which are smaller than their

2Mathworks: http://www.mathworks.com

structuring element S. The same applies for itsuse for background estimation. By choosing thestructuring element to be a similarly shaped ob-ject, circular in this case, we can eliminate objectsfrom the image and obtain the background. There-fore, the size of the structuring element has to begreater or equal than the actual object. To deleteboth warm and cold objects we have to use open-ing twice. At first we apply it to the original imagefr which gives us the interim result Ba. As mor-phological operators we used ◦ for opening, forerosion, ⊕ for dilation and C for building the com-plement.

Ba = fr ◦ S = (fr S)⊕ S (2)

The interim result still contains the cold objectsas only positive blobs were cropped. To erase coldobjects as well, we need to build the complementof the interim result and apply opening again. Seeequations 3 and 4, with n denoting the image’s bitdepth.

BCa (x, y) = −Ba(x, y) + (2n − 1) (3)

BC = BCa ◦ S = (BC

a S)⊕ S (4)

As this background image is still in its comple-ment version we need to build the complementonce more to obtain the estimated background B.

B(x, y) = −BC(x, y) + (2n − 1) (5)

Because this method calculates the backgroundfor each frame, previously calculated backgroundswould be lost. Therefore, a running average,adopted from [10], was implemented; this buildsthe background estimation using a fraction of thenewly calculated background and the previouslyestimated background. The remembrance rateα defines to what extent the new or the previ-ous background is used for building the currentbackground frame. The range of α is defined as0 ≤ α ≤ 1. Thus, the background of frame i canbe determined as seen in equation 6.

B(i) =

{B̃(i) if i = 1

αB̃(i) + (1− α)B(i−1) otherwise(6)

Alternatively, we can express this in a mathemat-ical manner with all frames taken into account.

B(i) =

i−1∑k=0

α(1− α)k ∗ B̃(i−k) (7)

VI Blob enhancement and detection

a) Fixed-scale Laplacian of Gaussian (LoG)

People standing close together are hard to distin-guish in low-resolution imagery as they merge to-gether and build a new single blob that is obviously

bigger. By using the Laplacian of Gaussian (LoG)we can separate merged blobs to some extent.

For this purpose all blobs are assumed to havea circular shape. Thus, applying the LoG with theright scale creates a maximum in the centre of eachsmaller blob, which is part of the merged blob [9].

During our work various person blobs havebeen analysed with a range of scale factors to finda reasonable diameter value for an average person.However, the maximum response and therefore thescale factor have been determined experimentallyand have to be adjusted if the camera properties,especially the camera height, change. We foundout that it is more accurate to determine the scalefirst and to derive an estimated blob diameter fromthat scale, instead of measuring person diametersand calculating the scale factor from them. Theconnection between estimated blob diameter dband scale factor σ can be seen in equation 8.

db = 2rb = 2√

2σ (8)

The normalized LoG function, as seen in equation9, was chosen as convolution kernel.

hn(x, y, σ) =x2 + y2− 2σ2

σ2exp− x2+y2

2σ2 (9)

b) Morphological area adjustment

Whereas the Laplacian of Gaussian was used forinitial separation of merged blobs, we now actu-ally detect and measure blobs. Therefore, we areespecially interested in a blob’s centre point posi-tion but also in its area, which is normally used tovalidate blobs.

We can estimate a value for an average area ofa person Ab, by using the estimated blob diameterdb introduced earlier.

Ab =

(db2

)2

π (10)

The requirements for a blob evaluation are basedon how blobs can appear. In the following thethree major cases are summarized.

• The easiest case is described by a single blobbelonging to a single person with no otherblob in contact. We can immediately say thatthe blob belongs to a single person.

• The second case occurs if people are close to-gether: they can be merged into a single blob.We would have to attempt separating themas far as possible. The main characteristic ofthis case is that each of the blobs, contribut-ing to the merged blob, belongs to a personand should have the area of Ab.

• The third case describes the phenomenon thata single person results in multiple blobs. For

example, if we imagine a person wearing athick sweater with outstretched arms, thehands can be detected as extra blobs. Theseblobs are then separated from the body’s blob.In this case we need to try to merge them to-gether.

The main problem we have is that case two andcase three are contradictory. In case two we tryto separate blobs and in case three we aim tobring them together. If we try to consider bothcases we can not simply apply an algorithm to thewhole image. Therefore, we had to find a way toevaluate blobs individually.

The found solution is called “area adjustment”.The basic idea is, when we adjust the area of eachblob to the estimated blob area Ab of a person,some of the small blobs belonging to case three willbe enlarged to a typical person size and thereforemerge together. On the other hand blobs with agreater area will be narrowed, with the aim of sepa-rating merged blobs belonging to multiple persons.Due to the actual implementation of the algorithmit is advantageous to use a slightly higher areavalue than Ab, which we introduce with Amax.Alsoa minimum value Amin was determined to eraseunusual small blobs.

Fig. 3: Overview of the area adjustment algorithm

The algorithm contains three stages:

• In the first stage all blobs with an area A ofAmin < A < Amax are dilated, until their areais greater than Amax.

• In the second stage all blobs with an area Aof Amax < A are eroded, until their area isbelow Amax. This includes also the dilatedblobs from the first stage.

• The final erosion for a blob can cause disrup-tion of blobs, which can then result in very

small, invalid blobs. Therefore, all blobs withan area A of Amin > A are erased during thelast stage.

By applying this three-stage algorithm we obtainadjusted blobs with an area value which is defi-nitely within Amin and Amax. The majority havean area close to Amax.

For the morphological operations, we used acircular shaped structuring element. For everyiteration all blob properties are determined fromthe binary description of the input image. Theseproperties are then provided for the three differentstages. After the final deletion of small blobs, theoutput image is in its binary form. Also, the finalblob properties are given, i.e. blob area and centrepoint. Note that the algorithm has to be appliedfor the separated warm and cold objects. Figure 3shows an algorithm overview.

VII Results

We use this section to outline the results achievedin relation to people detection and occupancy mea-surement.

a) b)

c) d)

Fig. 4: a) Rescaled input image; b) Warm objects afterbackground subtraction and LoG; c) Binary image after

the area adjustment; d) Result image

In figure 4a) we can see the rescaled inputimage from figure 1 using bicubic interpolation.After subtracting the background using morpho-logical background estimation and applying theLaplacian of Gaussian, we can see a reasonableseparation of warm objects in figure 4b). Figure4c) shows the output of the area optimisationalgorithm. In this example all persons have beenseparated successfully, where the similar size ofeach blob highlights the principle of the areaoptimisation. After determining the blobs’ centrepoints, we can output the analysed image, figure4d), by superimposing crosses at each centre point

onto the input image.

We also extract the room occupancy by deter-mining people counts for the observed area. Fig-ure 5a) shows an occupancy measurement of acrowded hallway, whereas figure 5b) shows the oc-cupancy results obtained after the main Univer-sity entrance. Smoothing the people count can de-crease the failure rate, but also decreases detectionrate for fast moving people. In this examples wesmoothed the results using a moving average over8 elements, which equals one second.

0 2 4 6 8 10 128

9

10

11

12

13

14

15

16

17

18

Time [s]

Per

son

s co

un

t

Detected people countActual number of people

a)

60 65 70 75 800

1

2

3

4

5

6

Time [s]

Per

son

s co

un

t

Detected people countActual number of people

b)

Fig. 5: a) Occupancy measurement of a crowded hallway;b) Occupancy measurement after the University’s main

entrance

Although the work was focused on post exami-nation of the recordings, we achieved a simulationtime that was near to real time for images rescaledwith the factor 4 (64x64 pixels). Even thoughthe scene was crowded, and therefore the iterativemorphological operations took up significantlymore time.

Due to noise and background subtraction de-viations, thresholding is applied for warm andcold objects after subtracting them from the back-ground. The threshold values describe the omit-ted temperature range around the estimated back-ground. Accurate results have been obtained usinga threshold of ±2◦C, which means that personsneed to appear at least 2◦C warmer or colder thanthe background for a successful detection. Lower-ing this threshold would increase the detection andfailure rate. Alternatively, increasing the thresh-old would decrease detection and failure rate.

In figure 6, the two circles highlight undetectedpersons. We can see they are hard to recognisefor a human observer as their temperature is closeto the background temperature. Thus, they are

Fig. 6: Not detectedand merged persons

Fig. 7: Backgroundestimation fault

neglected due to the threshold explained previ-ously. The marked area in the left of the imageshows two people being detected as one, becausethey are close together and neither the Laplacian ofGaussian nor the area adjustment separated them.If we lower the estimated area for the area ad-justment algorithm we can enhance separation ofblobs. However, we would also increase the era-sure rate and therefore increase the number of un-detected persons.

Figure 7 shows detections, based on image noiseand inaccuracies of the background subtraction. Inthis case the threshold value was too low to elim-inate the influence of noise and the gap betweenestimated and real background.

VIII Conclusion and future work

In this paper we present a state of the artsolution for room occupancy measurement inlow-resolution thermal imagery. We outlinepossible application areas; we focus on intelligentbuilding systems. A people detection algorithm ispresented in detail, where we discuss the imple-mented background subtraction methods as wellas blob detection and evaluation. We introducea novel approach for blob classification, usingiterative morphology to adjust area properties.An extensive set of Matlab code was programmedto implement and evaluate the algorithm. Wekept the algorithm’s sensitivity adjustable toallow adaptation, based on the requirements ofthe desired application area. Even though wewere using calculation intensive methods, thesimulation time could be held close to real timeas a result of the limited resolution. We have alsooutlined the sources and effects of failed detections.

Experiments with various recorded scenesshowed promising results, including the separa-tion of close persons. For further improvementsof the results, we will enhance the backgroundsubtraction methods, to reduce the necessarythresholds and therefore increase detection rate.Furthermore, we will investigate to what extentpeople tracking can be realised. Also, we planto evaluate results of recordings, taken with areasonable lens angle to widen the observed scene.

References

[1] James W. Davis and Vinay Sharma. Ro-bust background-subtraction for person de-tection in thermal imagery. Computer Visionand Pattern Recognition Workshop, 2004.CVPRW ’04., page 128, July 2004.

[2] James W. Davis and Mark A. Keck. A two-stage template approach to person detectionin thermal imagery. Seventh IEEE Work-shops on Application of Computer Vision(WACV/MOTION’05), 1(7):364–369, Jan-uary 2005.

[3] Paul Viola, Michael J. Jones, and DanielSnow. Detecting pedestrians using patterns ofmotion and appearance. International Jour-nal of Computer Vision, 63(2):153–161, July2005.

[4] Robert H. Dodier, Gregor P. Henze, Dale K.Tiller, and Xin Guo. Building occupancy de-tection through sensor belief networks. En-ergy and buildings, 38(6):1033–1043, 2006.

[5] A. Kaushik, N. Lovell, and B. Celler. Evalua-tion of pir detector characteristics for mon-itoring occupancy patterns of elderly peo-ple living alone at home. Engineering inMedicine and Biology Society, 2007. EMBS2007. 29th Annual International Conferenceof the IEEE, pages 3802–3805, August 2007.

[6] Alistair Armitage, David Binnie, Jon Ker-ridge, and L Lei. Measuring pedestrian tra-jectories with low cost infrared detectors: Pre-liminary results. Pedestrian and EvacuationDynamics: 2nd International Conference heldat the Old Naval College, University of Green-wich, London, 20-22, pages 101–110, August2003.

[7] J. Kerridge, R. Kukla, A. Willis, A. Armitage,D. Binnie, and L. Lei. In Traffic and Granu-lar Flow 03, pages 383–391. Springer, BerlinHeidelberg, 2005.

[8] Stefan Hinz. Fast and subpixel precise blobdetection and attribution. IEEE Interna-tional Conference on Image Processing, 2005.ICIP 2005., pages 11–14, 2005.

[9] Tony Lindeberg. Feature detection with au-tomatic scale selection. International Journalof Computer Vision, 30(2):79–116, 1998.

[10] Massimo Piccardi. Background subtractiontechniques: a review. In SMC (4), pages3099–3104, 2004.