socially aware interactive playgrounds

40 PERVASIVE computing Published by the IEEE CS n 1536-1268/13/$31.00 © 2013 IEEE

U n d e r s ta n d i n g a n d C h a n g i n g B e h av i o r

socially aware interactive Playgrounds

P lay has been widely studied in the human sciences because of its importance in children’s physi-cal and cognitive development.1 Recently, computer scientists have

also begun to address play by building interac-tive playgrounds—technology-enhanced instal-lations that combine the fun and immersion of digital games with the benefits of traditional free play. The size of these colocated spaces can vary

from a few square meters to the size of public squares, and they can be equipped with a wide range of interactive elements, such as slides,2 toys,3or cam-era-projector combinations.4 Interactive playgrounds aim to provide engaging, entertaining,

and immersive game experiences, while promot-ing physical activity, social interactions, or cog-nitive development.3

Designing such interactive playgrounds is a challenging task. Ben Schouten and his col-leagues argue that three criteria must be ob-served when designing the game experience: context-awareness, adaptation, and personal-ization.5 However, although many interactive playgrounds consider such criteria when design-ing interactive experiences for children, few of them address all three criteria simultaneously. We envision playgrounds that automatically sense and interpret social interactions during play (context-aware), adapt the game mechanics

(adaptation), and learn from the sensed behav-ior to create engaging and fun experiences for all players (personalization). Moreover, such playgrounds could promote or discourage certain types of behavior by adapting the game mechanics while the children play.

To this end, we propose using social signal processing (SSP),6 an emerging field of research that addresses the automatic analysis of non-verbal social behavior. Applying SSP to inter-active playgrounds could lead to a new genera-tion of playgrounds, where each game unfolds differently depending on the children playing, how they play, and the type of behavior being promoted or discouraged.

Play and interactive PlaygroundsPlay helps children explore their bodies’ capa-bilities and develop their fine and gross motor skills and hand-eye coordination.7 Activities such as jumping, running, and tumbling can help develop and maintain muscular fitness and flexibility while creating positive social relation-ships between the players.1 Play also presents children with opportunities for cognitive de-velopment.7 Children practice their planning skills when determining what to do, improve their decision-making skills when several op-tions are presented, and foster their creativity by inventing games and rules. Lastly, play allows children to feel part of a group, satisfying the inherent need of humans to belong to something and interact with others.

Social signal processing could enhance interactive playgrounds, letting them automatically sense and interpret children’s social interactions during play, adapt game mechanics to induce targeted social behavior, and learn from the sensed behavior to meet players’ expectations and desires.

Alejandro Moreno, Robby van Delden, Ronald Poppe, and Dennis ReidsmaHuman Media Interaction, University of Twente

july–sEptEmbEr 2013 PERVASIVE computing 41

elements of interactive PlaygroundsInteractive playgrounds are composed of three main elements: sensors, ac-tuators, and gameplay (See Figure 1). Sensors are used to obtain information from the environment and the players therein and can range from cameras to touch-sensitive surfaces. Actua-tors are elements—such as projectors, speakers, or lights—through which the playground provides feedback to the children. Finally, gameplay refers to how the children and playground interact. Interactive playgrounds can be placed at various locations, such as schools, streets, or gyms.

Current game experiencesThe type of experience that play-grounds are designed to elicit can dif-fer significantly between playgrounds. Some interactive playgrounds focus on encouraging social competency, while others focus on promoting physi-cal fitness. At the same time, these playgrounds might use cameras and pressure sensors as their main input modalities, while other could use mi-crophones or accelerometers. For exam-ple, Daniel Tetteroo and his colleagues designed an interactive playground to stimulate not only social interaction but also physical activity.4 The playground contains a projector and an infrared camera, and children wear hats with in-frared reflectors. Additionally, they are given wristbands and balls with wire-less motion sensors. The playground measures the children’s locations and motions during the game using camera input. Colored shapes are projected onto the floor with which the children can in-teract. Additional kinds of interactions are made available when using the balls.

The Stomp platform, by Peta Wyeth and her colleagues, was also designed to physically and cognitively engage players—specifically, those with intel-lectual disabilities.8 However, instead of camera-based tracking, their instal-lation requires children to stomp, slide, or employ other body movements to

activate pressure sensors located on the floor.

Joan Soler-Adillon and Narcís Parés focused on context-awareness and de-veloped an interactive slide to encour-age physical activity and socialization in children.2 Their work projects a game onto a physical slide. A camera senses the movements of children slid-ing down and uses this information to control the game mechanics.

Hisakazu Ouchi and his colleagues additionally focused on personaliza-tion, using pressure sensors to monitor children’s play behavior on an interac-tive climbing wall.9 Afterward, they created a model of how a child of a cer-tain height would climb the wall, using the collected data. Alireza Derakhshan and his colleagues also implemented personalization by observing and cre-ating behavioral models for different types of players.10 In addition, they implemented adaptation by using ma-chine learning techniques to change game mechanics to match different types of players.

sensing and inducing social BehaviorChildren’s experiences in interactive playgrounds could be improved by

considering the social component of play to automatically adapt and per-sonalize the game mechanics. The au-tomatic sensing and interpretation of social behavior would help us to

• interpret social interactions (context- awareness),

• change the interactions with the play-ground (adaptation), and

• analyze the effects of these interactions on a personal basis (personalization).

We’re particularly interested in adapt-ing the game mechanics to encourage positive social interactions and to dis-courage negative ones. Additionally, children could be persuaded to assume different roles to explore and learn dif-ferent behavioral patterns.11 We discuss scenarios and supporting research to convey how the inclusion of SSP could improve interactive playgrounds.

Child exclusionConsider the following scenario. Five children are playing in the playground, competing to gather balls that can glow in different colors and produce sound. One child is shy and doesn’t grab any balls so as not to affront the other children. The playground recognizes

Figure 1. Impression of an interactive playground.

42 PERVASIVE computing www.computer.org/pervasive

Understanding and Changing Behavior

that four children are competitively playing while one has been separated from the group.

To prevent further exclusion, the playground starts rewarding play-ers for social interaction. The balls only start to glow and play a sound if they’re exchanged. Because the ex-change between each pair of children results in a different glow and sound pattern, the four children start includ-ing the shy child to explore the patterns presented. Eventually, the playground starts to reward quick exchanges to different players by providing increas-ingly interesting visuals and playing cheering sounds.

Various studies support such a sce-nario. For example, Tilde Bekker and her colleagues have evaluated how children use “mixers”—that is, objects with similar affordances that change appearance based on the amount of movement or shaking or the presence of other mixers.3 Without any instruction regarding the types of interactions or games they could play, children ended up playing as a group.

Moreover, Alireza Derakhshan and his colleagues successfully implemented a “luring” strategy in their adaptive in-teractive playground.10 They achieved an increase in physical activity with several strategies—for example, luring combined with changing the speed and distance of the targets.

Also, previous research has indicated that human observers can distinguish between competitive and collaborative endeavors, which hints that automatic recognition might also be possible.12 Furthermore, there are good indica-tions that deliberately assigning a more dominant or central role in an interac-tive game to a shy child can positively influence the child’s behavior.11

restoring engagementIn this scenario, six children are play-ing cooperatively, attempting to stomp colored circles projected onto the floor. The children are talking to each other, assigning different areas for each child

to cover, pointing and encouraging excitedly. As the game develops, the level of noise diminishes as does the overall movement of the children. The play-ground infers that this is caused by a loss of interest and diminished engagement.

The playground then creates new goals by modifying the feedback. The colored circles suddenly start glowing, and when children stomp on them, they split into two. Additionally, the rate at which they appear increases. The chil-dren discuss new strategies and are again engaged in their play. The play-ground senses that the noise goes up and the children are moving actively again, which it interprets as a regained interest in the game.

For a similar playground setup, Daniel Tetteroo and his colleagues found that catch-the-shapes is a natu-rally occurring made-up game.4 We’ve also witnessed that players frequently attempt stomping behavior in the in-teractive playground.12 Furthermore, informal observations of traditional playgrounds during primary school breaks show that stomping behav-ior occurs even without interactive technology.

Georgios Yannakakis and his col-leagues used the absence of feedback, goals, and adaptive behavior to ex-plicitly create a nonentertaining play-ground game.13 They noticed that these games were neither desirable nor engaging.

dominance and Conflict resolutionImagine that five children enter a play-ground and start exploring how the environment responds to the different sounds they make. After some time, the playground senses that one child is dominant, continuously yelling to evoke a response from the playground. The other children become fed up and lose interest, because they don’t get the chance to explore the interactions.

The playground starts to ignore the dominant child and only responds to

the other children. The dominant child recognizes this, stops yelling, and ob-serves how the other children interact. After a while, the playground again considers the dominant child but starts to respond with more elaborate audio and visuals if the children interact si-multaneously—singing together, for example.

Yun-Gyung Cheong and her col-leagues proposed a set of five underly-ing phases in conflict resolution when researching a serious game on con-flict resolution.14 Their first phase is “conflict generation,” which typically occurs when the goal of one player is blocked by another. The second and third phases are “conflict detection” and “player modeling and conflict strategy prediction.” The next phase is “conflict management,” which leads to the final phase of “conflict resolution.” Jesse Schell stated that “players need to think that they have a chance of achiev-ing the goal. If it seems impossible to them, they will quickly give up.”15 Therefore, by preventing the dominant child from hogging all the attention, the playground manages to solve the con-flict and later persuade the children to play together.

adapting to Children with disabilitiesCerebral paresis is a permanent deficiency in motion function that can be treated using therapy. Con-sider a scenario where two children with cerebral paresis play in an inter-active playground, where differently shaped objects are placed throughout the space. The children like horses and, on a big screen, a cartoon horse is dis-played that asks them to find a specific shape. When a child returns with the object, he or she presses it on a grid of pressure sensors like a stamp. This will enforce a specific grip that the child must practice. The cartoon horse is happy when the children can keep the grip for a sufficient amount of time. It also offers encouragement when two children play together. The children


can help each other find objects to in-crease social interaction between them. To make the game more challenging, objects such as ladders or ropes can be added to train additional hand grips.

The use of interactive technology to stimulate children with cerebral paresis to train their grip seems promising, es-pecially when combined with personal-ization in new ways of therapy.16 Also, for children with a mild intellectual dis-ability, interactive installations can lead to enjoyable experiences.8 However, the development of the sensing technology, the realization of the environment, and the evaluation of the effects for sensi-tive target groups, such as children with physical or mental disabilities, is chal-lenging. Classical principles of user-centered design and standard methods of evaluation won’t always work.

People with disabilities might have difficulty expressing themselves or might express their states and inten-tions differently, leading to misunder-standings. The same is true for their perception, depending on whether their hearing, vision, or cognition is affected. In addition, there’s the risk of overstimulation, which can cause se-vere problems. Automatic sensing and interpretation technology thus must be tailored to such user groups, requiring a highly personalized approach. In ad-dition to offering interactive feedback on the playground, such an approach might also support other kinds of mon-itoring and care or help diagnose mental or social impairments, such as autism.

social signal ProcessingSSP is an emerging research area that focuses on the sensing, interpretation, and reproduction of social signals—ele-ments that humans use when communi-cating nonverbally. SSP applies theories from the behavioral sciences using prac-tices from computer science fields, such as machine learning and computer vi-sion. As such, SSP studies how humans produce, interpret, and detect social signals. It also considers how computer systems can be equipped with these

characteristics to make them socially intelligent. Although SSP encompasses distinct research fields, collaboration between researchers is possible because the fields share common principles.6

studying social signalsA social signal is defined as a “com-municative or informative signal that, either directly or indirectly, conveys in-formation about social actions, social interactions, social emotions, social attitudes and social relationships.”17 Social signals are often ambiguous, and their interpretation depends on the environment and must account for the inherent uncertainty of recognition algorithms or cues exhibited at differ-ent time scales.

Nonverbal cues aren’t always asso-ciated with a single social signal, and the simultaneous presence of other cues might indicate a totally different social signal. Behavioral cues exhibited in parallel thus shouldn’t be excluded from any analysis. Context has a big influence on the social signals people elicit, and this variation is a key chal-lenge in SSP.18 This applies not only to the interpretation of social signals from various modalities but also to signals from multiple people.

Pervasive ssPSSP research has typically addressed the automatic audiovisual processing of social signals in controlled environ-ments, such as meeting rooms, offices, or debate stands.19 Most of these stud-ies analyze face-to-face or small group interactions, because they represent the most common forms of interac-tion.20 Information in these settings is typically measured across modalities to

ensure high quality and avoid missing cues that aren’t present in a given modality.

For example, numerous studies use sensors that measure gaze or facial expressions. This restricts not only sensor placement but also the type of behavior that can be exhibited. When facial expressions are being observed, the subject under analysis can only move in the limited area covered by the camera.

In contrast, interactive playgrounds should provide an immersive, fun, and engaging experience. Therefore, sen-sors that could restrict children’s be-havior or make them uncomfortable are inappropriate. For example, equip-ping children with microphones facili-tates the measurement of vocal cues but restricts movement, because chil-dren typically run, crouch, and bump into each other during play. Instructing them to mind the microphones would prevent them from playing as they nor-mally would, diminishing the levels of engagement and immersion.

Interactive playgrounds thus must consider sensors that can measure in-formation unobtrusively, embedding them into the playground such that they don’t obstruct children’s play. Some SSP researchers have recently started

exploring this direction, processing social signals in surveillance settings where cameras are located at consid-erable distances from the subjects.21 Such a setting facilitates studying social and affective information that drives human behavior, especially kinesics and proxemics.

Kinesics. Kinesics concerns the study of body movements as a mode of

Interactive playgrounds should provide an

immersive, fun, and engaging experience.

sensors that restrict children’s behavior

are inappropriate.



communication (body language).22 Postures and gestures are important kinesic cues. The former refer to static body configurations, while the latter are movements of the body over time, typically performed with the hands or arms. Both can have both a communi-cative and affective meaning.

For example, crossing the arms is considered a closed body pose that might convey boredom or disagree-ment. In addition to the conscious display of body language, the analysis of social signals also focuses on body movements exhibited unconsciously.

In interactive playgrounds, these ad-vances can be used to determine the affective state of the players. As such, it can be determined whether players are engaged or frustrated by the cur-rent gameplay. For example, reduced movement of the arms, as well as a less upright posture, might be cues of diminished engagement. In addition, these techniques provide opportunities to further analyze interactions between players. The amount of interpersonal synchrony or the imitation of postures and movements between players might provide valuable social information, in-dicative of the type of play that occurs in the playground.

Proxemics. Proxemics refers to the study of how people use the space around them in social settings—that is, how they group together and arrange them-selves.23 This includes not only the dis-tance between individuals but also the physical arrangement of groups. The idea that individuals negotiate personal space during interaction dates back to Michael Argyle and Janet Dean, who observed that two people dynamically adapt their physical proximity, pos-tures, gestures, and gaze depending on the level of psychological intimacy be-tween them.24

Similarly, Edward Hall suggested that interpersonal distance is related to social distance and proposed four concentric circles corresponding to different levels of regulation when

interacting.23 The social distances range from personal to public relation-ships and are affected by space con-straints. Marco Cristani and his col-leagues conducted an experiment with groups of people gathered inside a de-fined area, which shrank during several iterations of social interaction.25 They observed that interpersonal distances were relative to social distances, even though the space constraints changed.

Another common concept in prox-emics research is the F-formation, which “arises whenever two or more people sustain a spatial and orienta-tional relationship in which the space between them is one to which they have equal, direct, and exclusive ac-cess.”26 F-formations represent com-mon patterns in which people arrange themselves in social settings such that all participants can maintain eye con-tact with the other group members. Researchers have used this informa-tion to identify groups in social set-tings based solely on people’s positions and orientations.27

The notion of proxemics is gain-ing attention in the computer vision domain, where considering the social and psychological forces that influ-ence grouping behavior has been found to improve our understanding of how groups of pedestrians behave.

For example, Georg Groh and his colleagues developed a system that classified when people were engaging in social interactions by observing in-terpersonal distances and body orien-tation.28 Similarly, Loris Bazzani and his colleagues used body orientation along with spatial cues to infer when and with whom people were interact-ing.29 These studies argue that human behavior analysis shouldn’t be focused solely on the individual but on groups, because people naturally group into higher-level structures that affect and are affected by the actions of the indi-viduals. Modeling this relationship can improve the understanding and recog-nition of human behavior of both indi-viduals and groups.

These recent advances in a surveil-lance setting have paved the way for unobtrusive behavior sensing and in-terpretation in interactive playgrounds. However, the variation in behavior of children in free play is arguably higher than that exhibited by groups of pedes-trians. On the other hand, the more controlled nature of interactive play-grounds enables the more robust and detailed observation of behavior. These differences present both challenges and opportunities for research and application.

toward socially aware interactive PlaygroundsEnhancing interactive playgrounds to automatically sense, interpret, and in-duce social behavior requires sensing behavior and offering feedback, mod-eling players’ profiles and appropriately adapting the system, and evaluating the playgrounds.

sensing Behavior and Providing FeedbackTo interpret children’s social behavior, sensors must capture behavior in suf-ficient detail. In addition, actuators should be used to provide feedback to the players to induce certain behavior. Both should be embedded into the en-vironment to let children play without being hindered or distracted.

Sensing. Cameras are commonly used in interactive playgrounds to measure grouping, position, interpersonal dis-tances, postures, and gestures. Sev-eral affordable types of cameras are currently available, each suitable for particular settings or to achieve par-ticular goals. Infrared cameras are useful in low-illumination conditions or when using projectors. Stereo vi-sion, time-of-flight cameras, or pat-tern overlays, such as those used in Microsoft’s Kinect, allow for the es-timation of depth, which facilitates the segmentation of foreground ob-jects. Computer vision algorithms are required to handle variations in


viewpoints, occlusions, and interper-sonal appearance.

Microphones could be used to ob-tain the level and type of sound, such as yelling or singing, as indicators of fun, immersion, and engagement. In addition, they can be used to analyze patterns of imitation and synchrony. When multiple microphones are em-ployed, the source of the sound can be determined. This is useful for the local-ization of players in more cluttered en-vironments, where cameras don’t have an unobstructed view.

Sensors can also be embedded into the environment or the objects therein. Ac-celerometers are widely used to recognize actions, sudden movements, and physical activity levels. Similarly, pressure sensors have been embedded in many kinds of objects such as mats,8 walls,9 and floor tiles. They can be used to track positions, detect physical arrangements, and recog-nize subtle differences in actions such as stepping or stomping. They can even be used to identify people by creating walk-ing profiles. The main disadvantage of accelerometers and pressure sensors is the limited amount of information that they provide.

We should keep in mind the limita-tions of recognizing social interactions during play from these sensors. Despite the advances in SSP, some social inter-actions remain ambiguous and hard to identify, especially with coarse data—for example, when using a distant camera or microphone. Multimodal approaches are beneficial if modalities carry complementary information.12

Feedback. Projectors are widely used ac-tuators that can be placed almost any-where in interactive playgrounds. They can display a wide variation of images, shapes, and animations. They require a projection surface, and the projector might need to be located fairly far from the playground to maximize this sur-face. Children might be between the projector and the projecting surface, casting shadows on the latter. This can be reduced by using multiple projectors

to fill in the image. LEDs are other light sources, but their limited size and low energy demands make them more suitable in building blocks or buttons to signal their presence and affor-dance.9,10 Furthermore, light fixtures, including spotlights and stage lighting, can be used for visual feedback.

Speakers can be located virtually anywhere, even outside the playground. They can provide sound effects in re-sponse to actions performed by the chil-dren or can play music during the game.4 Speakers might interfere with the speech of the children, so careful volume con-trol is recommended. Alternatively, di-rectional speakers could be employed.

Haptic feedback employs technolo-gies that exploit the sense of touch. Many devices can provide haptic feed-back using accelerometers or pressure sensors embedded in objects. Touch carries a strong emotional component, and mediated touch thus focuses on so-cial touch and affective interaction.30

User Modeling and adaptationAspects such as personal experience, culture, age, gender, and physical

and cognitive abilities influence how children play, so they must be care-fully considered when designing play-grounds. This is especially true in user modeling and the adaptation of game mechanics. We discuss two strategies for adapting game mechanics that don’t depend on the characteristics of the playground and thus can be applied in a wider range of scenarios.

One strategy is to change the game mechanics when specific events or be-havior are observed. This strategy is es-pecially useful when behavior is sensed that should be stimulated or discour-aged. This might be the same behav-ior for all players or different types of behavior depending on the specific player. For example, rough-and-tumble play often occurs in games that favor competition,12 but when this behavior evolves into aggression, it should be discouraged. When a playground de-tects rough-and-tumble play, the game mechanics could change to encourage more passive or collaborative play and prevent further hostility. In this ap-proach, thresholds for detecting ag-gression might differ between players.

Figure 2. Children engaging in play in an interactive playground.4 Shapes were projected onto the floor while music played. Cameras detected the children’s movement.



Another strategy is to create personal user profiles to adapt game mechanics to meet the player’s expectations or style of play. The benefit of this user modeling, personalization, and con-sequent adaptation is that children re-main immersed in the experience. This can be achieved by developing models that reflect the player’s behavior or preferences.

For example, Alireza Derakhshan used a platform that learned (using neural networks) to recognize several player styles based on tempo, age, and continuity of play.10 The game tempo was adapted, along with other charac-teristics, to match what the player felt comfortable with to successfully pro-mote physical activity.

Along these lines, children could be given different roles depending on their personality. Koen Hendrix and his col-leagues designed a board game aimed at helping shy children overcome their social incompetency.11 User profiling was carried out prior to the game by asking the teachers to fill in question-naires to identify shy children. The gameplay adaptation was simple: out of two roles, architect and builder, shy children were always given the architect role, which required leading others to successfully complete the game.

These two strategies could also be combined to adapt the gameplay. A playground could learn about players during play and apply the information in later sessions while still reacting to certain events. Moreover, given enough time, the playground could learn the player’s profile and personalize the game throughout the play session. This would be especially suitable for children who need special care, because the sensed be-havior could be used to adapt the game when they’re feeling discomfort.

User experience evaluationTo determine whether interactive play-grounds have met their goal of provid-ing the players with rich experiences, the interactions in the playground must be evaluated quantitatively or qualitatively.

In addition, other goals, such as the promotion of specific behavior, should be observed. In this case, validating whether the behavior has been encour-aged becomes part of the evaluation.

The evaluation of entertainment in-stallations—particularly those aimed at free play—is difficult.31 Current play-grounds are usually evaluated using questionnaires, group discussions, or observational studies. SSP could help in the evaluation process by automatically measuring raw data instead of anno-tated, subjective data. This could help better determine whether goals were achieved. For example, interpersonal synchrony is usually attributed to rap-port and a feeling of belonging,32 which could imply high levels of engagement and fun. Automatically measuring syn-chrony could thus help evaluate some forms of user experience. In a similar vein, body movement is positively cor-related with players’ perceived level of engagement.33 Moreover, if the high-end goal of the playground is to pro-mote social interaction, automatically recognizing social behavior could help decide whether the goal was met with-out requiring manually annotated and recorded play sessions.

T he benefits of using SSP in playgrounds go beyond improving the game expe-rience for the players. For

example, the evaluation of playgrounds could be automated by considering be-havioral and affective cues interpreted from the observed behavior. It would also allow for the study of play in a broader sense—such as how changing the game mechanics affects play. Furthermore, the continuous sensing and interpretation of behavior could benefit groups of children requiring constant care and supervision. The interactive playground then also adopts a monitoring task.

Introducing socially aware tech-nology in interactive playgrounds of-fers many opportunities for improved game experience in a broad range of

scenarios. With advances in SSP, we ex-pect current challenges to be addressed in the near future.

ACknowledgmentSthis publication was supported by the Dutch national program COmmIt.

ReFeRenCeS 1. A.D. Pellegrini, The Role of Play in

Human Development, Oxford Univ. Press, 2009.

2. J. Soler-Adillon and N. Parés, “Interactive Slide: An Interactive Playground to Pro-mote Physical Activity and Socialization of Children,” Proc. Int’l Conf. Human Factors in Computing Systems (CHI 09), ACM, 2009, pp. 2407–2416.

3. T. Bekker, E. Hopma, and J. Sturm, “Creating Opportunities for Play: The Influence of Multimodal Feedback on Open-Ended Play,” Int’l J. Arts and Technology, vol. 3, no. 4, 2010, pp. 325–340.

4. D. Tetteroo et al., “Design of an Inter-active Playground based on Traditional Children’s Play,” Proc. Int’l Conf. Intel-ligent Technologies for Interactive Enter-tainment (Intetain 11), Springer, 2011, pp. 129–138.

5. B.A.M. Schouten et al., “Computer Analy-sis of Human Behavior,” Human Behavior Analysis in Ambient Gaming and Playful Interaction, Springer, 2011, pp. 387–403.

6. A. Vinciarelli et al., “Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Pro-cessing,” IEEE Trans. Affective Comput-ing, vol. 3, no. 1, 2010, pp. 69–87.

7. K.R. Ginsburg, “The Importance of Play in Promoting Healthy Child Development and Maintaining Strong Parent-Child Bonds,” Pediatrics, vol. 119, no. 1, 2007, pp. 182–191.

8. P. Wyeth, J. Summerville, and B. Adkins, “Stomp: An Interactive Platform for Peo-ple with Intellectual Disabilities,” Proc. Conf. Advances in Computer Entertain-ment (ACE 11), ACM, 2011, p. 51.

9. H. Ouchi et al., “Detecting and Modeling Play Behavior Using Sensor-Embedded Rock-Climbing Equipment,” Proc. Int’l Conf. Interaction Design and Children (IDC 10), ACM, 2010, pp. 118–127.

10. A. Derakhshan, F. Hammer, and H. Lund, “Adapting Playgrounds for Children’s Play Using Ambient Playware,” Proc. Int’l


Conference Intelligent Robots and Systems (IROS 06), IEEE, 2006, pp. 5625–5630.

11. K. Hendrix et al., “Increasing Children’s Social Competence through Games, An Exploratory Study,” Proc. Int’l Conf. Interaction Design and Children (IDC 09), ACM, 2009, pp. 182–185.

12. A. Moreno et al., “An Annotation Scheme for Social Interaction in Digital Play-grounds,” Proc. Int’l Conf. Entertain-ment Computing (ICEC 12), Springer, 2012, pp. 85–99.

13. G.N. Yannakakis, J. Hallam, and H.H. Lund, “Entertainment Capture through Heart Rate Activity in Physical Interac-tive Playgrounds,” User Modeling and User-Adapted Interaction, vol. 18, nos. 1–2, 2008, pp. 207–243.

14. Y.-G. Cheong et al., “A Computational Approach towards Conflict Resolution for Serious Games,” Proc. Int’l Conf. Foundations of Digital Games (FDG 11), ACM, 2011, pp. 15–22.

15. J. Schell, The Art of Game Design: A Book of Lenses, Morgan Kaufmann, 2008, p. 148.

16. R. Delden, P. Aarts, and B. Dijk, “Design of Tangible Games for Children Undergoing Occupational and Physical Therapy,” Proc. Int’l Conf. Entertainment Computing (ICEC 12), Springer, 2012, pp. 221–234.

17. I. Poggi and F. D’Errico, “Cognitive Mod-elling of Human Social Signals,” Proc. Int’l Workshop on Social Signal Process-ing, ACM, 2010, p. 21.

18. L. Riek and P. Robinson, “Challenges and Opportunities in Building Socially Intelligent Machines,” IEEE Signal Pro-cessing Magazine, vol. 28, no. 3, 2011, pp. 146–149.

19. M. Cristani et al., “Human Behavior Analysis in Video Surveillance: A Social Signal Processing Perspective,” Neuro-computing, vol. 100, 2013, pp. 86–97.

20. D. Gatica-Perez, “Automatic Nonver-bal Analysis of Social Interaction in Small Groups: A Review,” Image and Vision Computing, vol. 27, no. 12, 2009, pp. 1775–1787.

21. G. Zen et al., “Space Speaks: Towards Socially and Personality Aware Visual Surveillance,” Proc. Int'l Workshopp Multimodal Pervasive Video Analysis (MPVA 10), ACM, 2020, pp. 37–42.

22. R. Birdwhistell, Kinesics and Context: Essays on Body Motion Communication, Univ. of Pennsylvania Press, 1970.

23. E. Hall, The Hidden Dimension, Double-day, 1966.

24. M. Argyle and J. Dean, “Eye Contact, Distance, and Affiliation,” Sociometry, vol. 28, no. 3, 1965, pp. 289–304.

25. M. Cristani et al., “Towards Computa-tional Proxemics: Inferring Social Rela-tions From Interpersonal Distances,” Proc. Int’l Conf. Social Computing (SocialCom 11), IEEE, 2011, pp. 290–297.

26. A. Kendon, Conducting Interaction: Pat-terns of Behavior In Focused Encounters, Cambridge Univ. Press, 1990, p. 203.

27. H. Hung and B. Kröse, “Detecting F-For-mations as Dominant Sets,” Proc. Int’l Conf. Multimodal Interfaces (ICMI 11), ACM, 2011, p. 231.

28. G. Groh et al., “Detecting Social Situa-tions from Interaction Geometry,” Proc. Int’l Conf. Social Computing (SocialCom 10), IEEE, 2010, pp. 1–8.

29. L. Bazzani et al., “Social Interac-tions by Visual Focus of Attention in

a Three-Dimensional Environment,” Expert Systems, vol. 30, no. 2, 2013, pp. 115–127.

30. G. Huisman, “A Touch of Affect: Medi-ated Social Touch and Affect,” Proc. Int’l Conf. Multimodal Interaction (ICMI 12), ACM, 2012, pp. 317–320.

31. A. Nijholt, D. Reidsma, and R. Poppe, “Human-Centric Interfaces for Ambient Intelligence,” Games and Entertainment in Ambient Intelligence Environments, Academic Press, 2009, pp. 393– 413.

32. L. Miles, L. Nind, and N. Macrae, “The Rhythm of Rapport: Interpersonal Syn-chrony and Social Perception,” Experi-mental Social Psychology, vol. 45, no. 3, 2009, pp. 585–589.

33. M. Pasch et al., “Immersion in Move-ment-Based Interaction,” Proc. Int’l Conf. Intelligent Technologies for Interactive Entertainment (Intetain 09), Springer, 2009, pp. 169–180.

the AuthoRSAlejandro Moreno is a phD candidate in the Human media Interaction group at the university of twente. His research interests include the automatic analy-sis of human social interactions, and he’s interested in bridging different fields, such as computer vision, social signal processing, and entertainment tech-nologies. more specifically, he studies social behavior of children during play in interactive playgrounds. moreno received his msc in color informatics and multimedia systems from the Erasmus mundus CImEt program. Contact him at [email protected].

Robby van Delden is a phD candidate working on socially adaptive interactive playgrounds in the Human media Interaction group at the university of twente. His research focuses on two aspects of socially adaptive interactive playgrounds: the automatic measurement of social behavior, especially nonverbal synchrony, and the design of ambient entertainment installations that direct social behav-ior. His work is directed toward finding the right connection between sensing and inducing social behavior. Van Delden has two master’s degrees, one in industrial design engineering and the other in human media interaction, both from the university of twente. Contact him at [email protected].

Ronald Poppe is a postdoctoral researcher in the Human media Interaction group at the university of twente. His research interests include the measure-ment and interpretation of human nonverbal behavior from videos and other sensors and the modeling and generation of communicative behavior for virtual characters in human-computer interaction. poppe received a phD in computer science from the university of twente. Contact him at [email protected].

Dennis Reidsma is an assistant professor at the Human media Interaction group and lecturer of the Creative technology curriculum at the university of twente. His research interests include computational entertainment and inter-active playgrounds, and he runs several research projects in this area. reidsma received his phD from the Human media Interaction group of the university of twente. His phD thesis was titled, “Annotations and subjective machines—of Annotators, Embodied Agents, users, and Other Humans.” Contact him at [email protected].

selected Cs articles and columns are also available for free at http://ComputingNow.computer.org.

socially aware interactive playgrounds

Documents