the kinect effect

Tracing the constituents of a technical artefact

eric alberts (3485595)

august 2012

ma thesis (mcmv10009) new media & digital culture

dr. ann-sophie lehmann (first reader)dr. imar de vries (co-reader)

the kinect effect

Kinect enacted: exploring Microsoft’s development incentives

Preface and acknowledgements

Conclusion

Epilogue

Notes

Kinect evolved: implementing hacks and building on dreams

References

Attachment - Interview with Thijs van Doorn (Microsoft NL)

Introduction

Kinect envisioned: an utterance of an interface sublime

The Kinect Effect

Nintendo and the revival of gestural gaming

A hacker’s delight

Opening the black box of technology

From GUI to NUI

The decay of a software giant

Gaining control of evolution

Kinect as object of study

Virtual reality and the revaluation of human-computer interaction

The struggle upwards (From GUI to NUI cont’d)

Dreams incorporated

Structure

How dreams are imprinted on Kinect

Contents

1. 24

10

6

10121821

242936

40404348

54545864

71

76

78

94

110

2.

3.

6

Preface and acknowledgements

My very own Kinect sits at the base of my television for several months now. The black tinted peripheral for the Xbox 360 video game console has become a part of my interior as much as the living room plant in the vase next to it. For someone who grew up with remote controllers in all shapes and sizes, the act of waving at a screen without holding some sort of device in my hand feels odd, yet, at the same time, quite familiar. Cycling through on-screen menus by using gestures still requires some getting used to, but it is probably even more intuitive than pinching, dragging, and flicking your way through applications on a tablet or smartphone. Kinect is an intriguing device that has rapidly found its way into the homes of many families but of which we have seen only the beginnings of its true potential. Soon after I started working part-time for Microsoft my fascination for Kinect grew and I realised my master’s thesis offered an excellent opportunity for finding answers that can help explain why we now have this specific technical artefact.

The reason why this study is called ‘The Kinect Effect’ is twofold. First, the title refers to the label Microsoft has given to the widespread hobbyist appropriations that emerged almost instantly after Kinect for Xbox 360 became available to consumers. As will become clear in the following chapters, this is a well-contemplated decision that exemplifies the company’s intentions to gain control over the evolution of Kinect. Second, the title refers to a tendency in popular (news) media and in some fields of

7

academia to explain and value technology based on the effects it produces – something that prevents obtaining a clear understanding of the true role of technology within politics, economy, culture, and society. I would like to regard this study into the constituents of Kinect as an example of studying technology from the inside – acknowledging the complexity and multifaceted nature of technological phenomena, which play an increasingly important role in today’s information society.

Approximately three years ago I was an upcoming graduate, en route to complete a four-year long education in multimedia design. On the one hand, I felt I was ready for the working life, as I had gathered four years of hands-on experience designing for new media. On the other hand, there still was this eagerness to learn more about the complexities of the new media I had learned to design for. Three years later, I am again an upcoming graduate. This time around, however, the nagging feeling I might miss the opportunity to learn more about the promiscuities of new media has lost its predominance. This change can be entirely attributed to the master programme New Media and Digital Culture at Utrecht University. This master programme is more than a suitable remedy to fulfil the need to learn all about the relationships between society, culture, politics and technology. I, therefore, sincerely thank all the people involved in the master programme who work tirelessly to offer students like myself a curriculum of the highest possible quality.

I, furthermore, thank my classmates who have made the past two and a half years a more than pleasant time. I look forward to our next barbecue and hope there will be many more after that. I am also grateful for the unconditional

8

the kinect effect

support of my parents who endorsed my choice to follow this master programme from the beginning and without question. I will never grow reluctant to explain the contents of this master programme anew, even after graduation. Last but not least, I thank my girlfriend for making me aware of this master programme, for her support, and especially for her inexhaustible patience.

Breda, August 2012

10

On November 4, 2010, the Redmond-based software giant Microsoft released Kinect: a small motion sensing, webcam-like peripheral for the company’s game console Xbox 360.1 Kinect enables users to interact with the Xbox 360 through a natural user interface (NUI) using hand-gestures and voice commands instead of a physical controller with its many buttons.2 Exactly one year after launch, Kinect officially claimed a Guinness World Record for fastest-selling consumer electronics device. With approximately 133 units sold a day, either as standalone unit or bundled with the Xbox 360, Kinect was even able to outsell Apple’s massively popular iPhone and iPad (Pereira 2011a). On the same day Microsoft released these impressive sales figures the company also launched a new blog and website dedicated to a rather different Kinect-endeavour. The first post to the new blog reveals that Kinect will be the spindle in a new commercial programme running parallel to the successful activities unfurled within the gaming periphery. The programme, called “Kinect for Windows”, features a software development kit (SDK), a developer programme, and Kinect hardware device specifically optimised for Microsoft’s operating system (OS) Windows (Eisler 2011a).3

Introduction

The Kinect Effect

It all started with a revolutionary sensor and amazing software that turned voice and movement into magic (Craig Eisler –

General Manager, Kinect for Windows – 2011a)

11

Given the success of Kinect for Xbox 360, the choice to bring Kinect to the PC seems logical. According to Craig Eisler, General Manager of Kinect for Windows, Kinect for Xbox 360 ‘showed the world how to re-imagine entertainment. Next year, with Kinect for Windows, we will help the world re-imagine everything else’ (Eisler 2011a). There is, however, another factor that played a significant role in Microsoft’s choice to push Kinect beyond gaming and media. In a short follow-up post published six days after the Kinect for Windows announcement, Craig Eisler first mentions “The Kinect Effect”. According to Eisler, this is a term Microsoft started using shortly after the launch of Kinect for Xbox 360 ‘to describe the amazing and creative ways Kinect was being applied to fields beyond gaming’ (Eisler 2011b). Accompanied by a short clip containing visionary depictions of various future Kinect applications, Eisler continues by stating that ‘it’s this potential that drives the enthusiasm my team and I have for developing Kinect for Windows’ (ibid.). Interestingly enough, the larger part of what Microsoft calls The Kinect Effect was almost entirely an accident and evolved largely outside and in spite of Microsoft (Carmody 2012).

According to renowned tech-magazine Wired, before Kinect for Xbox 360 launched in 2010, hackers4 already ‘foamed at the mouth to reverse-engineer and control the device to their heart’s desires’ (Mosher 2011). Within a few days after Kinect became available, freelance hacker Héctor Martin cracked the device’s source code and was awarded a three thousand US dollar bounty by do-it-yourself initiative Adafruit Industries5 (ibid.). Microsoft responded edgy to the hacking contest, telling technology website CNET that the company ‘does not condone the modification of its products

12

the kinect effect

[and will] work closely with law enforcement […] to keep Kinect tamper-resistant’ (Terdiman 2010a). Remarkably, only a few weeks after this initial hard-line response, two Microsoft employees went on the air stating how the company was now inspired by how fans and hobbyists were adapting Kinect (Terdiman 2010b; Carmody 2010a). The few weeks in between these opposing statements something must have happened causing Microsoft to change its position on the various Kinect appropriations.

A brief look into the events preceding the launch of Kinect for Windows, roughly described above, defuses the claim that the twinkling of dollar signs in the eyes of the corporate executive was the sole incentive for Microsoft to steer Kinect towards the PC. The numerous Kinect alterations must have also weighed in on the decision, as hobbyists opened doors to all sorts of fields ranging from robotics to medical research. The prelude to the emergence of Kinect for Windows gives rise to the question whether there are more factors that have steered Kinect towards the PC. In order to acquire a comprehensive understanding of why we have specific technical artefacts, such as Kinect, it is necessary to be aware of the heterogeneous character of technological development. In order to account for this heterogeneity one should take on a broad perspective and focus on the various constituents rather than on the effects of the technical artefact.

Opening the black box of technologyIn his historiography of the electrical power system, historian Thomas Hughes stresses the necessity to realise that technological affairs ‘contain a rich texture of technical matters, scientific laws, economic principles, political

13

introduction

forces, and social concerns’ (Hughes 1983, 1). According to Hughes, one ‘must take the broad perspective to get to the root of things to see the patterns’ (ibid.). When regarding a technical artefact as self-evident and obvious to the observer the complex relationships that constitute it are rendered invisible. The technical artefact is consequently “black-boxed” – making it difficult to answer questions such as why and how we have the technologies that we do. In order to answer these questions the black box needs to be opened – investigating the ways in which a variety of social and technical aspects are associated and come together as a durable whole (Cressman 2009, 6). To assume that Kinect for Windows is the inevitable next step in the device’s successful career, for instance, is to dismiss the complexity and multifaceted nature of this technical artefact.

Examining how Kinect is constructed is an effort to open the black box – making visible the relationships that constitute it. This approach of technology owes much to actor-network theory (ANT) developed and pioneered over the past decades by the likes of social scientists Bruno Latour, John Law, and Michel Callon. In essence, ANT rejects the traditional linear model of scientific and technical development. In a linear model, researchers emphasise the materiality and effects of technology, showing little to no interest in its emergence, how it is shaped, or even in the details of its uses (Marx and Smith 1994, x; Flichy 2007a, vi). A technical artefact, then, is rendered a given – either coming out of nowhere or from the brain of a brilliant inventor. From the 1980s onwards, social scientists like Latour began opposing this “ready-made” model of scientific and technical development. They started paying attention to how technologies are used in research laboratories, at

14

the kinect effect

home, and at the office by examining technical artefacts – both successes and failures – from the inside: describing the processes of trail and error and the different routes taken (Latour 1987).

From the electric car (Callon 1987) to the failed TSR2 aircraft (Law and Callon 1992) and the 17th century Portuguese expansion (Law 1987), ANT offers a solid constructivist approach to the study of interactions between elements or “actors” that make up the technical artefact. When approaching Kinect from the perspective of ANT, one would regard this technical artefact as a heterogeneous network that consists of various human and non-human actors. This network should then be “reassembled” in order to reveal how and to what purpose particular actors are connected to each other (Latour 2005). By considering both human and non-human elements equally as actors enrolled within a network, ANT offers an exploded view that makes visible the entanglements between economical, political, social, and technological elements that make up the technical artefact.

Taking on a broad perspective on technological affairs, moreover, implies we also have to be aware that technological development occurs in specific historical and social contexts. The work of cultural historian Wolfgang Schivelbusch, for instance, shows us how a broad concept of history can shed light on the ways in which technical artefacts are embedded in the complex discursive fabrics and patterns reigning in a culture.6 The shift from an object-centred towards a multi-layered account of history has led media historians to acknowledge that a society can formulate specific ways and means of what new media are supposed to do. By placing technologies into their cultural and discursive contexts,

15

introduction

media historians have found how the ways in which we think, talk, and write play an important role in the construction of technologies. On meeting the discourses attending the development and diffusion of the steam engine (Marx 1964), electricity (Marvin 1988), the wireless (Douglass 1988), the telephone (Ronell 1989), aviation (Corn 1983), the moving image (Boddy 1994), digital television (Flichy 1999) and the Internet (Flichy 2007a), media historians have discovered how these technologies can act as catalysts or vehicles for the expression of ideas about human existence and social life (Lister et al. 2009, 67).

The discursive construction of technology entails that the usage of a particular technological artefact can be socially “framed” in different and sometimes contradictory ways. This is something sociologist Patrice Flichy argues with regard to the emergence of digital television in the 1990s. In his article ‘The Construction of New Digital Media’ Flichy describes how three imaginary uses or “imaginaires” of digital television did not necessarily correspond with credible expectations of how the medium was likely to be used. Instead, each of the three imaginaries was a representation of a particular use and corresponded with the context in which it was created.7 In other words, the conditions that set the frame of reference for digital television were social rather than technical (Flichy 1999, 34). The “imaginaire” or imaginary has its roots in psychoanalytic theories of Jacques Lacan and is understood as a realm of images, representations, ideas and intuitions of fulfilment, wholeness and completeness that human beings project onto an “other”. When applied to technology, the role of the other is performed by some new technological achievement onto which dissatisfactions with social reality and hopes for

16

the kinect effect

a better society are projected (Lister et al. 2009, 67).The significance of the imaginary in the development

of technology is further emphasised in Flichy’s work The Internet Imaginaire in which he distillates a collective vision tying together different actors related to the construction of the Internet. In his introduction, Flichy states that ‘by focussing on the management of technical projects, we see to what extent the construction of the technical object is connected to the values of the environment in which it was born’ (Flichy 2007a, 5). Research undertaken by Carolyn Marvin, moreover, shows how the technological imaginary is well at work long before technical possibility becomes technical reality. As early as in the late nineteenth century, before the introduction of most twentieth century breakthrough media (e.g. radio, television), electricity already invigorated ‘endless fascination and fear, and provided constant fodder for social experimentation’ (Marvin 1988, 4). Research shows that the technological imaginary is a considerable element we should bare in mind when opening the black box of technology.

Research of the technological imaginary shows us that when new technologies become socially available they are understood in terms of existing cultural values (Lister et al. 2009, 67). Histories of technologies, then, form our contemporary responses to new technologies. In this sense, as media historian Erkki Huhtamo notes, ‘history belongs to the present as much as it belongs to the past’ (Huhtamo 1996). This claim is substantiated by the sense of “historical déjà vu” frequently taking hold of media historians when studying the many discourses attending the emergence of technologies.8 The sense of déjà vu does not so much concern the actual historical repetition of

17

introduction

technologies themselves but rather a reoccurrence of deeply ingrained ways in which we think, talk, and write about these technologies (Lister et al. 2009, 69). In other words, scholars have found certain reoccurring cyclical elements in the analyses of discourses attending the process of adopting technological inventions, ‘which (re)appear and disappear and reappear over and over again in media history and seem to transcend specific historical contexts’ (Huhtamo 1996). Such findings underscore the importance of realising that technical artefacts are not in a permanent fixed state but rather in a permanent state of flux, embedded in and shaped by specific social contexts.

Taking on a broad perspective on the emergence and development of Kinect implies that I need to make visible the relationships between actors that constitute this technical artefact whilst being aware of social forces, such as the technological imaginary, orienting these actors in their mutual configurations. Methodologically, ANT appears to offer the necessary tools for such an analysis. As Patrice Flichy points out, however, this method ‘cuts out the study of initial intentions and innovative projects’ (Flichy 2007a, 3) thereby dismissing the notion of an underlying framework or a collective vision orienting technological development. When using an ANT approach exclusively, the thought that Kinect might in part be the product of an “imaginaire” would be shunned. Notwithstanding this methodological conflict, I shall draw upon both the theoretical principles of ANT and the notion of social forces “framing” technological development in order to obtain a complete understanding of Kinect’s development. An argument for why I need both approaches in conjunction can be found in Microsoft’s video The Kinect Effect, which depicts how Kinect could work in

18

the kinect effect

areas other than gaming. The release of this video suggests that the company is well aware of Kinect’s social context and actively responds to imaginary uses accompanying the device in order to influence its further development.

In the case of Kinect it is necessity to examine how different views on the way the device “should work” have influenced the development of the device. Reoccurring hopes and dreams about how technology should work do not only accompany the development process but also orient this process and offer a course of direction around which the technical artefact will eventually be articulated. According to historian Joseph Corn, even the most naïve fantasies ‘are part of the same cultural milieu in which actual invention takes place and technology is adopted and diffused’ (Corn 1987, 228). Technologies certainly depend on actual and detailed paths taken but we should, at the same time, be aware that at the roots of their ideological framework there lie specific myths that are powerful moving forces behind technology (Bailey 2005, 32). Despite the theoretical discrepancy between the approaches of Latour and Flichy, I need to draw from both approaches in order to touch upon essential elements in Kinect’s development.

Kinect as object of studyThis analysis of Kinect entails a micro scale study of the relationships that constitute this specific technical artefact in which proper attention is paid to macro scale elements such as the regulative force of myths in technological development. In order to achieve an analytical combination of the micro and macro scale I would like to build on the multiscalar approach proposed by media scholar Imar de Vries in his work Tantalisingly Close.9 In order to understand

19

introduction

myths of idealised communication manifested in discourses of mobile wireless communication technologies, De Vries proposes an ‘evolutionary account of technological history’ (De Vries 2012, 95). This approach draws upon analogies from the theory of biological evolution to circumnavigate the limitations of approaches such as technological determinism and social constructivism.10 In an evolutionary account of technology three characteristics can be distinguished:

First, a non-teleological view on history that resists the temptation to only concentrate on ‘big breakthroughs’ or ‘brilliant inventors’, while still acknowledging the occurrence of chance discoveries or sudden events that quickly surpass paradigmatic boundaries; second, a focus on the importance of environmental factors that can account for the diversity of different uses and appropriations of technology, and finally, a recognition of forceful fantasies, moving myths and seductions of the sublime that work as recurrent selection mechanisms within the evolutionary process. (ibid., 96)

An important reason why an evolutionary understanding of technology can be advantageous for studying Kinect is because it accounts for the multitude of factors that make media evolution an erratic and unpredictable process without shying away from the notion that these environmental factors might partially be oriented by an underlying framework (ibid., 23). De Vries refers to the work of media historian Brian Winston as a useful way to approach environmental factors in technological development (ibid., 98). Winston’s understanding of technological development is based on the idea that this process cannot be separated from the social and cultural forces that shape it (Winston 1998). I would like to, however, also emphasise

20

the kinect effect

the observation by Martin Lister et al. that when ‘human knowledge and actions shape machines, machines might shape human knowledge and actions’ (Lister et al. 2009, 261). I would, therefore, like to point up the agency of the technology itself – that the physical form, design and capabilities of the technical artefact can shape the ways to which Kinect can be put to use. In other words, I recognise that the device’s “affordances”, a term coined by psychologist J. J. Gibson, affect the range to which the technology can be put to use.

Accounting for the affordances of technologies means that I am aware of the agency of non-human actors in technological development. According to design academic Donald Norman, who further theorised Gibson’s term, ‘[a]ffordances reflect the possible relationships among actors and objects’ (Norman 1999, 42). This key principle of ANT allows me to discuss the reciprocal relationships between artefacts and social groups that are elementary to unravelling a technical artefact’s emergence and development (Mackenzie and Wajcman 1999, 22). In other words, the theoretical principles of ANT allow me to unravel the entanglements between actors grounded in economics, politics, society, and technology, which have influenced the development of Kinect. As media scholar Mirko Schäfer notes, ‘[s]imply using the terminology provided by ANT already allows scholars to […] trace various constituents of emerging media practices. […] Investigating the visual surface [and] agents hidden behind the opaque surfaces are invaluable resources, revealing ideological connotations and the framing of technology’ (Schäfer 2012).11

Heeding the principles of ANT within an evolutionary understanding of technological development, then,

21

introduction

does not necessarily obstruct tracing down ideological connotations shaping the development and use of Kinect. Ideological connotations, which media scholar William Boddy calls ‘instrumental fantasies’ (Boddy 1999), can be located by uncovering how cyclically recurring elements and motives underlying and guiding the development of media culture are “imprinted” on Kinect. According to Erkki Huhtamo, such discursive elements, which he calls “topoi”,12 contribute ‘to [a technical artefact’s] identity in terms of socially and ideologically specific webs of signification’ (Huhtamo 1996). By placing Kinect into its cultural and discursive context the function of discursive regularities in Kinect’s development can be recognised. Uncovering Kinect’s discursive foundation, which tells us how the device “should work”, can subsequently help support my expectation that imaginative uses of Kinect are actively deployed by Microsoft to regulate the device’s future. In sum, uncovering the various actors related to Kinect combined with the excavation of a discursive framework orienting its materialisation and further development contribute to a comprehensive understanding of the emergence of this specific artefact.

StructureThe general structure of this study is divided into three main parts and roughly follows the time path in which Kinect has evolved from Xbox 360 peripheral to hardware for Windows-based computers.13 In chapter 1 I start with pointing out the hopes and dreams surrounding Microsoft’s research division. My body of discourse consists of researchers, developers, executives, and others, mainly associated with Microsoft or the company’s research division, which have

22

the kinect effect

played a large role in Kinect’s development from idea to incubation programme to commercial artefact. I will, subsequently, stop and point out the similarities between the hopes and dreams that underlie Microsoft’s research strategy and utopian discourses attending human-computer interaction (HCI) and virtual reality (VR) to stipulate that these dreams transcend historical context. Finally, I explicate how recurrent hopeful expectations surrounding Microsoft Research relate specifically to Kinect. This chapter excavates the recurrent discursive elements that impel Microsoft’s research efforts and uncovers to what extent these elements are imprinted on Kinect.

In chapter 2, I shift my focus to Microsoft’s incentives for initiating the development of Kinect. This chapter accounts for the instability and unpredictability of technological development by reflecting on the various social, political, economic, and cultural factors, which have played and still play important roles in Kinect’s development. The findings from this chapter are related to the findings from chapter 1 in order to place the role of hopes and dreams in the development of Kinect for Xbox 360 further into perspective. The findings from the first and second chapter, subsequently, allow us to consider the “next stage” in the evolutionary process of Kinect and focus on the emergence of Kinect for Windows and Microsoft’s plans for this product. Chapter 3, then, reflects on the conjunction of discourses and environmental factors affecting the development of Kinect in order to offer an understanding of how Microsoft acts upon how Kinect “could work”. I will present and analyse several examples, including Microsoft’s clip called The Kinect Effect, which support the claim that the company is not only aware of the seductive and regulative

23

introduction

qualities of the imaginary in technological development but also actively deploys these qualities to help constrain the influence of unforeseen environmental factors in the evolution of Kinect.

The goal of this study is to flesh out the constituents of an emerging technical artefact and can be placed in line with seeking answers to questions why we have certain technologies, how they came to existence, and how they have evolved. This study focuses on the causes instead of the symptoms and approaches its object of study from a broad perspective. It accounts for the complex reciprocal relationships between human and non-human, material and non-material actors related to Kinect whilst being keenly aware of the agency of a discursive framework underlying these actors. The development of technology, as professor of religion Lee Worth Bailey notes, is ‘teeming with dreams, visions, hopes, goals, expectations, and imaginative premises’ (Bailey 2005, 17). These dreams exist in the same milieu in which actual technological development takes place and are to be reckoned with when attempting to unravel how a technical artefact is constructed. Recurrent hopes and dreams culminate into rhetoric of progress that can affect technological development and it should receive at least the same amount of attention as the inventors standing at the cradle of breakthrough technologies. On the whole, I intend to provide a nuanced overview of Kinect’s development and contribute to considering technological development more from an evolutionary rather than revolutionary perspective.

24

1. Kinect envisioned: an utterance of an interface sublime

From GUI to NUI

We are at a shift in the paradigm where computers will no longer strictly be used by graphical user interfaces but increasingly will be

used in a more natural way. (Craig Mundie – Chief Research and Strategy Officer at Microsoft – 2011)

According to Sarah Rotman Epps of research agency Forrester we are currently amidst a period of transition in which computing is becoming ‘increasingly ubiquitous, casual, intimate, and physical’ (Epps 2012). The subject of Epps’ investigation is the so-called “post-PC” trend, a buzzword frequently used to denounce the next stage in computing. In her research report, Epps notes that MIT computer scientist David Clark, in 1999, first used the phrase in a talk to describe the inevitable heterogeneous future of computing in which the personal computer will be less important (cited in Lohr 1999). Five years later, in 2004, former CEO of Sun Microsystems Jonathan Schwartz reincarnated the phrase by telling the New York Times that we are in the “post-PC era” for four years, as wireless handsets now largely outsell personal computers (cited in Markoff 2004). More recently, in 2011, Steve Jobs used the term during the release event of the iPad 2 when he announced Apple receives the majority of its revenue from post-PC devices like the iPad, iPhone and iPod (Epps 2012). When Frank Shaw, Corporate Vice President of Corporate Communication at Microsoft, responded to Jobs’ remarks a few months later, stating ‘we

25

thinks it’s far more accurate to say […] the PC isn’t even middle aged yet’ (Shaw 2011), the debate around a new era in computing is (again) vivacious.

The term post-PC is used more than ten years now, yet, it is unclear whether we are still in a process of transition or whether we have already entered this era. An answer to the question whether the PC is dead or whether tablets and smartphones are just additional niches in an ever-changing computer marketplace is in the context of this chapter perhaps not that relevant. What is relevant, though, is that the recurrent use of the term post-PC in public discourse represents a human tendency to believe technologies have the power to change society (Smith and Marx 1994, xiv). The tendency to attribute technologies with transformative powers becomes noticeable when looking at statements from computer scientists. According to Ed Lazowska, Professor of Computer Science and Engineering at the University of Washington, for instance, computer research is ‘the most important field in terms of changing our lives’ (MicrosoftResearch 2011a). A phrase like post-PC gains significance because, as Ray Ozzie, the former Chief Technical Officer and Chief Software Architect at Microsoft, notes, ‘[to] close our eyes and form a realistic picture of what a post-PC world might actually look like, if it were to ever truly occur, [is] precisely what our competitors and our customers will ultimately do’ (Ozzie 2010). Speaking in more general terms about the value of contemplating the future, Ozzie continues by stating that ‘[t]hose who can envision a plausible future that’s brighter than today will earn the opportunity to lead. In our industry, if you can imagine something, you can build it’ (ibid.).

Microsoft, a company that has built its empire around

26

the kinect effect

the PC and has played a defining role in the creation of its ecosystem for more than thirty years, has, from an early stage on, tried to stay ahead of the game through large investments in research and computer science. Microsoft Research, founded by former Microsoft CEO Bill Gates and former Chief Technology Officer Nathan Myhrvold in 1991, is an environment in which, according to Peter Lee, Managing Director of the research annex in Redmond, researchers are ‘living the future’ (MicrosoftResearch 2011b) and receive the opportunity to realise what they dream about (MicrosoftResearch 2011c). At Microsoft Research, according to Anthony Finkelstein, Dean of Engineering Sciences Faculty at University College London, they ‘believe in the transformative, the changing power of technology’ (MicrosoftResearch 2011d). According to Gordon Bell, who was Key Advisor in the formation of Microsoft Research, ‘thinking about computers and where they are going and being part of making them go there […] is exciting’ (MicrosoftResearch 2011b) because, as Chief Research Officer Rick Rashid states, Microsoft Research is ‘the number one basic research organisation in the field of computer science where we are having a tremendous impact with our products’ (ibid.).

The recurrent use of the phrase post-PC and the ways in which researchers at Microsoft talk about the potential impact of technology on society corresponds with what Imar de Vries explains are “necessary fictions”, which ‘tell of purpose, utopian progress, and final resolutions’ (De Vries 2012, 33). The view of technology as a driving force in history, well-established during the Enlightenment, ‘still resonates in present-day conceptions of technology’ (ibid., 44). People involved in computer research dream of a better tomorrow

27

1. kinect envisioned: an utterance of an interface sublime

because they believe they have the ability to create progress in the human condition through technological innovation. According to Ray Ozze, ‘the first step for each of us is to imagine fearlessly; to dream’ (Ozzie 2010). Researchers at Microsoft dream; but, more importantly, they also draw from these dreams in their work. In this context we can investigate what Brian Winston calls the ‘mysterious mental forces’ (Winston 1998, 5) that occur when a ‘technologist envisages the device’ (ibid.). In order to grasp the mental forces underlying Kinect, we should turn our attention to the ways in which Microsoft Research contemplates the role of the user interface in the future of HCI.

One of the characteristics of the post-PC era, as identified by Sarah Rotman Epps of Forrester Research, is that computing shifts from abstracted to physical. This entails that abstracted interaction with content that relies on the mouse and keyboard is shifting towards more direct physical manipulation of content in two-dimensional space through touchscreens on our smartphones and tablets (Epps 2012). In addition to touch, ‘[c]ameras with facial recognition, voice sensors, and motion sensors like those on the Microsoft Kinect for Xbox 360 permit an even wider range of physical interaction with devices’ (ibid.). This trend is what Craig Mundie, the current head of Microsoft’s research division and its 850 computer science researchers, likes to call the transition from the graphical user interface to the natural user interface, or, GUI to NUI. According to Mundie, we now ‘stand at the beginning of a new era in computer interfaces [where] the interface between man and machine becomes more natural […] more like us’ (Mundie 2011).

Mundie, who sets out Microsoft’s overall long-term technology strategy, claims that the progression towards

28

the kinect effect

more natural ways of HCI essentially falls in line with Microsoft’s core philosophy, established when the company started in 1975. According to Mundie, ‘the birth of Microsoft [...] was really around [allowing] people to get the computer to do something that they wanted to do without having to learn to become a computer programmer’ (ibid.). The advancement towards GUIs, over the past decades, has lowered the threshold and learning curve of using computers drastically, yet, now, according to Mundie, we want more natural ways to interact with computers. Mundie explains this shift in the interface paradigm as a response to computing becoming more ubiquitous – all around us – and invisible (MicrosoftResearch 2011e). Research into NUIs can help, as Roy Levin, Managing Director of Microsoft Research in Silicon Valley, puts it, deliver on the dream to make computing ‘invisible to its consumer, to focus on “the what” rather than “the how”’ (MicrosoftResearch 2011f).

NUI is a field of research Microsoft is exploring for many years now and, according to Steve Clayton, editor of the official Microsoft blog, Microsoft’s exploration into this area is “multi-dimensional”; looking beyond sensory related interaction such as touch and voice (Clayton 2011a). This entails that, according to Craig Mundie, Microsoft wishes to incorporate more human senses, such as speech, vision, and hearing, which will allow computers ‘to interact with us at a higher semantic level’ (MicrosoftResearch 2011g). NUIs, then, could eventually help ‘liberate us from technology’ (ibid.), as pointed out by Andrew Blake, Managing Director of Microsoft Research Cambridge. When computers become more natural, more like us, as Mundie puts it, ‘people who have absolutely no concept of dealing with the computer itself are, in essence, interacting with it like it was another

29


person’ (Mundie 2011). Mundie believes that NUIs will allow us to move towards ‘a world where the computers will really integrate, and help us in ways that we don’t quite imagine now’ (ibid.).

An important goal at Microsoft Research is to make technology more human centric so that people can construct a world in which, according to Clayton, ‘[i]nteracting with technology becomes as easy as having a conversation with a friend’ (Clayton 2011a). Making technology more natural is a ‘very big topic’ (Clayton 2011b) at Microsoft that ‘spans product teams and research alike’ (ibid.). According to Clayton, ubiquitous computing, but also cloud computing, social networks, advancements in display technology, large amounts of data, and the Internet of Things14 are all instigators of the trend in which users seek to interact with technology in an easier and more natural way (ibid.). Clayton, here, points at certain technological inventions and consumer demand as instigators for developing more natural ways of computing. Without claiming Clayton is wrong, one could approach the matter from a different angle and try to explain the desire to make HCI more natural from a historical context. Such an angle allows us to critically address the term “natural” and further explicate the origins of this desire.

Virtual reality and the revaluation of human-computer interaction

Designing human-computer experience isn’t about building a better desktop. It’s about creating imaginary worlds that have a special relationship to reality – worlds in which we can extend, amplify, and enrich our own capabilities to think, feel, and act.

(Brenda Laurel 1999, 32-33)

30

the kinect effect

As mentioned above, Microsoft’s core philosophy, ever since the company was founded up to now, is to make the threshold for people to participate in HCI as low as possible, or, as Andrew Blake puts it, ‘we don’t want people’s interaction with technology to be driven by the technology, but driven by the tasks that they are trying to achieve’ (Microsoft Research 2011g). In order to reach this goal, Microsoft invests substantially in research where the aim, according to Andrew Fitzgibbon, a principal researcher at Microsoft Research at Cambridge, is to really look ‘at the future of computer-human interaction, away from just the mouse and keyboard into a domain where anyone could interact with computers at any time’ (MicrosoftResearch 2011i). When we compare Microsoft’s perspective on HCI with discourse surrounding VR similarities can be found. The VR concept, according to researcher Patrice Flichy, is ‘probably one of the most polysemic in the literature on information technology’ (Flichy 2007a, 129) and the many opinions on its definition differ widely. Nevertheless, certain representations of VR relate specifically to HCI or ‘the individual’s relationship with his or her computer’ (ibid.).

Ivan Sutherland, one of the pioneers of research on VR, already looked beyond GUIs in his 1965 article ‘The Ultimate Display’ in which he contemplates new forms of “man-computer dialogue”. In his article – according to Wired author Bruce Sterling ‘a seed-bomb of emergent technologies’ (Sterling 2009) – Sutherland suggests how integrating more of mankind’s senses can enrich HCI.

The computer can easily sense the positions of almost any of our body muscles. So far only the muscles of the hands and arms have been used for computer control. There is no reason

31


why these should be the only ones […]. (Sutherland cited in Sterling 2009)

Sutherland’s forecast of a wider integration of the human senses in HCI, almost five decades ago, corresponds with statements by Microsoft’s Craig Mundie when he addressed a large Cleveland crowd in January of 2011 telling the audience that ‘we want to get computers that can emulate many if not all the human sensory capabilities’ (Mundie 2011). This way, according to Mundie, computing can become ‘more like us’ (ibid.). Moreover, at a time when most computers were still huge and unwieldy machines, Joseph Licklider, who is considered one of the most important figures in computer science, envisioned how ‘human brains and computing machines will be coupled together very tightly’ (Licklider 1960). The aim of such “man-computer symbiosis”, according to Licklider, is to overcome problems like inefficiency when using computers in conventional ways. In order to interact with computers the same way as one would interact and think with a colleague ‘will require much tighter coupling between man and machine’ (ibid.). Licklider, here, expresses a similar hope for making computers more in tandem with mankind.

The desire to progress towards HCI that is more natural, or, more like us, as pronounced by Microsoft representatives, overlaps discourse surrounding VR research because both fields of research strive to reach the same “telos”, or, final goal. The goal of VR, according to VR researcher Scott Fischer, is to match ‘visual display technology as closely as possible to human cognitive and sensory capabilities in order to better represent direct experience’ (Fischer 1991). This objective, to reach complete “immersion”, where all of

32

the kinect effect

the user’s senses are mobilised, has been a major topic in the arts, research, and industry for decades (Flichy 2007a, 134; Fischer 1991). Furthermore, besides involving as many human senses as possible, Microsoft researchers also talk about making technology disappear, as it stands in the way of users and the tasks they want to achieve. We find quite similar reasoning in VR research when we look at Meredith Bricken’s 1991 article ‘Virtual Worlds: No Interface to Design’ in which she talks in almost identical terms about technology fading into the background.

In a virtual world, we are inside an environment of pure information that we can see, hear, and touch. The technology itself is invisible, and carefully adapted to human activity so that we can behave naturally in this artificial world. (Bricken 1991)

More than twenty years before Craig Mundie’s talk in Cleveland – perhaps in more abstract but quite similar terms – Meredith Bricken contemplates a shift in the interface paradigm, which she labels a transition from interface to inclusion (Bricken 1991). According to Bricken, the computer screen – the surface on which the interface is traditionally present – forms a boundary between the information environment and the person trying to access the information. Only when we pass through the barrier of the computer screen we achieve true participation – direct interaction with people and information. Inclusion will characterise the new generation of computing in which we will be able to get “inside” information. In this manner, the computer becomes a far more powerful tool and much easier to use. When inclusion is achieved, Bricken states,

33


‘[y]ou don’t have to know anything about computers, you don’t even have to know how to read’ (ibid.).

The dream of breaking through the barrier of the screen – stepping through the picture into another space – can be traced all the way back to the use of perspective in the construction of paintings and fresco’s in the Renaissance.15 To this day, the thought of stepping through the window – making physical and virtual space merge – remains persistent, even though, as theorist Lev Manovich notes, we still have not left the era of the screen’ (Manovich 1995). Like centuries ago, indeed, we are generally still looking at flat rectangular surfaces such as screens, which are more omnipresent today than ever (ibid.). The concept of VR, or, the “digital virtual” in which a human being is completely submerged in an artificial reality, then, is predominantly a discursive object, or rather, an “object to think with”, which instigates a revaluation of previously fixed notions of concepts of reality, representation, immersion and simulation (Lister et al. 2009, 112).

For early computer researchers after the Second World War, the VR concept offered new ways of thinking about and experimenting with HCI design. According to media researchers Jay David Bolter and Richard Grusin, VR was understood by techno-enthusiasts as ‘the next step in the quest for a transparent medium’ (Bolter and Grusin 2000, 162). When we, as Ivan Sutherland states, should ‘break the glass and go inside the machine’ (cited in Penny 1992), or, as present-day VR theorist and researcher Jaron Lanier puts it, when ‘the technology “goes away” because we are inside it’ (ibid.), we can truly enter the world of the image, information, or content. Mark Weiser, a principal researcher at Xerox PARC16 who coined the term “ubiquitous computing”, opens

34

the kinect effect

his influential article ‘The Computer of the 21st Century’ with the notion that ‘[t]he most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it’ (Weiser 1991, 94).17 At Microsoft Research the desire of rendering technology invisible – pushing the technology into the background – is present as well. This becomes especially evident through the words of Jim Larus, Director of Microsoft’s eXtreme Computing Group (XCG), who states that researchers at Microsoft aim to ‘make computers that just disappear so that consumers don’t have to struggle with them’ (MicrosoftResearch, 2011h).

A major part of the foundation of Microsoft’s quest to reach its interface sublime was established in the 1950s and 60s, soon after the invention of the digital computer, when, according to Bolter and Grusin, computer pioneer Alan Turing and the ‘millennial rhetoric of artificial intelligence […] refashioned the computer from a mere adding machine into a processor of symbols’ (Bolter and Grusin 2000, 161-162).18 Especially in the 1990s, subsequently, VR became the label for the digital revolution and refashioned the computer into a processor of perceptions (ibid.).19 Computer researchers but also science fiction writers and the popular news media draw from the concept of VR, as Brenda Laurel puts it, as ‘tool for thought’ (Laurel 1991, 136) – as a means of ‘incorporating the body and, more broadly, the entire individual in the computing world’ (Flichy 2007a, 139). When looking at the ways in which Microsoft researchers talk about the purpose of their work it becomes clear that they draw from the concept of VR as well.

The number of practical applications for VR remains relatively small, even though concepts and prototypes have

35


been described and developed for many decades (Bolter and Grusin 2000; Fischer 1991).20 Materialisations of the VR concept, however, might not affect computer researchers as much as the discourse surrounding it. According to Lister et al., ‘little could have exercised the minds and imaginations of technologists, journalists […] or academics as much as VR throughout the 1990s’ (Lister et al. 2009, 106). This enthusiasm was in part the result of euphoric techno-utopian expectations in that period, which abruptly ended when the dotcom bubble burst at the end of the 90s. Although public interest in VR has waned over the past decade, traces of the hopes and dreams surrounding VR linger on in the minds of present day computer researchers. Indeed, research into new forms of VR and how the human sensorium responds to full-body immersion has returned, as computers are now many times faster (ibid.).

According to computer pioneer Ted Nelson, the scientific basis, like the foundations of a cathedral, only supports what rises from it. It is the unifying vision that matters more (Nelson 1974). This collective vision, which Patrice Flichy calls “imaginaire”, is one of the basic components of how the usage of a new technology is framed (Flichy 2007b, 125). Microsoft’s collective vision to create natural forms of HCI consists of dreams of reaching full-body immersion, making technology transparent, attaining man-computer symbiosis, seamless HCI, and so forth. These dreams are discursive elements, which, in their configuration and functionality, are quite similar to the “topoi” Imar de Vries finds within the necessary fictions of communication improvement (De Vries 2012, 102-103).21 Despite the fact that, as De Vries points out, ‘these hopes and dreams have been denied by reality, they keep resurfacing and keep setting agendas for further

36

the kinect effect

development’ (ibid., 103). Microsoft’s vision of a NUI future might forever remain an unattainable desire. The hopes and dreams that impel this desire, however, do influence Microsoft’s research efforts and product development. With the latter in mind, we can examine to what extent these discursive elements are “imprinted” on Kinect for Xbox 360.

How dreams are imprinted on Kinect

[Kinect is] the first step in making technology disappear – and the start of a shift that will put our industry on a more human path.

(Alex Kipman – General Manager of Incubation at Microsoft IEB – cited in Microsoft News Center 2012)

The actual work on Kinect – an incubation programme Microsoft code-named “Project Natal” after a city in Brazil (“natal”, appositely, also means “birth”) – took place under the umbrella of Microsoft’s Interactive Entertainment Business division (IEB), responsible for Xbox 360, Zune Marketplace, and Xbox Live. During the programme, the IEB division worked in close collaboration with Microsoft Research.22 We should be aware of the fact that Kinect is actually an assembly of separate technologies (see figure 1), each of which has been under development at Microsoft Research for years. At Microsoft Research Asia, for instance, a lot of research had already been conducted on sensing depth in video and at the Cambridge division they were already working on extracting 3D body positions from video. As Baining Guo, Assistant Managing Director at Microsoft Research Asia, aptly puts it in a video highlighting the research contributions to Kinect, ‘in research you sometimes work on things, which you actually don’t

37


know were other applications’ (MicrosoftResearch 2011i). Microsoft started researching and developing technologies such as voice recognition, identity recognition, and human tracking without knowing in advance that they would end up in a specific technical artefact.

In order to bring Kinect to the market, as Alex Kipman, General Manager of Incubation at IEB puts it, ‘an infinite amount of miracles needed to happen’ (MicrosoftResearch 2011i). One of these “miracles” that needed to happen was the origination of a common vision within the company that would herald Project Natal. This incubation programme for Kinect, subsequently, is responsible for bringing together the various technologies that were under development at Microsoft Research for a long time. The technologies that are now merged within Kinect were originally stand-alone

Figure 1: A still from a video by Microsoft Research depicting the various technical components of Kinect for Xbox 360. The 3D depth sensor consists of an infrared structured laser projector (left) and a monochrome camera (right) (MicrosoftResearch 2011j).

38

the kinect effect

research projects, each of which, as described above, an effort by Microsoft to turn its desire of making natural technologies into scientific fact. In this respect, we could regard these stand-alone research projects as efforts driven by the hopes and dreams underlying Microsoft’s desire to reach a state of computing that is invisible, easy to use and human centric, in short, natural. Can we, therefore, automatically claim that Kinect, as a whole, is a direct utterance of Microsoft’s interface sublime as well?

According to Alex Kipman, ‘Kinect is the first step to transitioning from a world where we have to understand technology to a world where technology understands us’ (Microsoft News Center 2012). Alex Kipman, here, regards Kinect as a “first step” within the company’s path towards more natural forms of HCI. Microsoft’s founding father and former CEO Bill Gates associates Kinect with Microsoft’s NUI strategy, too, when he notes that ‘with Kinect, we are seeing the impact when people can interact with technology in the same ways they interact with each other’ (Gates 2011).23 When we follow Kipman and Gates in assessing Kinect retrospectively we could probably, indeed, regard Kinect an utterance of Microsoft’s desire to create more natural forms of HCI. We should, however, be critical in valuing these statements. In his theoretical explication of how to approach the emergence of a collective vision for the Internet, Patrice Flichy notes the importance of avoiding retrospective constructions. An “a posteriori” justification, according to Flichy, deprives viewing the main lines and intended uses of the technical project (Flichy 2007a, 12).

What we can deduct from this chapter, then, are not so much the actual incentives for the initiation of Project Natal, but rather Kinect’s ideological connotations. Analysing the

39


discourse surrounding Microsoft Research has revealed the ideological web of signification that has contributed to and has become an inherent part of the identity of Kinect. The discourse surrounding Microsoft Research is permeated with terms such as “more natural”, “more human”, and “more transparent”. Microsoft wants to create products that are less complex, easier to use, and more like us, which should allow anyone, at any time, and at any place to wield these products without effort. This desire corresponds with transcendental dreams of making technology disappear, to “step through the window” and be “inside” information. These dreams, which reoccur throughout history, have, for a large part, contributed to the framing of Kinect – they are the building blocks that provide, as Huhtamo notes, ‘“pre-fabricated” moulds for experience’ (Huhtamo 1996). This chapter has unveiled Microsoft’s representation of Kinect rather than showing the actual motives for initiating the Project Natal incubation programme. In order to trace these motives, we should focus on the various actors related to the initiation of Project Natal. The following chapter reveals how and why Kinect for Xbox 360 was developed, which allows us to place the findings from this chapter further into perspective.

40

We have seen, so far, how Microsoft’s desire to create more natural forms of HCI is largely driven by recurrent hopes and dreams about HCI and VR. The ideological connotations uncovered above, however, are not the primary causes for the actual initiation of the development of Kinect for Xbox 360. As we will see in this chapter, Microsoft’s hopes and dreams are important constituents but should not be regarded as determinants in the emergence of Kinect. We need to open the black box further – look at elements other than hopes and dreams to cover all aspects of the development of Kinect. In this chapter I focus on the various actors, such as Microsoft’s competition in the gaming market, that have contributed in significant ways to the initiation of Project Natal. The findings of this chapter can help explain why Kinect was developed and help to further interpret and value the findings from chapter 1.

Nintendo and the revival of gestural gaming

In a way, you could see the [Nintendo] Wii as [Kinect’s] biggest competitor because they started with motion technology. (Thijs

van Doorn – Marketing Manager, Xbox and Xbox Live at Microsoft Netherlands – 2012)

When Kinect was officially announced at the Electronic Entertainment Expo (E3) in 2009, Japanese game company Nintendo had already set the bar for gestural gaming a few years earlier with its game console called Wii. Launched

2. Kinect enacted: exploring Microsoft’s development incentives

41

in 2006, Nintendo’s Wii was set to compete with Sony’s PlayStation 3, also released in 2006, and Microsoft’s Xbox 360, which was released in 2005. Unlike Sony’s and Microsoft’s high-end game consoles, which bet on high-definition graphics and better performances for the hard-core gamer, Nintendo chose to emphasise its new controller system in order to attract the more casual gamer as well (Acohido 2006). According to Shigeru Miyamoto, a member of Wii’s development team, when Nintendo began working on the console in 2001 ‘we wanted to come up with a unique game interface [because] power isn’t everything for a console [and] too many powerful consoles can’t coexist’ (cited in Hall 2006). Miyamoto explains how the development team decided on the basic technology in late 2004, early 2005. By then, they had come up with Wii’s remote controller and decided on the motion sensor, infrared pointer, and the layout for the buttons (ibid.). According to Miyamoto, an important reason for Nintendo to change the classic game controller, which had not changed radically in more than two decades, was because ‘for a long time, we thought that changing the interface would broaden game design and loosen creative constraints on programmers. We found that to be true when we released the DS’ (ibid.).24

Nintendo’s decision to choose innovation over going head-to-head with its competitors in terms of technological muscle proved to be a resounding success. In July of 2008, the NPD Group, an American market research company, published US sales figures, which showed that Wii had surpassed Xbox 360 in less than two years after Microsoft got a one-year head start (Kim 2008). Nintendo showed its competitors how to successfully diversify a video game audience that had been demographically standardised over

42

the kinect effect

the years. As Reginald Fils-Aime, President and COO of Nintendo America, puts it, ‘our goal is to […] expand this business here in a way that it really hasn’t been expanded’ (Acohido 2006). Nintendo’s successful formula obviously did not slip by Sony and Microsoft unnoticed and, afraid of losing the battle for the emerging market of the casual gamer, both companies jumped on the gestural gaming bandwagon. At the E3 of 2009, Microsoft officially announced Project Natal whilst Sony introduced PlayStation Move, a wand-shaped controller for motion-controlled games.25

According to Shane Kim, at the time of Project Natal’s announcement Corporate Vice President for Strategy and Business Development at Microsoft IEB, ‘[Project Natal] has nothing to do with Sony or Nintendo [and] everything to do with breaking down barriers and getting to the mass market’ (cited in Takahashi 2009a). Without Wii, however, there probably would not have been Kinect. As David Pogue of the New York Times puts it, ‘Microsoft owes a huge debt to the Nintendo Wii’ (Pogue 2010). Pogue here refers to the fact that, for the first time, a company was able to successfully bring gestural gaming to a mass audience. As early as in 1989, game company Mattel already released the Power Glove for the Nintendo Entertainment System. The glove enabled users to control on-screen items with a wave or twist of hand, but failed to become a widespread success, partly due to a lack of precision (Herold 2012). More recently and before the release of its wand-shaped Move, Sony also ventured into gestural gaming with the EyeToy camera for the PlayStation 2. Similar to Kinect, the peripheral did not request the need for a physical controller, as it could track the body of a player. Unfortunately for Sony, a lack of tracking precision, which, according to Charles Herold,

43

2. kinect enacted: exploring microsoft’s development incentives

‘happened more often than not’ (ibid.) also meant EyeToy’s downfall. The Wii certainly is not the only notable attempt at advocating different ways of controlling video games but, unlike its predecessors, this device became a major hit.

What, then, caused Wii to become a breakthrough success for gestural gaming? Brandon Sheffield of game journal Gamasutra claims that the success was a consequence of Nintendo’s effective and thought-through marketing efforts. According to Sheffield, Nintendo was telling moms, grandmothers, and grandfathers that this device was fun for everyone, proving that ‘Nintendo knew the market it was going for and targeted it perfectly’ (Sheffield 2010).26 Whether it was through marketing or through actual performance, all the same, Nintendo managed to popularise gestural gaming and paved the way for similar and derivative products such as Kinect and Move to emerge. Not only did Nintendo show Microsoft that gestural gaming was the key to reach a mass market, or, as game industry analyst John Taylor puts it, to take ‘the 360 out of a teenage boy’s bedroom and putting it in the living room where everybody can use it and enjoy it’ (cited in Peterson 2011), in a broader sense, and perhaps more significantly, Nintendo showed Microsoft that the game industry was an ideal environment to profit from motion and voice sensing technology – technologies Microsoft has been working on for many years.

The decay of a software giant

[Microsoft’s] failures make up quite a flop parade. (David Pogue – New York Times tech correspondent – 2010)

With the Wii, Nintendo was able to outpace and outwit

44

the kinect effect

Microsoft in moulding together complex technologies, such as voice recognition and motion tracking, into a successful consumer product. This may seem remarkable given the fact that Microsoft spends nine billion US dollars a year, larger than the seven billion dollar budget of the US National Science Foundation, on research and development (Tu 2012). It may seem even more remarkable given the fact that voice recognition, a technology now welded into Kinect, is a part of Microsoft research efforts for a long time. According to Michael Cherry of independent research institution Directions, ‘Microsoft has been doing voice recognition for a long time […] [b]ut if people were to say to me, computers using speech, [Apple’s] Siri would be the example today’ (cited in Tu 2012). Microsoft’s inability to capitalise from long-term research investments becomes even clearer when looking at touch technology. According to Cherry, Microsoft incorporated touch-screen technology in Windows way before Apple came up with the iPad but when people think of touch these days they do not think of Windows 7 but of the iPad and iPhone instead (ibid.).

Microsoft’s inability to capitalise from voice and touch technology is illustrative for a software company that encounters difficulties attempting to create successful hardware products. As tech-journalist David Pogue aptly puts it, ‘[w]ith the money Microsoft has spent on failed efforts to design hardware, you could finance a trip to Mars’ (Pogue 2010). Microsoft’s efforts on the mobile phone market are probably best illustrative for this symptom. Over the course of a few years, Windows-based mobile phones have been almost unattainably surpassed by Apple’s iPhone and Samsung smartphones running Google’s operating system Android. In 2010, research firm Canalys already

45


predicted ‘Microsoft will have a difficult year’ (Canalys 2010) with market share for the Windows Phone platform shrinking 1.3 per cent in contrast to 37.9 per cent overall growth of the smartphone market (ibid.). Microsoft had no conclusive answer ready for the dark forecasts and responded convulsively with the 2010 release of the KIN, a phone that integrated social networking and, according to Robbie Bach, former President of the Entertainment and Devices Division at Microsoft, would bring a mobile experience just for the social generation (Bach 2010). The KIN, however, is, as Simon Hackett of Business Spectator puts it, ‘a single application device in a world where it’s now all about dynamic application ecosystems’ (Hackett 2010). The KIN was cancelled only six weeks after it was announced and was soon thereafter labelled ‘Microsoft’s worst failure’ (James 2010).

The KIN is but one out of an impressive number of failed attempts by Microsoft at creating valuable consumer hardware products. In 1997, Microsoft paid 425 million dollars for the acquisition of interactive TV company WebTV (Lazarus 1997), renamed it MSN TV, and eventually spun it off as a separate company (Kaplan and Segan 2008). Despite all of Microsoft’s marketing efforts, the device never gained much popularity (D’Amico 2009). Microsoft’s weakness in hardware design also becomes evident in the case of its portable media player Zune, developed to compete directly with Apple’s popular iPod line. A huge embarrassment related to Zune occurred at New Year’s Eve of 2008, when a technical glitch caused some versions of the device to stop working (Pham 2009). After the release of the latest standalone player in 2009, the Zune HD, CNET reported how the Zune hardware ‘fades into the background’

46

the kinect effect

(Bell 2010). Although the Zune software service still exists, hardware upgrades have not been released since 2009.

According to David Pogue, ‘[Microsoft’s] failures make up quite a flop parade. […] [I]f this were ancient Greece, you’d wonder what Microsoft had done to annoy the gods’ (Pogue 2010). The fact that Microsoft more often than not fails at producing successful hardware products may, however, not be so easily attributed to sheer bad luck. According to Simon Hackett, because ‘Microsoft never did integrate itself with hardware manufacture’ (Hackett 2010) it has grown a structural problem. Apple, on the other hand, has created an integrated line of hardware and software into a consistent form of user experience and this ecosystem, according to Hackett, is ‘seamlessly tied together’ (ibid.). Unlike Apple and Google, Microsoft has missed the boat in terms of becoming a vertically integrated computer experience provider and should therefore return to its roots, which lie at making successful software applications such as Windows (ibid.).

The KIN and Zune debacles made Microsoft decide to lie low in the hardware production department and partner with third-party companies for making hardware instead. Perhaps the most notable example of this change of tactics is the deal with Finnish mobile phone company Nokia.27 The partnership resulted in a line of Nokia Lumia smartphones running the latest version of Windows Phone OS. Yet, one year later, despite desperate efforts to get software developers on board, Windows Phone has just a 3.9 per cent share of the smartphone market in the US, compared to 51.1 for Google Android and 30.2 for Apple iOS (Ovide and Sherr 2012). Sales of the Windows-based Nokia Lumia have not taken off to the extent in needs to break the hegemony

47


of Google and Apple in the retail environment (Ricknás 2012), even though Windows Phone receives critical acclaim (Molen 2011) and even praise from Apple co-founder Steve Wozniak (Smith 2012).

Microsoft’s difficulties in the mobile phone market exemplify that its problems are not confined to the occasional hardware flop. In 2005, Microsoft was the number one most valuable brand in Millward Brown’s annual list of the top 100 most valuable global brands.28 In its heyday, Microsoft’s closest competitors were non-tech companies such as General Electric and Coca-Cola. Since then, the once all-mighty Microsoft has plummeted down the list (Bekker 2011).29 According to tech-journalist Mike Elgan of Computerworld, the downfall of Microsoft is a consequence of a world that has changed; a world in which its business model – partnering with hardware companies to manufacture systems for your operating system – does not work anymore (Elgan 2012). According to Elgan, Microsoft should adopt the Google model: a hybrid model in which you ‘[make] hardware but also licence your OS to hardware partners who make products of their own’ (ibid.).30 31 The latest developments indicate Microsoft, indeed, alludes to Google’s hybrid model. On June 18, 2012, Microsoft announced its own tablet device named Surface (Nguyen 2012). Despite of Surface’s technical ingenuity and slick looks, the timing of Surface’s announcement is awfully late and again illustrates how Microsoft has lost its leading role. Instead of setting the trend and leading by example, the software-giant now makes every effort to catch up with the competition.32 There is, however, a big exception to Microsoft’s overall impairment: the Xbox 360.

48

the kinect effect

The struggle upwards (From GUI to NUI cont’d)

[Y]ou’ll experience more and more natural user interface through Metro. (Steve Ballmer – Microsoft CEO - 2012)

The dark clouds above Microsoft’s head are far from gone but, in the meantime, the company can hold on to the ray of hope that is Xbox 360. At the 2012 edition of the Consumer Electronics Show (CES) in Las Vegas, Microsoft revealed that there now have been 66 million Xbox 360 systems sold (Pereira 2012). These sales figures came just two months after it became known that Microsoft sold over 960 thousand consoles in the US during the week of “Black Friday”, the day after Thanksgiving when, traditionally, Christmas shopping begins. This figure was good enough for Microsoft to call it the biggest sales week in the history of Xbox (Pereira 2011b). The same goes for Kinect, which has been a huge success from the start. With 18 million Kinect units sold since launch it now holds the Guinness World Record for fastest selling consumer electronics device (Pereira 2012). As described above, however, sales figures for the Xbox 360 and Kinect would probably not have been this high if Microsoft did not have Nintendo to lead by example. In 2007, one year after the release of Nintendo’s Wii in the US, two years before Kinect was first announced, Bloomberg’s Dina Bass picked up the first rumours about Microsoft hinting at price cuts and ‘emulating rival Nintendo Co.’ (Bass 2007).

Peter Moore, former Vice President of Microsoft IEB, which oversees Xbox 360, is quoted in the Bloomberg article stating ‘if we don’t make that move, make it early and expand our demographic, we will wind up in the same place as with Xbox 1, a solid business with 25 million people

49


[whilst] I need […] 90 million people [for the Xbox 360]’ (cited in Bass 2007). At the time Moore was quoted, Wii was rapidly outselling Xbox 360 in the US by appealing to women, children, and the elderly. Microsoft had a dire need to adopt this strategy to prevent Xbox 360 from decisively falling behind the Wii. Predictions for Microsoft were poor, as research firm IDC estimated Nintendo would sell almost 16.1 million consoles in 2007, compared to 9.69 million for Xbox 360. Peter Moore ultimately admitted that he and his team did not foresee that Nintendo, not Sony, would be the biggest threat to Xbox 360. As soon as Moore saw Satoru Iwata, Nintendo’s President, swing the Wii controller at a conference he knew his rival had a winner (ibid.).33

Afraid of losing the momentum of one of its most successful hardware products, Microsoft struck back by doing the same thing as it has now done in the tablet market. It charged the competition head-on by developing its own piece of hardware. As Craig Mundie puts it, ‘we threw everything we had at that problem’ (Tu 2012). The Microsoft Research division worked directly with the Xbox business team for the first time on a mass-market product. The research division provided technologies such as voice- and face-recognition but Microsoft bought and teamed-up with many vendors of low-cost but sophisticated technology as well. It bought the Israel-based companies PrimeSense and 3DV Systems in 2009 for obtaining a low-cost 3D motion-detection camera and a strong patent portfolio concerning various motion-detection systems (Takahashi 2009b; Takahashi 2009c). Microsoft also teamed-up with US-based interactive technology company GestureTek for developing a software layer that would help interpret the data coming from Kinect’s cameras (Takahashi 2009c). The first

50

the kinect effect

prototype of Kinect took four months and approximately 30 thousand dollars to build (Tu 2012).

Kinect brought the boost for Xbox 360 Microsoft had hoped for and the device now addresses those market segments the Wii had to itself (Peterson 2011). But Kinect brought Microsoft more than larger market shares and raving sales figures. According to Mike Isaac of Wired, Kinect has also brought Microsoft ‘back into the innovative limelight, a space it long since occupied’ (Isaac 2012). In his review of Kinect, David Pogue describes how it provides ‘an experience you’ve never had before’ (Pogue 2010) and Tim Carmody of Wired describes Kinect as ‘something different […] communal, continuous, and general: a [NUI] for multimedia, rather than a GUI for gaming’ (Carmody 2010b). Here, Carmody highlights probably Kinect’s biggest achievement. Kinect has pushed NUI, something Microsoft has been talking about for years, beyond a research and development driven “future-of-computer” hobby to the company’s first widespread commercial success (Carmody 2012). The success of Kinect validates the premise that there is a future for more powerful and natural ways of interacting with computers than through GUIs or even through voice and touch.34 Microsoft’s strategic thinker Craig Mundie has played an important role in the emergence of Kinect and the company’s gradual transition from GUI-focused to NUI-oriented. Concerning the former, Don Mattrick, the current President of IEB, was struck by Mundie’s willingness to blend the business and research sides of Microsoft in the development of Kinect – ‘an example of where one plus one equals three’ (Mattrick cited in Tu 2012). Concerning the latter, Mundie sees himself as the biggest, steadiest champion inside the company for that transition (Tu 2012).

51


Notwithstanding Kinect’s success, the road towards a future without buttons is still long and first needs to be paved. A lot depends on whether Microsoft is able to catch up on Apple and Google with voice and touch. At this point, Microsoft’s top priority is its latest operating system Windows 8, which will be released in the fall of 2012. Windows 8 will feature an entirely new user interface called Metro, which is optimised for voice and touch and specifically developed for cross-platform implementation, which should lead to a convergence of its now separate platforms (e.g. Xbox, phones, and PCs). According to Wired, Windows 8 and its block-based Metro interface, ‘is a radical departure, not just for Microsoft, but for everything we know about so-called desktop computing’ (Bonnington 2012a). So far, the signs for the new Windows OS are good. According to website Techcrunch, the Metro interface, which is the most significant change in Windows 8, will herald ‘Microsoft’s rebirth’ (Biggs 2012). According to The Guardian, Microsoft has done a great job with Windows 8 in lining up the whole platform to survive a shift to a post-PC era (Baxter-Reynolds 2012).

In January of 2012, Microsoft held its last of fifteen annual keynote presentations at CES. Whilst other big-name companies such as Apple over the years scaled back their use of trade shows for major product announcement, Microsoft stayed put. According to Wired, Microsoft’s continued participation at CES was somewhat analogous to the company’s position in the market: ‘Out of touch. Antiquated. Lacking innovation’ (Isaac 2012). The departure from CES is probably symbolic for the “new” Microsoft – the company that now makes every effort to change. The company’s transition is exemplified by its new user interface

52

the kinect effect

called Metro, which, according to Steve Ballmer, ‘will drive the new magic across all of our user experiences’ (Ballmer 2012). Where Metro provides the current conditions for the company’s change, Kinect offers a peek into its future. At CES, Ballmer officially announced Kinect for Windows, a PC-optimised hard- and software kit designed to integrate Microsoft’s products at a higher level than Metro (Isaac 2012). Soon after Kinect for Windows launched on February 1, 2012, rumours emerged about Microsoft integrating Kinect technology with the new Windows Phone (Valich 2012) and The Daily reported on laptop prototypes incorporating Kinect sensors (Hickey 2012). Microsoft is lining up its current and upcoming products for a post-PC, or, NUI-oriented era and Kinect could well play an important role in this process.35

Although Microsoft now alludes to positioning Kinect as a key shackle in its near future, we can defuse the claim that the company knew of this scenario on forehand. On the basis of this chapter, we know that actors such as Nintendo’s strategy for the Wii, a fear of losing market share for the Xbox 360, Microsoft’s political-economic situation and loss of face are primary incentives related to the initiation of Project Natal. Each of these actors is at odds with Microsoft’s representation of Kinect as a well-planned first step in the company’s gradual path towards a NUI-oriented era of computing. The findings of this chapter, however, do not detract from the agency of Microsoft’s ideological rationale in the development of Kinect. Although a pre-established ideology has not been a primary motivation for the initiation of Project Natal, Microsoft’s ideological connotations have contributed significantly in creating and shaping the ways in which Kinect is generally perceived.

53


Approaching Kinect for Xbox 360 from a broad perspective reveals the heterogeneous character of the development of the device. The discursive dimension of this development process, revealed in chapter 1, cannot be seen separate from the non-discursive dimension of this process revealed in this chapter. Social and technical elements are woven together by continuous action, all the way down to a micro level, making it problematic to analytically separate these two dimensions (Latour 1999). As media researchers Bernhard Rieder and Mirko Schäfer note, ‘[m]eaning is deeply embedded in the non-discursive […]. Technology is not only surrounded by discourse, it is discourse’ (Rieder and Schäfer 2008, 161). The development of Kinect is not a process that begins and ends with Microsoft Research but rather a continuous process that takes place in heterogeneous and contradictory environments.

According to Rieder and Schäfer, the way we create technical artefacts influences the cultural role they will play – they integrate and propagate human values (ibid.). I believe that Microsoft is well aware of this mechanism and I expect that the company’s representation of how Kinect should work plays a crucial role in the further development of the device and the emergence of Kinect for Windows. The latter will become clear in the following chapter, which focuses on the “next phase” in Kinect’s evolution and the ways in which social and technical forces related to Kinect are closely entwined.

54

So far, we have fleshed out how the complex relations between multiple actors – a combination of human and non-human aspects – have set conditions for how Kinect “should work”. With the recent release of Kinect for Windows, Microsoft shows us ways of how Kinect “could (also) work”. Based on the success of Kinect for Xbox 360 and given the fact that Microsoft aims to achieve deep integration of its separate product lines, it might seem obvious that Microsoft decided to push Kinect beyond gaming and make it a more integral part of the company. The porting of Kinect to Windows-based machines, however, is something Wired calls a surprise and even ‘a remarkable turnaround’ (Carmody 2012). This chapter accounts for the various actors in Kinect’s evolution from a gaming peripheral to PC hardware, demonstrating the erratic and unpredictable character of technological development; a process made even more complex by the blurring of boundaries between producers and consumers and increasing entwining of social and technical elements. We will see in this chapter, moreover, how Microsoft cautiously attempts to gain control over the evolution of Kinect by toning down the unpredictability of this process and leaving as little as possible to chance.

A hacker’s delight

We’ll see a million ideas flourish once anyone can plug [Kinect] in to whatever they want. (Phillip Torrone – Creative Director at

Adafruit Industries – cited in Terdiman 2010a)

3. Kinect evolved: implementing hacks and building on dreams

55

On November 4, 2010, moments after Kinect went on sale in North America, New York-based Adafruit Industries kicked off the Open Kinect contest by offering a one thousand US dollar bounty for the first person that managed to build an open-source driver for Kinect (Terdiman 2010a). The main motive for Adafruit to encourage people to decouple Kinect from the Xbox 360 can be found in Phillip Torrone’s convincement that ‘[Kinect] is amazing hardware that shouldn’t just be locked up. […] It’s “radar camera” being able to get video and distance as a sensor input from commodity hardware is huge’ (cited in Terdiman 2010a). Immediately after Adafruit announced the bounty offer, tech-media website CNET asked Microsoft for a response. A company spokesperson replied that ‘Microsoft does not condone the modification of its products [and] will work closely with law enforcement and product safety groups to keep Kinect tamper-resistant’ (ibid.). This hard-line response prompted Adafruit to increase its bounty to three thousand dollars and add a donation of two thousand dollars to the Electronic Frontier Foundation (EFF),36 just in case, as Wired’s Tim Carmody puts it, ‘Microsoft decided to start suing the pants off of everybody’ (Carmody 2010a).

Encouraged either by the prize money or the fame (or both), hackers from across the globe started working on opening up Kinect to other platforms. On November 8, four days after Kinect’s launch, Alex P, someone who previously hacked the PlayStation 3 EyeToy camera, posted a message in the forum of NUI Group37 in which he showed he was able to access data from the Kinect through his PC (Carmody 2010c). Reportedly, Alex P was not interested in the prize money, which led to Héctor Martin, another contest competitor, claiming the prize money (Hollister

56

the kinect effect

2010; Carmody 2010d; Paul 2010).38 Within just a day, consequently, another code enthusiast used Martin’s drivers to program Kinect for manipulating photos whilst someone else was able to open up Kinect to Apple’s operating system OSX (ibid.). The Open Kinect contest and the release of the open-source drivers sparked many hobbyist initiatives almost simultaneously and led to further development of the drivers at an astonishingly rapid pace.

Interestingly enough, Microsoft did not respond to the hacks by invoking law enforcement. Microsoft, in fact, did not respond at all until Alex Kipman and his colleague Shannon Loftis went on the air in NPR’s radio show Science Friday.39 When a listener asked on Twitter about Adafruit’s Open Kinect contest, Loftis replied how she was “very excited” and “inspired” to see how people were finding new uses for Kinect within less than a week. When the host of the show asked if anyone would get in trouble, both Loftis and Kipman replied with a decisive “no” (Terdiman 2010b; Carmody 2010a).40 Kipman claims that hacking would mean someone got to the Kinect’s algorithms or that someone used a device for means of cheating (Carmody 2010a). Both have not happened, according to Kipman, because ‘we’ve put a ton of work and effort to make sure [that] doesn’t actually occur’ (cited in Carmody 2010a). Loftis’ and Kipman’s remarks led Adafruit to write a blog post congratulating its community for turning ‘“work closely with law enforcement” to “inspired” by […] finding new uses for Kinect’ (Adafruit 2010).41

It was not until three months later, in February of 2011, when Microsoft officially announced an official non-commercial software development kit (SDK) beta version for Kinect, that it became known that John Chung Lee, at

57

3. kinect evolved: implementing hacks and building on dreams

the time a researcher at Microsoft Applied Sciences and a core contributor to Project Natal, had secretly approached Adafruit to put on the Open Kinect contest (Mosher 2011). Lee, who became well known for hacking the Wii remote controller, revealed his secret in a blog post, stating that ‘[i]t’s unfortunate [the Kinect SDK beta] couldn’t have happened closer to launch day. […] When my initial efforts for a driver stalled, I decided to approach AdaFruit [sic] to put on the Open Kinect contest’ (Lee 2011). For reasons unknown, Microsoft decided not to pursue Lee’s proposal for a SDK at the time of Kinect’s launch but opened up to his idea once it saw how Kinect was being appropriated.

The diligence of the open-source community turned out to create what Lee had anticipated. Numerous hobbyist enthusiasts and grass roots initiatives applied Kinect to an astounding variety of applications ranging from controlling flying robots to using Kinect for generating sound through your body (Chen 2011). Lee, who by then had already went on to work for Google, notes how the wide variety of Kinect hacks ‘is showing us the future. This is happening today, and this is happening tomorrow’ (cited in Mosher 2011). Microsoft was left with no other option but to admit Lee had it right from the beginning. In an official Microsoft blog post on February 21, 2011, a few months after the Open Kinect contest started, Steve Clayton writes how ‘[t]he community that has blossomed since the launch of Kinect for Xbox 360 […] shows the breadth of invention and depth of imagination possible when people have access to ground-breaking technology’ (Clayton 2011c). According to Clayton, it is for this reason Microsoft will release a non-commercial Kinect for Windows SDK later in 2011 ‘[to] ignite further creativity in an already vibrant ecosystem

58

the kinect effect

of enthusiasts’ (ibid.).42 As sincere as this thought behind the SDK might sound, though, we here see the first sign of Microsoft attempting to gain control over Kinect’s evolution by consolidating the efforts of the open-source community.

Gaining control of evolution

We can’t wait to see what’s next when […] Kinect for Windows will be available for commercial use. Then, we’ll see even more new

ideas which will continue to inspire us, and others, to keep driving the innovation forward. (Craig Eisler – General Manager, Kinect

for Windows – 2011b)

By the time the non-commercial SDK for Kinect was released in June of 2011, Adafruit’s Phillip Torrone wrote an article on MAKE magazine’s website in which he criticises the restrictive nature of the SDK’s license. After examining the license, Torrone concludes ‘[y]ou can’t start a business, make money, sell services or consulting, it’s all non-commercial and in my opinion, a dead-end for most/all developers’ (Torrone 2011). This is an issue John Chung Lee also raised moments after Microsoft announced the SDK in February. In his blog post, Lee applauds Microsoft’s decision to approach the independent developer community but, at the same time, questions whether Microsoft can give developers a good reason to leave their projects behind and make them use Microsoft’s own development platform instead (Lee 2011). For that same reason Torrone wonders out loud why anyone would want to use the non-commercial SDK when you can build a business and company around the open-source drivers (Torrone 2011). Wired’s Mike Isaac also questions whether Microsoft is able to build its own

59


developer base for Kinect when there is little financial incentive for them to join (Isaac 2011).

Perhaps it is for that reason approximately four months later, in November of 2011, one year after Kinect for Xbox 360 was released, Microsoft officially announced Kinect for Windows, a tailor-made product to fit commercial endeavours. This commercial variant includes a Kinect device tweaked specifically for non-gaming usage. It offers something called “Near Mode” for close-proximity gesture tracking and has a modified USB connector and better protection against noise and interference, all designed to better incorporate the Kinect hardware within the PC environment (Carmody 2012). When Kinect for Windows was announced, Microsoft already had tied more than two hundred companies to a pilot programme for commercial entities, including companies such as Toyota and Nissan (Tu 2011a). By the time Kinect for Windows was released in the beginning of February 2012, Craig Eisler, General Manager of Kinect for Windows, told BusinessWeek almost 350 companies, including Boeing, are working with Microsoft for custom Kinect applications (Bass 2012). These undertakings led Forrester Research to conclude that Kinect for Windows is ‘a turnaround chance for Microsoft. […] [I]t’s about the future of everything’ (McQuivey cited in Bass 2012).

In the time frame of one year, Microsoft completely turned on the subject of an open-model Kinect programme. The company switched from threatening to prosecute anyone who would attempt to open up Kinect to platforms other than the Xbox to fully embracing the potential benefits of commercial Kinect applications outside the gaming sphere. Probably startled by the idea it would miss out on

60

the kinect effect

commercially interesting appropriations for Kinect in other fields than gaming, Microsoft spared neither cost nor effort to intervene in Kinect’s further development. The company, for instance, organised an event called Code Camp at its headquarters in Redmond, just after the SDK beta release in June of 2011, to which it invited a group of developers to see what kinds of applications they could come up with (Isaac 2011). When Microsoft announced Kinect for Windows in November of 2011, the company also set up an accelerator incubation programme together with company TechStars43 to challenge start-up businesses to invent commercial applications for Kinect in exchange for a twenty thousand US dollar investment (Tu 2011b).

Another indication Microsoft made profound efforts to gain control of Kinect’s evolution can be found in the fact that Microsoft decided to abandon the core principles of the business model that helped the company become the colossus it is today. Microsoft, a company that has grown a giant selling and licensing software, basically decided to give away its software for free and sell its hardware instead – something Tim Carmody describes as ‘very un-Microsoft’ (Carmody 2012).44 In Carmody’s report of Microsoft’s keynote at CES 2012, Phillip Torrone argues that the general press will not fully comprehend the meaning of the biggest software company in the world just saying ‘we’re giving away the software and selling the hardware’ (cited in Carmody 2012). According to Torrone, Microsoft was very smart to adopt the Open Kinect model and turn it into a business of its own. Kinect got away from Microsoft for a moment when hackers, gamers, designers, artists, doctors and scientists opened up the device’s possibilities but the company adapted to it, ratified it, took a leadership position

61


and claimed it as its own – ‘they had to’ (Torrone cited in Carmody 2012).

In the light of recent history, indeed, it is noteworthy Microsoft has chosen to appropriate and not cross the efforts of the open-source community and, moreover, to adapt these efforts through specific marketing strategies in order to commercialise a renewed version of the original product that was hacked. For many years, especially during the turn of the century, Microsoft has continuously sought to counteract new media practices and initiatives that might form a threat to its business model in order to protect and preserve its leading role in the market.45 Microsoft, for instance, has attacked open-source initiative GNU/Linux46 for years, as it formed a direct threat to its established business model (Van den Boomen and Schäfer 2005). The strategy to frustrate the appropriation of established conventions of production is labelled by Mirko Schäfer as “confrontation” (Schäfer 2010). The strategy Microsoft used in the case of Kinect, however, can be referred to as what Schäfer calls “implementation”.

Unlike confrontation, implementation is less obvious and attracts less attention. It is a subtle and often neglected process that takes advantage of certain user activities. Primarily taking place at the level of design, implementation channels user activities to create new business opportunities. (Schäfer 2010, 126)

Implementation is a strategy not unfamiliar to Microsoft. When the company started the Shared Source Initiative47 in 2001 it made an effort to create the impression of being “open” itself by giving restricted access to some of its source

62

the kinect effect

codes (Van den Boomen and Schäfer 2005). Microsoft believed that by alluding to the concepts of the open-source movement the competing movement might eventually become insignificant or even disappear (Kuehnel 2007). The Xbox 360, furthermore, can be regarded the result of an implementation strategy. According to Mirko Schäfer, Microsoft translated and implemented many of the design suggestions of do-it-yourself hobbyists into the GUI and design of the Xbox 360 and, moreover, provided an integrated SDK to regulate the practice of homebrew software (Schäfer 2012, 146). Microsoft’s undertakings related to Kinect show clear parallels with Schäfer’s analysis of Xbox 360. In the case of Kinect, Microsoft, too, decided to release multiple SDK versions to regulate the proliferation of homebrew applications. The company did not fight Adafruit and the Open Kinect contest but chose to implement user activity in the hard- and software design of Kinect for Windows.

Reverse-engineering Kinect for Xbox 360 has led to countless innovative appropriations, which have affected the development of the device as a commercial product. Despite recommendations from John Chung Lee to utilise the inventiveness of the open-source community from the beginning, Microsoft instead chose to black-box a heterogeneous technology – a device that essentially is an assemblage of multiple distinct technical features (e.g. motion tracking and face recognition). Regarding Kinect for Xbox 360, it seems that Microsoft, intentionally or unintentionally, did little proving it learned from its experiences with the first Xbox, which was marketed as a video game console but built on common PC architecture deliberately limited to function as such. This made the first Xbox a prime target for hackers to bypass its built-in

63


restrictions and open up its PC functionalities (Flynn 2003). In a similar way, the hacker community could hardly wait to open up the black-boxed technical capabilities of Kinect. A continuous process of “bricolage” – a process of trying-out and experimentation (Ciborra 2002) – has affected the development of Kinect profoundly, which shows that media practice and material affordance are mutually dependent (Schäfer 2010).

The motive behind the hacking of Kinect for Xbox 360, encouraged by Adafruit Industries, reveals an intrinsically social factor William Boddy calls “instrumental fantasies”. These fantasies are carried with the launch of every electronic media product and refer to an ideological rationale, or, an implicit fantasy scenario of a technology’s domestic consumption (Boddy 1999). Traces of how Adafruit and the open-source community perceive Kinect are found in the words of Phillip Torrone when he claims that ‘we’ll see a million ideas flourish once anyone can plug [Kinect] in to whatever they want’ (Torrone cited in Terdiman 2010a). This chapter, so far, shows that there exists a discrepancy between Adafruit’s representation of Kinect and Microsoft’s idea of how Kinect “should work”. The collision between both fantasy scenarios has affected Kinect for Xbox 360’s evolution profoundly and can be regarded as a primary cause for the emergence of Kinect for Windows.

Kinect for Windows, however, is neither solely the outcome of social factors nor was the hacking of Kinect for Xbox 360 solely provoked by its technical affordances. The theoretical principles of ANT allow us to sidestep such engrained conceptual distinctions and help us to acknowledge the heterogeneous character of technological development. The development of Kinect for Xbox 360

64

the kinect effect

and the emergence of Kinect for Windows both consist of a process of various social groups and technical aspects affecting each other and trying to assign various roles to each other. This understanding of technological development allows us to further investigate how attempts by Microsoft to employ its agency to regulate Kinect’s evolution reach beyond the implementation of user activity on the level of design. In other words, I expect Microsoft to be well aware and make good use of what Imar de Vries calls ‘our susceptibility to myths’ (De Vries 2012, 165). Both De Vries and Erkki Huhtamo argue that it is increasingly important to note that, especially in this age of commercial and industrial media culture, recurrent hopes and dreams ‘can be consciously activated and ideologically and commercially exploited’ (Huhtamo 1996).

Dreams incorporated

[Microsoft] is the only operation in the world of any kind – public, private or otherwise – where some of the things I’ve dreamed

about could actually be brought out into the world. (Jaron Lanier – author and Partner Architect at Microsoft Research – cited in

Microsoft News Center 2011)

In one of his articles for Wired, Tim Carmody makes us aware of what he calls “Secret Microsoft”, a side of Microsoft that is less obvious and attracts less attention than the side of the company that is associated with its commercial products such as Windows and Windows Phone (Carmody 2011).48 We have already seen a hint of Secret Microsoft in the account of the company’s undertakings to regulate Kinect’s evolution by implementing the efforts of the open-

65


source community. I would argue that Secret Microsoft – in an equally quiescent way – also consists of a company that builds upon and incorporates what Imar de Vries calls ‘the myth that we move forward along a line of successive and successful media improvements’ (De Vries 2012, 20). An important indication Microsoft is indeed conscious of the persuasive potential of imaginative future scenarios in relation to Kinect becomes clear by looking at a promotional video for Kinect called “The Kinect Effect”.49

In the short clip we see multiple visionary depictions of how Kinect could be used – ranging from playing invisible instruments to applications in healthcare and education – accompanied by a voice-over that creates and emphasises an aura of mysticism around the device. The voice-over mentions, for instance, how ‘it all started with a sensor that turned voice and movement into magic’ (Xbox 2011) and how ‘the world started to imagine things [with Kinect] we hadn’t even thought of’ (ibid.). These rhetorical and visual strategies are not uncommon to marketing efforts in general but the point I intend to make here is that it strengthens the claim that there is a symbiosis between the power of imagining what Kinect could be and Microsoft acting upon it – a similar finding Imar de Vries unveils in relation to communication technologies (De Vries 2012, 165).50

In the video, the visual presence of material technology is reduced to a minimum (see figure 2). There is hardly a sign of a PC, laptop, tablet of smartphone – the technologies that are omnipresent today have completely vanished. The only technical objects that are visible are screens, in all shapes and sizes, and, indeed, Kinect, although the device is generally positioned in the background.51 Using gestures that appear natural to the human body, the people in the video appear

66

the kinect effect

67


Figure 2: Stills from “The Kinect Effect” depicting visionary applications and uses for Kinect. Top-left: a student giving a classroom presentation. Middle-left: a child at a physiotherapist’s office stretching his leg. Bottom-left: a surgeon scrolling through x-rays. Above: a professor guiding his audience through a projection of the galaxy (Xbox 2011).

to interact with the screens whilst Kinect monitors their movements. The interactions seem effortless, uninterrupted, and not limited by physical locations – the user does not have to adapt to the information, the information seems to adapt to the user. The displayed digital information appears to be instantaneously retrieved without loading-time and seamlessly entwined with the physical world. These observations correspond with Microsoft’s desire to progress towards natural, human-centred, and transparent ways of computing, outlined in the first chapter of this study. Microsoft translates its vision of future HCI into this video and, at the same time, emphatically points at Kinect as the key shackle in the route towards this future. Furthermore, simply using the term “Kinect Effect” shows how Microsoft is able to appropriate something potentially negative (i.e. the hacking of its product) and to rebrand it as something

68

the kinect effect

potentially beneficial.Microsoft does not only incorporate user activities on

the level of technical design, the company also implements user activities on a rhetorical and metaphorical level in its marketing efforts. Microsoft uses Kinect’s seemingly endless potential in market segments other than gaming to popularise its vision of what computing in the near future should become. This vision does not impel Microsoft’s research efforts unconsciously; it is knowingly integrated in the backbone of Microsoft Research. Dan Reed, Corporate Vice President of Microsoft XCG, for instance, states that he has been working hard to accelerate change in hardware, systems, platforms and applications (Reed 2009). The efforts of XCG are purposely aimed at ‘reshaping our fundamental assumptions and practices, including computing on a massive, unprecedented scale’ (ibid.). XCG partners with Microsoft Research, Microsoft product groups, and industry hardware partners to explore ideas and build a range of prototypes of which some have already been incorporated into Microsoft products (ibid.).

One of the better-known faces of Microsoft’s XCG division is computer scientist and author Jaron Lanier.52 His role within XCG is to work on ‘making some pretty extraordinary advances’ (cited in Bishop 2011). Lanier also offers his services to Microsoft Research and was closely involved as a consultant in the development of Kinect (Microsoft News Center 2011). According to Lanier, Kinect is ‘a rare instance where consumer technology leaps past what is happening in research labs’ (ibid.) and it is ‘something I was waiting for for decades’ (cited in Bishop 2011). When Lanier heard of Project Natal he realised that this company could very well be the organisation that could realise some

69


of the goals and technologies he has been enthused about for years (Brustein 2011). Kinect, according to Lanier the latest and greatest advancement in VR, has the ‘ability to input yourself bodily [which is] something fundamentally different than we had before and something fundamentally beautiful and exciting’ (ibid.). Appointing Lanier, according to TIME magazine one of the 100 most influential thinkers of 2010 (TIME 2010) and someone who ‘has been dreaming about technology for a long time’ (Microsoft News Center 2011), specifically to a division that works on the future of computing, exemplifies Microsoft’s intentions to build upon dreams of the future – dreams of which Microsoft is well aware have the agency to shape expectations of technology profoundly.

According to former Microsoft executive Ray Ozzie, ‘[t]hose who can envision a plausible future that’s brighter than today will earn the opportunity to lead’ (Ozzie 2010). The computer hard- and software industry with major companies such as Microsoft, Google, Apple, and Facebook, therefore, is, as Lee Worth Bailey puts it, inherently ‘teeming with dreams, visions, hopes, goals, expectations and imaginative premises’ (Bailey 2005, 17). Upholding the thought that there is always an improvement on the horizon is part of the blood that runs through this industry’s veins – it is in part what keeps the production lines running. It is in the industry’s best interest that the need for change is not only preserved within the industry itself but also amongst the general public.53 It is for this reason, companies such as Microsoft, as Imar de Vries puts it, uphold ‘a sense of non-fulfilment that continuously needs to be addressed’ (De Vries 2012, 20).54

Showing people what the near future of computing might

70

look like and telling them that technological development is the way to improve the human condition is an important way of keeping the people interested. According to Imar de Vries, this is what “the new” does.

[The new] perpetually gives our technological imaginary, our yearning for wholeness and completeness that is projected upon technology, fresh impulses by portraying existing technologies as inadequate, and, in the same sweep, by introducing us to the next big thing as a solution. (De Vries 2012, 165-166)

This notion makes us aware of the far-reaching power of the technological imaginary. It does not only act as an undercurrent guiding technological development, its seductive nature can also be knowingly applied as a regulative mechanism within the evolutionary process of a technical artefact. Kinect for Windows, on first sight, might seem the obvious next step in Kinect’s short but successful lifespan. Broadening our perspective, however, shows how, similar to the emergence of Kinect for Xbox 360, Kinect for Windows can be regarded a consequence of various actors affecting each other over time.

71

Our journey through the development of Kinect started with the rather straightforward question why we now have this specific technology. I proposed to take on a broad perspective, which would avoid us being lured to explain the device’s emergence merely as a response to market demand or as the inevitable next step in a successive line of similar products. I have taken the complexity and multifaceted nature of technological development into account, first, by focussing on the discourse surrounding Microsoft Research, a division of the company that has played a crucial role in the emergence of Kinect. In chapter 1 we see how there exists a desire amongst Microsoft representatives and executives to progress towards more “natural” forms of HCI. This potential shift in the interface paradigm is referred to as a transition from GUI to NUI. More natural forms of HCI, which are more adapted to our cognitive and sensory abilities, or, “more like us”, should lower the participation threshold for people to a degree that allows everybody at any place and any time to benefit from computing. This desire forms a substantial part of Microsoft’s overall long-term research and product development strategy.

In the second half of the first chapter I have approached Microsoft’s desire to make HCI more natural from a historical perspective in order to unveil this desire’s underlying hopes and dreams. Microsoft’s research strategy corresponds with recurrent hopes and dreams related to VR research such as reaching full-body immersion, making technology transparent, and achieving man-computer symbiosis. These

Conclusion

72

the kinect effect

transcendental dreams have set the agenda for Microsoft’s research efforts and the company considers Kinect for Xbox 360 to be an important and successful first step in reaching these dreams. These statements, however, do not so much reveal the actual incentives for the initiation of Project Natal but rather combine to form a representation of what Kinect according to Microsoft should be and how the device should work. Microsoft’s ideological rationale for Kinect has contributed for a large part to the framing of the device’s functionality and identity. As Patrice Flichy points out, however, one should critically approach such retrospective justifications when attempting to trace the initial motives for the development of a technical artefact.

Focussing specifically on Microsoft’s incentives for initiating Project Natal debunks the claim that Kinect for Xbox 360, as a consumer product, is somehow a logical outcome of Microsoft’s research efforts. Chapter 2 shows how interactions between actors such as the Nintendo Wii, the fear of losing market share in the gaming market, and the decay of Microsoft’s image as an innovative company can be regarded as primary incentives for the development of Kinect for Xbox 360. Each of these actors is at odds with Microsoft’s representation of Kinect as a well-planned first step in the company’s gradual path towards a NUI-oriented era of computing. The motives that lie at the roots of Project Natal are primarily of a political-economic nature, hardly showing a hint of the hopes and dreams excavated in chapter 1. Although Microsoft’s ideological connotations have not played a direct role in the initiation of the Kinect project, these discursive elements have contributed in shaping the public perception of Kinect for Xbox 360 and creating a scenario for how the device should work.

73

conclusion

Patrice Flichy suggests that an ANT approach offers no room for tracing a collective vision or “imaginaire” guiding technological development. I would claim that, in the case of Kinect, heeding the theoretical principles of ANT does not obstruct investigating the agency of an ideological framework underlying technological development. I believe that in a micro scale analysis of technological development one needs the notion of a network of various human and non-human actors affecting each other in conjunction with the notion of hopes and dreams acting as an undercurrent influencing actors in their mutual configuration in order to cover all elementary aspects of a technical artefact’s emergence and further development. The evolution of Kinect exemplifies the heterogeneous character of technological development and underlines the necessity of the broad perspective – acknowledging how the evolution of a technical artefact consists of continuous interactions between various social and technical forces.

One can pose the legitimate question whether Microsoft’s desire to progress towards a NUI future has played a substantial role in initiating the development of both Kinect for Xbox 360 and Kinect for Windows. Microsoft needed a competitor to show them that the path to a broader audience for the Xbox 360 was through gestural gaming. The inventiveness and enthusiasm of amateur programmers, subsequently, showed Microsoft that Kinect’s potential exceeds its role as gaming peripheral. It is, indeed, hard to deny the fact that Microsoft primarily needed incentives stronger than the desire to progress towards natural forms of HCI to decisively employ all of its efforts and commence development of Kinect for Xbox 360 and Kinect for Windows. The agency of this desire,

74

the kinect effect

however, lies in its ability to construct and shape the ways in which both technical artefacts are represented and perceived, which, in turn, set conditions for further actions. I believe that this claim is substantiated by the findings of chapter 3, which reveal how Microsoft makes good use of consumers’ susceptibility to the temptation of “the new” and implements conflicting ideological connotations on the level of design as well as on a rhetorical level to regulate the evolution of Kinect. Microsoft actively draws upon hopes and dreams, both internally and externally, for keeping one step ahead of the competition, keeping the public interested, and keeping revenues high. Kinect, thus, is developed in an environment that is, by necessity, teeming with hopes, dreams, and fantasies.

This micro scale study illustrates how it is a hard if not impossible task to get a full comprehension of a technical artefact’s construction when one is not fully aware of the agency of hopes and dreams in technological development. Utopian rhetoric does not affect technological development coincidentally; its seductive nature can be consciously inserted as a regulative mechanism within the evolutionary process of a specific technical artefact. I believe that the case of Kinect illustrates the necessary to be aware of the reciprocal relationships between utopian discourses and technical affordances when investigating technological development on a micro level. Scholars, moreover, should be careful to not let the enchanting nature of hopes and dreams cloud their critical judgement. In order to obtain a full and nuanced comprehension of a technical artefact’s construction the black box needs to be fully opened – investigating the ways in which various social and technical aspects come together as a durable whole.

76

This study, as I have attempted to make clear above, focuses on the causes for Kinect. Let us, as an end to our journey through the construction of Kinect, finally, take a short look at its effects. On June 21, 2012, the day Microsoft announced Windows Phone 8, Adrian Covert of technology website Gizmodo writes how he believes ‘that Microsoft is the most innovative consumer tech company right now […] a company reborn’ (Cover 2012). Bit by bit the software giant is reclaiming an image of innovation amongst tech pundits, a process the Redmond-based corporation is silently but steadily working on since it released the Xbox 360 about six years ago. Kinect’s commercial success and its critical acclaim surely have accelerated this process. I would note, however, that Kinect is more than just a push in the back of a company that is struggling to reclaim the leading role it was forced to renounce a long time ago. The success of Kinect for Xbox 360 and its widespread appropriations in fields other than gaming have, in a way, ushered a frantic quest for the best successor to the graphical user interface. A company like Leap Motion, for instance, shows what is next for gesture interfaces. The small device, which costs around 70 dollars, is able to detect the smallest of movements, surpassing touch interfaces on accuracy (Mims 2012). Apple, reportedly, also works on a Kinect-like interface system, which recognises all kinds of movement, including hand gestures (Staska 2011). Furthermore, the likelihood we will see Kinect-like televisions in the near future increased significantly when Samsung presented its new line of Smart

Epilogue

77

TV’s at CES 2012, which enables users to control the device in much the same way as Kinect does (Lawler 2012).

Many probably see Kinect as an amusing video game add-on – fun but insignificant. Probably a whole less people comprehend what Kinect actually means for the future of HCI in general. With Kinect, Microsoft managed to successfully gather and push forth technologies that were available only to research and development divisions of tech companies into the living rooms of common families. Technologies that were unaffordable only five years ago are now available for less than two hundred dollars. The emergence of a multitude of non-gaming appropriations of Kinect was, for some, an unexpected side effect of advanced technologies becoming available to a mass-audience. The numerous hobbyist appropriations and Kinect’s instant and widespread success for the Xbox 360 indicate how the general public has embraced this device wholeheartedly. Regardless whether Kinect is seen as a just another gaming peripheral or as a new project for the homebrew hacker community, the device appears to be unified and characterised by the emission of an almost inexhaustible potential. Time will tell whether we will eventually switch to an era of computing in which technology becomes “natural”. If so, the role of Kinect in this transition will likely be considered substantial. Even though we understand Kinect’s development more accurately in terms of evolution – by analysing the device from the inside, it is my opinion that when a new technical artefact yields such widespread acceptance and enthusiasm it seems, based on its effects, acceptable to consider Kinect a revolution.

78

Notes

Although Kinect was released five years later than the first Xbox 360 model, the peripheral is compatible with all Xbox 360 models. Kinect is currently sold either as a standalone product or together with the latest Xbox 360 S model. According to the NUI Foundation, a non-profit organisation that provides a global platform for sharing and developing NUI technologies (also see note 37), the term “natural user interface” refers to an emerging form of human-computer interaction (HCI), which focuses specifically on human abilities such as touch, vision, voice, motion, and higher cognitive functions such as expression, perception, and recall. For more information see: http://nuigroup.com/faq/Kinect for Windows was officially released on February 1, 2012 in twelve countries (the US, Australia, Canada, France, Germany, Ireland, Italy, Japan, Mexico, New Zealand, Spain, and the UK) for a retail price of 249 US dollars (Eisler 2012a). On March 26, 2012, Microsoft announced it would release Kinect for Windows in nineteen more countries. Hong Kong, South Korea, and Taiwan have access to the product from May, 2012. Austria, Belgium, Brazil, Denmark, Finland, India, Netherlands, Norway, Portugal, Russia, Saudi Arabia, Singapore, South Africa, Sweden, Switzerland, and the United Arab Emirates have access from June, 2012 (Eisler 2012b).Senior writer for Wired Magazine, Steven Levy, has written an extensive work on hackers and hacker-culture. Hackers are, according to Levy, ‘computer programmers and designers who regard computing as the most important thing in the world’ (Levy 2010, ix). Levy notes how hackers are often seen as ‘either nerdy social outcasts or “unprofessional” programmers’ (ibid.) whilst he found them to be quite different. According to Levy, hackers are united by a common philosophy of ‘sharing, openness, decentralization, and getting your hands on

1.

2.

3.

4.

79

machines at any cost to improve the machines and to improve the world’ (ibid.).New York based open-source initiative Adafruit Industries is a big name in open-source hardware development and is led by Massachusetts Institute of Technology (MIT) Media Lab’s Limor Fried and MAKE magazine’s Senior Editor Phillip Torrone.Wolfgang Schivelbusch is a cultural historian who, according to media historian Erkki Huhtamo, continues the tradition of German-Jewish philosopher Walter Benjamin. According to Huhtamo, Schivelbusch and Benjamin both consider culture – buildings, technologies, commodities, illustrations and literary texts – as inscriptions, which ‘could lead us to understand the ways in which a culture perceived itself and conceptualized the “deeper” ideological layers of its construction’ (Huhtamo 1996). Schivelbusch’s work The Railway Journey, for instance, shows how factors other than a technological imperative have driven the development of the railroad in Europe. This work is a pre-eminent example of treating history as ‘a multi-layered construct, a dynamic system of relationships [instead of] a predominantly chronological and positivistic ordering of things, centred at the artefact’ (ibid.).Flichy points out three views on how digitisation should be applied to television: 1) HDTV (high-definition digital television), which, according to Flichy, is a primarily European concept with its roots in the French habit to think of television “cinematographically”, 2) personalised, interactive television (push media) championed by MIT chairman and Wired co-founder Nicholas Negroponte, and 3) multi-channel cable and satellite television, which uses increased bandwidth to multiply the number of TV channels and provide more of the same (Flichy 1999).Tom Gunning registered a sense of déjà vu when comparing the ways in which people experienced and expressed anxieties related to nineteenth century “new” transportation and communication technologies with contemporary reactions to newer forms of technology. According to Gunning, ‘with new

5.

6.

7.

8.

80

the kinect effect

technological topologies confronting us, I believe we look back at the first experiences with technology with an uncanny sense of déjà vu. Not only do we confront the same ambivalence of optimism and anxiety but the scenarios constructed around these primal ambiguities seem even more clearly legible’ (Gunning 1991, 185). According to Imar de Vries a “multiscalar approach” offers a ‘balanced overview [that creates] a perspective on technology that, on the one hand, takes into account the social, cultural, political, and economic factors that c0-construct specific uses of specific technological artefacts, but on the other hand also recognizes the psycho-cultural processes that have been present in various guises during technological development, as an undercurrent influencing the ways we think about, develop, promote, use, and value communication technology’ (De Vries 2012, 95).Technological determinism and social constructivism are generally considered two important but fundamentally opposing currents within media studies. Both “schools” have been criticised for favouring the agency of either social or technical forces over the other. On the whole, technological determinism regards technology as external to society, acting as an autonomous force shaping society. Social constructivism, on the other hand, regards technology as obtaining no explicit objective and, therefore, as inherently subdued to and dependent of social forces defining its outcome. According to Imar de Vries, ‘neither technological determinism nor social constructivism satisfactorily provides the means to find an answer to the question how myths of progress shape communication technologies’ (De Vries 2012, 91).Strictly and exclusively following an ANT approach, as it is understood, for instance, within the social sciences, would take months of writing down observations and conducting interviews with Microsoft employees. This, of course, is not what I intend to do. I draw from ANT’s theoretical principles as a means for creating an angle to approach my object of study.According to Erkki Huhtamo, ‘“topoi” can be considered as

9.

10.

11.

12.

81

notes

formulas, ranging from stylistic to allegorical, that make up the “building blocks” of cultural traditions; they are activated and de-activated in turn; new topoi are created along the way and old ones (at least seemingly) vanish. In a sense, topics provide “pre-fabricated” moulds for experience. Even though they may emerge as if “unconsciously”, they are, however, always cultural, and thus ideological, constructs’ (Huhtamo 1996). The interesting thing about “topoi”, according to Huhtamo, is not whether the representations depicted in visual or literary discourses actually happened but rather that these representations are recurrent. Huhtamo notes that it could even be claimed that ‘the reality of media history lies primarily in the discourses that guide and mould its development, rather than in the “things” and “artefacts” that [presumably] form the core around which everything (r)evolves’ (ibid.).I follow the development of Kinect roughly in a chronological or linear time path to avoid jumping back and forth in time and reduce the chance of confusing the reader. I would like to defuse the notion that the structure of my research alludes to pursuing a teleological view on technological development and history. As pointed out by Imar de Vries, ‘[a]n evolutionary perspective on technology, when applied in its Darwinian non-progressive version, is […] very useful for avoiding teleological notions of development’ (De Vries 2012 97). I, therefore, do not want to regard Kinect’s development as working towards a “telos”, a pre-given final destination. By pursuing an evolutionary understanding of technological development I aim to emphasise the erratic character of Kinect’s development. I, moreover, follow De Vries when he intends ‘to expose teleological thinking rather than practice it myself ’ (ibid., 101).Former Proctor & Gamble (P&G) brand manager Kevin Ashton coined the phrase Internet of Things in 1999 to link the new idea of radio-frequency identification (RFID) to P&G’s supply chain. Ashton used this term to argue for a greater focus on actual things – the notion that our economy, society and survival are not based on ideas or information but on physical things (Ashton 2009). The implementation of RFID chips

13.

14.

82

the kinect effect

and sensor technology would ‘enable computers to observe, identify and understand the world – without the limitations of human-entered data’ (ibid.).Scholars frequently describe the immersive experience of VR as “stepping through Alberti’s window”. This phrase is used as a metaphor for the concept of passing through an image or picture to enter the space that is depicted on the surface. Leon Battista Alberti, an early fifteenth century art theorist, formulated his groundbreaking method of constructing images using perspective in his work Della Pittura first published in 1435 (Lister et al. 2009, 115).Xerox PARC (Palo Alto Research Center Incorporated) was founded in 1970 as a research and development division of Xerox Corporation and is well known for its contributions to the development of information technology and hardware systems. Xerox PARC incubated many of the technologies we commonly use today including the GUI with its windows and icons, the mouse, and the laser printer. PARC became an independent company in 2002 and is now dedicated to develop advances in science and business concepts with the support of commercial partners and clients. For more information see the following links: http://en.wikipedia.org/wiki/PARC_(company) and http://www.xerox.com/innovation/business-technology-research/enus.html.Weiser’s article shows more parallels with Microsoft’s vision of the future of computing and the company’s motive for transitioning from GUI to NUI. Weiser points to the limitations of the PC – at that time the pinnacle of information technology – by stating that it is a machine that cannot truly make computing an integral, invisible part of the way people live their lives. In order to achieve the real potential of information technology, according to Weiser, we must ‘conceive a new way of thinking about computers in the world, one that takes into account the natural human environment and allows the computers themselves to vanish into the background’ (Weiser 1991, 94).Alan Turing was a British mathematician whose work is

15.

16.

17.

18.

83

notes

considered the basis for the modern theory of computation. During the Second World War, Turing played a vital role in deciphering the encrypted messages from the German Enigma machine. After the war, Turing focused his research on computer science and developed a body of work that helped to form the field of artificial intelligence. For more information on Alan Turing see: http://en.wikipedia.org/wiki/Alan_Turing.The idea of “telepresence”, for instance, which allows someone to receive enough sensory feedback so he or she feels like really being at a remote location, began to intrigue our culture at large, as this and other forms of VR became more widely known to scholars of visual culture (Fischer 1991; Lister et al. 2009, 114).An example of a visual display technology attempting to match as closely as possible human cognitive and sensory capabilities is “stereo imaging”. Developed in the early 1950s, stereo imaging is a technology in which a perception of depth was created by presenting a slightly different image to each eye of the viewer. Other examples are the Cinerama, which uses three projectors to present a wide display image, and the Sensorama, built in 1962, which uses images, sound, vibration, and winds to simulate a motorcycle ride through New York City (Fischer 1991).Imar de Vries has found recurrent discursive elements in the necessary fictions of communication improvement to consist of “topoi” of world peace, ultimate understanding, cultural and social unification, unlimited progress, a return to a pre-Babel existence, the sharing of a universal language, a relief of all anxiety, and the realisation of utopia (De Vries 2012, 102-103).According to Alex Kipman, General Manager of Incubation at IEB, ‘the journey we started with Kinect could not have been possible without the amazing partnership with Microsoft Research’ (MicrosoftResearch 2011i).Bill Gates underlines the importance of creating natural forms of computing, as he is convinced that NUI technologies are transformative technologies that, for the first time, will allow computers to adapt to our needs and our preferences. According to Gates, NUIs will allow people to use technology

19.

20.

21.

22.

23.

84

the kinect effect

without any special knowledge or training, which opens up a whole new range of possibilities, for instance, to people with physical and cognitive disabilities (Gates 2011).The Nintendo DS (Dual Screen) is Nintendo’s portable game console first released on November 21, 2004, and features two separate LCD screens of which the lower is a touchscreen. In the frequently asked questions section on Nintendo’s website, Nintendo claims that “DS” also stands for “Developers’ System” because ‘we believe it gives game creators brand new tools to which will lead to more innovative games for the world’s player’ (see: http://www.nintendo.com/consumer/systems/ds/faq.jsp#ds). The success of the creative games that were made to utilise the innovative hardware design of the DS was an important reason for Nintendo to approach the design of its Wii console in a similar way.The PlayStation Move is a wand-shaped motion-sensing game controller for Sony’s PlayStation 3 video game console. An additional PlayStation Eye camera tracks the wand’s position whilst internal sensors in the wand detect its motion. Similar to Kinect for Xbox 360, Move was first revealed at the E3 of 2009. The device was released on September 15, 2010, in Europe and Asia and on September 17 in North America and the UK. See: http://en.wikipedia.org/wiki/PlayStation_Move for more information.This is also observed by researcher Bart Simon, who states that in Wii advertisements ‘we see players move their hands, arms, torsos and even their whole bodies in excited, broad and enthusiastic gestures [that] seem far in excess of the gestures required for the game’ (Simon 2008, 11).The partnership between Microsoft and Nokia, first announced in February of 2011, would lead to sharing of technology and ecosystems, royalties, cash injections, and intellectual property licensing. When the deal sealed in April of 2011, expectations amongst analysts were high. Research firms Gartner and IDC predicted that Windows Phone would catch up Apple’s iOS and lie second only to Android by 2015 (Bright 2011).The Millward Brown BrandZ Rankings, first released in 1998,

24.

25.

26.

27.

28.

85

notes

is an annual ranking of the top 100 most valuable global brands issued by Millward Brown, a global marketing research organisation focussed on brands, media and communications.Google took over first place in 2007 and in 2010 IBM and Apple roared by with Microsoft dropping down to fourth. In 2011, Microsoft sank down to fifth place and sat behind a non-technology company (McDonald’s) for the first time (Bekker 2011).On August 15, 2011, Google announced it acquired Motorola Mobility for an estimate of 12.5 billion US dollars (Wauters 2011). This biggest acquisition ever for Google is, according to CNN Money / Fortune analyst David Goldman, ‘a watershed moment for the company, marking […] Google’s transition from a search-and-software company to a consumer gadget maker’ (Goldman 2012).Although Microsoft did make hardware (e.g. Kin and Zune) and still does (e.g. mice and keyboards), it has never made its own Windows-based computer aside from the Tablet PC, which was first announced in 2000 by former CEO Bill Gates but never really took off (Arar 2000).With this move Microsoft seems to abandon the business model that helped the company climb to its greatest successes. Whether this change of tactics will herald the much needed turnaround success for the software giant remains to be seen. David Pogue notes how the story of Surface seems familiar. According to Pogue, when ‘Apple comes up with a hit product […] Microsoft comes up with a rival that’s nicely designed’ (Pogue 2012). Pogue predicts a tough climb ahead for Surface when he points to the fact that ‘the iPad’s been around for two years; it’s awfully late for Microsoft to begin its pursuit now’ (ibid.).During the whole of 2007 the prospect for the Xbox 360 remained pretty grim. Chris Kohler of Wired notes how, at that time, Nintendo’s Wii is ‘absolutely gobbling up [the family-friendly] market while Microsoft sleeps’ (Kohler 2007). Microsoft needed to wake up fast in order to win over a bigger audience – a hard thing to do because, as Kohler puts it, ‘Xbox

30.

29.

31.

32.

33.

86

the kinect effect

360 is an expensive, complicated, high-end piece of machinery aimed at technophiles and people who like shooting things’ (ibid.) and the company’s initial attempts to target children did not live up to expectations. According to Bloomberg analyst Dana Bass, Microsoft initially attempted to conquer the family-friendly market, especially small children, by creating family-friendly content. The game “Viva Piñata”, for instance, a game in which children can build a garden an fill it with piñata-looking animals, did not make it into the top 20 of biggest selling console games in the US (Bass 2007).According to Wired, the success of Kinect has convinced even Steve Ballmer, someone who has never been able to communicate enthusiasm about NUIs the same way he can fire himself up for almost anything else Microsoft does, to now own Kinect, take control of it, and position it as a key component in the future of the company (Carm0dy 2012).Microsoft labels its plan for the future the “three screens and a cloud” strategy. The plan is to have Microsoft software, preferably the upcoming Windows 8, on the PC (including the tablet), the TV, and the phone, with Microsoft-powered cloud-based servers (permanently online remote servers allowing personal files such as photos to be accessible at any given moment at any given place) tying everything together (Bright 2010). This almost three year old strategy was given a new boost when Microsoft announced a new Xbox service called SmartGlass at the E3 conference in June of 2012. SmartGlass allows users to synchronise their entertainment viewing across multiple devices (Davis 2012). According to Thijs van Doorn, Marketing Manager for Xbox at Microsoft Netherlands, SmartGlass illustrates well the conversion of screens. When somebody is watching a movie on his phone or tablet on the way home from work, for instance, you can simply continue watching the movie on your TV. From that moment on, your tablet or phone becomes a “second screen” giving additional information about the movie. According to Van Doorn, Kinect will certainly play a role in this “convergence of screens” philosophy in the near future (Van Doorn 2012). See page 108 of

34.

35.

87

notes

this study for the full interview with Thijs van Doorn.The Electronic Frontier Foundation (EFF) was founded in July of 1990 by Mitch Kapor, former president of Lotus Development Corporation, John Perry Barlow, lyricist for The Grateful Dead, and John Gilmore, an early employee of SUN Microsystems. They formed EFF to work on civil liberties issues raised by new technologies. To this day, EFF continues to fight for people’s rights in cyberspace. See: https://www.eff.org/about/history for more information about EFF’s history, their mission and vision.The Natural User Interface Group (NUI Group) is an open-source community founded in 2006 that creates and shares interaction techniques and standards that benefit designers and developers throughout the world. According to their own website, NUI Group offers a collaborative environment for scientists that are interested in learning and developing modern HCI methods and concepts. See http://nuigroup.com/go/lite/about/ for more information.Héctor Martin was the first to post open-source drivers to Github – a website for hosting software development projects – thereby meeting the Open Kinect contest’s criteria.Science Friday is a weekly science talk show broadcasted over public radio stations in the US as part of National Public Radio’s (NPR) Talk of the Nation. According to their own website, Science Friday covers current science topics that are in the news and tries to bring an educated, balanced discussion by using panels of expert guests joining host Ira Flatow. The panel and host discuss science topics partially by taking questions from listeners during the call-in portion of the show. See http://sciencefriday.com/about/about-the-radio-program.html for more information on NPR’s Science Friday.Alex Kipman even went as far as to state that Kinect was not actually hacked and that the USB output, which transmits colour, depth, motion, and audio was left “open” by design (Carmody 2010a).What caused Microsoft to suddenly change positions on the matter? Tim Carmody believes Microsoft initially came

38.

39.

40.

36.

37.

41.

88

the kinect effect

out with a hard line response because it was reluctant of people using Kinect’s sophisticated hard- and software for unauthorised or even illegal purposes. If Kinect were seen as insecure, according to Carmody, it would be a nightmare (Carmody 2010a). Once Microsoft saw how the open-source drivers were being used ‘it was easier to officially soften its stance’ (ibid.).In the blog post of February 21, 2011, a few months after the Open Kinect contest started, Steve Clayton writes that Craig Mundie, Microsoft’s Chief Research and Strategy Officer, and Don Mattrick, President at IEB, announced plans to release a non-commercial Kinect for Windows SDK in spring of 2011. The announcement was, according to Clayton, held at TechForum, an annual ‘intimate gathering to discuss the company’s vision for the future’ (Clayton 2011c) attended by ‘a handful of Microsoft thinkers and select media convening for a lively show-and-tell discussion’ (ibid.).TechStars is a US start-up accelerator initiative offering programs in cities such as Boston, Boulder, New York City, and Seattle. Over 75 different venture capital firms and investors support TechStars in their mission to invest in promising start-ups. See http://www.techstars.com/program/ for more information.Microsoft’s decision to rely on hardware sales and give away its software for free is notable also given Microsoft’s line of unsuccessful hardware products. See chapter 2 for an overview of Microsoft’s most prominent hardware failures.This tactic is often referred to as “fear, uncertainty, and doubt” or “FUD” in short. Gene Amdahl defined the term, with its current reference to marketing and sales tactics, after he left IBM to found his own company, Amdahl Corp., in 1975. According to Amdahl, ‘FUD is the fear, uncertainty, and doubt that IBM sales people instill in the minds of potential customers who might be considering [Amdahl] products’ (cited in Eric Raymond’s “Jargon File” - an online glossary of computer programmer slang. See: http://www.catb.org/~esr/jargon/). For more information see http://en.wikipedia.org/

44.

45.

42.

43.

89

notes

wiki/Fear,_uncertainty_and_doubt and http://www.cavcomp.demon.co.uk/halloween/fuddef.html.GNU/Linux is a term that refers to operating systems that include the GNU operating system, developed by the GNU Project, and the Linux kernel, the program in the system that allocates the machine’s resources to run other programs, developed by Linus Torvalds. According to Richard Stallman, founder of the GNU Project, the version of GNU, which is widely used today, is often called “Linux” whilst many of its users are not aware that it basically is the GNU system with Linux added. See http://www.gnu.org/gnu/linux-and-gnu.en.html for Stallman’s argument why Linux should actually be called GNU/Linux. See http://en.wikipedia.org/wiki/GNU/Linux_naming_controversy for more information on the GNU/Linux naming controversy.The Shared Source Initiative was launched by Microsoft in May of 2001 to share source code with customers, enterprises, governments, and partners for debugging and reference purposes (see: http://www.microsoft.com/en-us/sharedsource/default.aspx). According to Michael Tiemann, President of the Open Source Initiative (OSI), ‘[s]hared source is a marketing term created and controlled by Microsoft. Shared source is not open source by another name […] it is an insurgent term that distracts and dilutes the Opens Source message by using similar-sounding terms and offering similar-sounding promises’ (Tiemann 2007).“Secret Microsoft” is a part of Microsoft that for the larger part is active behind the scenes and is involved in pursuits such as acquiring corporations and services, partnering with governments, and expanding patent portfolios. This is the part of Microsoft that is responsible for the fact that the company is not stalling in growth, even though its services are under severe pressure from fierce competition (Rosoff 2011a). Underneath the surface, extracted from the public eye, Microsoft sells long-term software contracts to corporate enterprises and government agencies. This so-called “unearned revenue” is collected from software products, which, in some cases, are not

46.

47.

48.

90

the kinect effect

even finished but have already been sold as a license (Rosoff 2011b). Secret Microsoft is also very active in patent registering and patent infringement issues. In 2003, for instance, Microsoft began cross-licensing its extensive patent portfolio – licensing patents in exchange for either royalties or patent access – to companies such as SUN, Toshiba and Siemens (Parloff 2007). Furthermore, in 2007 Microsoft also secretly held shares in a company that started suing GNU/Linux for the presumable infringement of more than two hundred of its patents (Parloff 2007; Honan 2007; Schäfer 2010).I am aware that this brief analysis of “The Kinect Effect” video does not meet the conditions of a thorough semiotic analysis. I do, however, intend to place this analysis within the semiotic tradition. I treat the video as a text and affirm the saussurian notion of language pre-existing the individual speaker – that human subjectivity is always already positioned by semiotic systems such as language. In other words, I wish to place my findings in the context of the semiotic notion of “intertextuality”, which reminds us that each text exists in relation to other texts, or, that a text does not exist in isolation but is always “framed” by other texts. By analysing this video I intend to support my claim that Microsoft is well aware and makes good use of people’s susceptibility to the seductive nature of “the new” – that there is a symbiosis between the power of imagining what Kinect could be and Microsoft acting upon it.According to Imar de Vries, ‘there is a clear symbiosis between imagining what improved communication could be, and acting upon it: on the one side, those working in the business of producing and selling new communication technologies make very good use of our susceptibility to myths about ideal communication, and on the other, we let them do so, as, in the end, we think the same’ (De Vries 2012, 165).The future of the screen is a topic that appears to receive much attention by Microsoft as well. In the video called “Productivity Future Vision (2011)”, released by Microsoft only two days prior to the release of “The Kinect Effect” video, we see transparent

49.

50.

51.

91

notes

and chromeless screens in all shapes and sizes (Officevideos 2011). The prominence of the screen in Microsoft’s vision of the future also becomes evident in the special report by Dutch website NU.nl in which the author got a peak behind the screens at Microsoft’s headquarters in Redmond. The report shows that Microsoft currently works on several screen technologies ranging from technology turning physical objects into touchscreens to screens that are able to “see” through camera pixels placed between regular pixels (Van Hoek 2012).Jaron Lanier is best known as a pioneer in the field of virtual reality ever since he coined the term in the 1980s. He is also renowned for is work as a composer, musician, author, and computer scientist. For more information see http://en.wikipedia.org/wiki/Jaron_Lanier.An example of how this mechanism works is provided by Kevin Kelleher’s article for CNN Money / Fortune in which he gives an overview of the current battle for the future of the mobile platform market (e.g. smartphones, tablets, etc.). There are currently three main competitors seeking dominance of the mobile platform market: Apple’s iOS, Google’s Android, and Microsoft’s Windows Phone 8. According to Kelleher, Apple, Google, and Microsoft have held conferences over the past several months ‘introducing new software, new devices and new strategies – all aimed at building or maintaining each company’s respective market share’ (Kelleher 2012). In the enumeration that follows, Kelleher shows to what lengths the three companies are going to in order to stay ahead in the mobile economy. According to Kelleher, ‘the competitive drive to stay ahead […] is pushing all three to take risks and push new ideas. Of course, the best ideas will be adopted by all companies, prompting them to new innovations. […] [T]hat kind of competition is what keeps the tech industry moving forward’ (ibid.).When Steve Ballmer is asked in Dutch television programme Nova College Tour how to energise a crowd he responds by stating that it is not that difficult since ‘people are always interested in hearing what’s new’ (NPS 2009). From this

52.

53.

54.

92

the kinect effect

statement we can deduce that upholding a sense of non-fulfilment is probably not the hardest task for a company like Microsoft.

94

References

Acohido, Byron. 2006. Nintendo hopes Wii spells winner. USA Today. August 15. http://www.usatoday.com/tech/gaming/2006-08-14-nintendo-qa_x.htm.

Adafruit. 2010. Microsoft now says it’s ‘inspired’ by community finding new uses for Kinect. Adafruit blog. November 19. http://www.adafruit.com/blog/2010/11/19/microsoft-now-says-its-inspired-by-community-finding-new-uses-for-kinect/.

Arar, Yardena. 2000. Gates wows Comdex crowd with Tablet PC. PCWorld. November 13. http://www.pcworld.com/article/34751/gates_wows_comdex_crowd_with_tablet_pc.html?tk=rel_news.

Ashton, Kevin, 2009. That ‘Internet of Things’ thing. RFID Journal. June 22. http://www.rfidjournal.com/article/view/4986/.

Bach, Robbie. 2010. Robbie Bach: KIN unveiling. Microsoft News Center. April 12. http://www.microsoft.com/en-us/news/exec/rbach/04-12kin.aspx.

Bailey, Lee Worth. 2005. The enchantments of technology. Urbana: University of Illinois Press.

Ballmer, Steve. 2012. Steve Ballmer: 2012 International Consumer Electronics Show. Microsoft News Center. January 9. http://www.microsoft.com/en-us/news/exec/steve/2012/01-09CES.aspx.

Bass, Dina. 2007. Microsoft adds Xbox games, may cut price to woo moms (update 2). Bloomberg. June 8. http://www.bloomberg.com/apps/news?pid=newsarchive&sid=aIPqS83Uf93I.

Bass, Dina. 2012. Can Kinect make Windows cool again? Bloomberg Businessweek – Technology. March 8. http://www.businessweek.com/articles/2012-03-08/can-kinect-make-windows-cool-again.

Baxter-Reynolds, Matthew. 2012. Windows 8, Surface and the post-PC world: Avoiding Cobol 2022. The Guardian – Technology blog. June 20. http://www.guardian.co.uk/technology/

95

blog/2012/jun/20/windows-8-surface-post-pc-cobol?utm_source=twitterfeed&utm_medium=twitter.

Bekker, Scott. 2011. Microsoft vs. the world: Can Microsoft knock out its 6 biggest rivals? Redmond Channel Partner. September 7. http://rcpmag.com/articles/2011/09/01/punching-up.aspx? sc_lang=en.

Bell, Donald. 2010. Is Zune dying, or more important than ever? CNET Reviews. October 25. http://reviews.cnet.com/8301-12519_7-20020634-49.html?tag=hotTopicsBody.1.

Biggs, John. 2012. Windows 8: The road ahead. TechCrunch. February 29. http://techcrunch.com/2012/02/29/windows-8-the-road-ahead/.

Bishop, Todd. 2011. Virtual reality visionary Jaron Lanier on his Microsoft gig: Kinect is ‘beautiful, exciting’. GeekWire. April 4. http://www.geekwire.com/2011/virtual-reality-visionary-jaron-lanier-microsoft-gig-kinect-beautiful-exciting/.

Boddy, William. 1994. Archaeologies of electronic vision and the gendered spectator. Screen 35 (2): 105-122.

Boddy, William. 1999. Redefining the home screen: Technological convergence as trauma and business plan. MIT - Media in Transition. December 19. http://web.mit.edu/m-i-t/articles/index_boddy.html.

Bolter, Jay D., and Richard Grusin. 2000. Remediation: Understanding new media. Cambridge, MA: The MIT Press.

Bonnington, Christina. 2012. Windows 8 hands-on: Your desktop is dead. Wired. February 29. http://www.wired.com/gadgetlab/2012/02/windows-8-consumer-preview/.

Boomen, Marianne van den, and Mirko T. Schäfer. 2005. Will the revolution be open-sourced? How open source travels through society. In How open is the future? Economic, social and cultural scenarios inspired by free and open source software, eds. Jan Cornelis and Marleen Wynants, 31-68. Brussels: VUB University Press.

Bricken, Meredith. 1991. Virtual worlds: No interface to design. In Cyberspace: First steps, ed. Michael Benedikt, 363-382. Cambridge, MA: The MIT Press.

Bright, Peter. 2010. A look ahead: 2012 is Microsoft’s turning point.

96

the kinect effect

ArsTechnica – Information technology. December 29. http://arstechnica.com/information-technology/2011/12/a-look-ahead-2012-is-microsofts-turning-point/.

Bright, Peter. 2011. Microsoft-Nokia pact signed, success by no means assured. ArsTechnica – Information technology. April 21. http://arstechnica.com/information-technology/2011/04/pact-signed-nokia-windows-phone-7-handsets-by-year-end/.

Brustein, Joshua. 2011. One on one: Jaron Lanier. The New York Times - Bits. May 23. http://bits.blogs.nytimes.com/2011/05/23/one-on-one-jaron-lanier/#.

Callon, Michel. 1987. Society in the making: The study of technology as a tool for sociological analysis. In The social construction of technological systems, eds. Wiebe Bijker, Thomas Hughes, and T. Pinch. Cambridge, MA: The MIT Press.

Canalys. 2010. North American smart phone shipments to exceed 65 million units in 2010. Canalys Newsroom. March 17. http://www.canalys.com/newsroom/north-american-smart-phone-shipments-exceed-65-million-units-2010.

Carmody, Tim. 2010a. How Microsoft learned to stop worrying and love Open Kinect. Wired. November 22. http://www.wired.com/gadgetlab/2010/11/open-kinect/.

Carmody, Tim. 2010b. How motion detection works in Xbox Kinect. Wired. November 3. http://www.wired.com/gadgetlab/2010/11/tonights-release-xbox-kinect-how-does-it-work/all/.

Carmody, Tim. 2010c. Kinect hacker won’t share, even for money. Wired. November 8. http://www.wired.com/gadgetlab/2010/11/kinect-hacker-wont-share-even-for-money/.

Carmody, Tim. 2010d. Kinect running on multiple platforms, looking cool. Wired. November 12. http://www.wired.com/gadgetlab/2010/11/kinect-running-on-multiple-platforms-looking-cool/.

Carmody, Tim. 2012. Microsoft bets big on Kinect for Windows, but splits its community. Wired. January 11. http://www.wired.com/epicenter/2012/01/microsoft-bets-kinect-windows/.

Carmody, Tim. 2011. There is one Apple, but many Microsofts: The company you don’t know. Wired. July 22. http://www.wired.com/business/2011/07/there-is-one-apple-but-many-

97

references

microsofts-the-company-you-dont-know/.Chen, Brian X. 2011. 10 hacks that make Microsoft’s Kinect a

killer controller. Wired. March 1. http://www.wired.com/gadgetlab/2011/03/kinect-hacks/.

Ciborra, Claudio. 2002. The labyrinths of information: Challenging the wisdoms of systems. Oxford: Oxford University Press.

Clayton, Steve. 2011a. Microsoft is imagining a NUI future. The official Microsoft blog. January 26. http://blogs.technet.com/b/microsoft_blog/archive/2011/01/26/microsoft-is-imagining-a-nui-future-natural-user-interface.aspx.

Clayton, Steve. 2011b. 1 year Next at Microsoft: Natural user interfaces. Next at Microsoft. November 21. http://blogs.technet.com/b/next/archive/2011/11/21/1-year-on-next-at-microsoft_3a00_-natural-user-interfaces.aspx.

Clayton, Steve, 2011c. Kinect for Windows SDK to arrive spring 2011. The 0fficial Microsoft blog. February 21. http://blogs.technet.com/b/microsoft_blog/archive/2011/02/21/kinect-for-windows-sdk-to-arrive-spring-2011.aspx.

Corn, Joseph. 1983. The winged gospel: America’s romance with aviation. Oxford and New York: Oxford University Press.

Corn, Joseph, ed. 1987. Imagining tomorrow: History, technology, and the American future. Cambridge, MA: The MIT University Press.

Covert, Adrian. 2012. Microsoft is the most exciting company in tech, hands down. Gizmodo. June 21. http://gizmodo.com/5889659/microsoft-is-the-most-exciting-company-in-tech-hands-down.

Cressman, Darryl. 2009. A brief overview of Actor-Network Theory: Punctualization, heterogeneous engineering & translation. Paper for Simon Frasier University ACT Lab/Centre for Policiy Research on Science & Technology.

D’Amico, Nate. 2009. Microsoft, the forgotten Web TV giant. GigaOm – NewTeeVee. January 13. http://gigaom.com/video/microsoft-the-forgotten-web-tv-giant/.

Davis, Justin. 2012. E3 2012: Microsoft announces Xbox SmartGlass. IGN. June 4. http://www.ign.com/articles/2012/06/04/e3-2012-microsoft-announces-xbox-smartglass.

98

the kinect effect

Doorn, Thijs van. Interview by Eric Alberts. Digital recording. Microsoft Netherlands headquarters, Schiphol, June 25, 2012.

Douglas, Susan. 1988. Inventing American broadcasting, 1899 – 1922. Baltimore: Johns Hopkins University Press.

Eisler, Craig. 2011a. It’s official: Kinect for Windows is coming soon. Kinect for Windows blog. November 3. http://blogs.msdn.com/b/kinectforwindows/archive/2011/11/03/it-s-official-kinect-for-windows-is-coming-soon.aspx.

Eisler, Craig. 2011b. The Kinect effect. Kinect for Windows blog. November 9. http://blogs.msdn.com/b/kinectforwindows/archive/2011/11/09/the-kinect-effect.aspx.

Eisler, Craig. 2012a. Starting February 1, 2012: Use the power of Kinect for Windows to change the world. Kinect for Windows blog. January 9. http://blogs.msdn.com/b/kinectforwindows/archive/2012/01/09/kinect-for-windows-commercial-program-announced.aspx.

Eisler, Craig. 2012b. What’s ahead: A sneak peak. Kinect for Windows blog. March 26. http://blogs.msdn.com/b/kinectforwindows/archive/2012/03/26/what-s-ahead-a-sneak-peek.aspx.

Elgan, Mike. 2012. Why Microsoft should make its own tablets (and phones and PCs). Computerworld. June 16. http://www.computerworld.com/s/article/9228150/Why_Microsoft_should_make_its_own_tablets_and_phones_and_PCs_?taxonomyId=15&pageNumber=1.

Epps, Sarah Rotman. 2012. The “post-PC” era: It’s real, but it doesn’t mean what you think it does. Forrester. May 17. http://blogs.forrester.com/sarah_rotman_epps/11-05-17-the_post_pc_era_its_real_but_it_doesnt_mean_what_you_think_it_does.

Fischer, Scott S. 1991. Virtual environments, personal simulation and telepresence. In Virtual reality: Theory, practice, and promise, eds. Sandra K. Helsel and Judith P. Roth, 101-110. London: Meckler Publishing.

Flichy, Patrice. 1999. The construction of new digital media. New Media & Society 1 (1): 33-39.

Flichy, Patrice. 2007a. The Internet imaginaire. London and Cambridge, MA: The MIT Press.

Flichy, Patrice. 2007b. Understanding technological innovation:

99

references

A socio-technical approach. Cheltenham: Edward Elgar Publishing Limited.

Flynn, Bernadette. 2003. Geography of the digital hearth. Information, communication and society 6 (4): 551-576.

Gates, Bill. 2011. The power of the natural user interface. The Gates Notes. October 28. http://www.thegatesnotes.com/Personal/The-Power-of-the-Natural-User-Interface.

Goldman, David. 2012. Google seals $13 billion Motorola buy. CNN Money – Fortune. May 22. http://money.cnn.com/2012/05/22/technology/google-motorola/index.htm.

Gunning, Tom. 1991. Heard over the phone: The Lonely Villa and the De Lorde tradition of the terrors of technology. Screen 32 (2): 184-196.

Hackett, Simon. 2010. Microsoft KIN is no iPhone rival. BusinessSpectator. April 19. http://www.businessspectator.com.au/bs.nsf/Article/Microsoft-pd20100419-4N2N7?opendocume nt&src=rss.

Hall, Kenji. 2006. The big ideas behind Nintendo’s Wii. Businessweek. November 16. http://www.businessweek.com/technology/content/nov2006/tc20061116_750580.htm.

Herold, Charles. 2012. The history of gesture gaming: From the Power Glove to the Wii U. About.com. April 26. http://nintendo.about.com/od/wiiliving/tp/The-History-Of-Gesture-Gaming-From-The-Power-Glove-To-The-Wii-U.htm.

Hickey, Matt. 2012. Microsoft’s latest home for Kinect technology: Portable computers. The Daily. January 27. http://www.thedaily.com/page/2012/01/27/012712-tech-kinect-laptop/.

Hoek, Colin van. 2012. Microsoft focust op “schermen van de toekomst”. NU.nl – Tech/Gadgets. April 26. http://www.nu.nl/gadgets/2796094/microsoft-focust-schermen-van-toekomst.html.

Hollister, Sean. 2010. Kinect does hacker’s bidding, but not for fortune or fame (update: more video). Engadget. November 7. http://www.engadget.com/2010/11/07/kinect-does-hackers-bidding-but-not-for-fortune-or-fame-video/.

Honan, Mat. 2007. Microsoft: Linux violates 235 of our patents. Wired. May 14. http://www.wired.com/business/2007/05/microsoft

100

the kinect effect

_linux/.Hughes, Thomas. 1983. Networks of power: Electrification in

Western society, 1880—1930. Baltimore: Johns Hopkins University Press.

Huhtamo, Erkki. 1996. From kaleidoscomaniac to cybernerd: Towards an archaeology of the media. De Balie – Dossier: Media archaeology. http://www.debalie.nl/artikel.jsp?dossierid=10123&subdossierid=0&articleid=10104.

Isaac, Mike. 2011. Microsoft releases Xbox Kinect SDK, hackers get to work. Wired. June 17. http://www.wired.com/gadgetlab/2011/06/microsoft-kinect-hackers.

Isaac, Mike. 2012. With CES sendoff, Microsoft insists it’s still cool. Wired. January 10. http://www.wired.com/gadgetlab/2012/01/microsoft-keynote-ces/.

James, Geoffrey. 2010. How KIN became Microsoft’s worst failure. CBSNews.com – Sales Machine. July 5. http://www.cbsnews.com/8301-505183_162-28550924-10391735/how-kin-became-microsofts-worst-failure/.

Kaplan, Jeremy, and Sascha Segan. 2008. 21 great technologies that failed. PCMag.com. July 18. http://www.pcmag.com/article2/0,2817,2325942,00.asp.

Kelleher, Kevin. 2012. In mobile, it’s now a three-way race. CNN Money – Fortune. July 16. http://tech.fortune.cnn.com/2012/07/16/in-mobile-its-now-a-three-way-race/.

Kim, Ryan. 2008. E3: Nintendo Wii pulls ahead of Xbox 360 in console sales. San Francisco Chronicle. July 17. http://blog.sfgate.com/techchron/2008/07/17/e3-nintendo-wii-pulls-ahead-of-xbox-360-in-console-sales/.

Kohler, Chris. 2007. Microsoft: Xbox success hinges on capturing casual market. Wired. June 8. http://www.wired.com/gamelife/ 2007/06/microsoft_xbox_/.

Kuehnel, Anne-Kathrin. 2007. How open is open? Microsoft’s Shared Source Initiative as an alternative to open source software. Paper contribution to BILETA annual conference 2007.

Latour, Bruno. 1987. Science in action: How to follow scientists and engineers through society. Cambridge, MA: Harvard University

101

references

Press.Latour, Bruno. 1999. Pandora’s hope: Essays on the reality of science

studies. London and Cambridge, MA: Harvard University Press.Latour, Bruno. 2005. Reassembling the social. Oxford: Oxford

University Press.Laurel, Brenda. 1991. Computers as theatre. Reading, MA: Addison-

Wesley.Law, John. 1987. Technology and heterogeneous engineering: The

case of Portuguese expansion. In The social construction of technological systems, eds. Wiebe Bijker, Thomas Hughes, and Trevor Pinch, 111-137. Cambridge, MA: The MIT Press.

Law, John, and Michel Callon. 1992. The life and death of an aircraft: A network analysis of technical change. In Shaping technology / building society: Studies in sociotechnical change, eds. Wiebe Bijker and John Law, 21-52. Cambridge, MA: The MIT Press.

Lawler, Ryan. 2012. How Samsung plans to out-Kinect Microsoft. Gigaom – NewTeeVee. January 9. http://gigaom.com/video/samsung-smart-interactions/.

Lazarus, David. 1997. Microsoft buys WebTV. Wired. July 4. http://www.wired.com/techbiz/media/news/1997/04/2987.

Lee, John C. 2011. Windows drivers for Kinect, finally! Procrastineering. February 21. http://procrastineering.blogspot .nl/2011/02/windows-drivers-for-kinect.html.

Levy, Steven. 2010. Hackers: Heroes of the computer revolution. Sebastopol, CA: O’Reilly Media.

Lister, Martin, et al. 2009. New media: A critical introduction - second edition. London and New York: Routledge.

Lohr, Steve. 1999. Economic view; Is Mr. Gates pouring fuel on his rival’s fire?’ The New York Times. April 18. http://www.nytimes.com/1999/04/18/business/economic-view-is-mr-gates-pouring-fuel-on-hisrivals-fire.html.

MacKenzie, Donald, and Judy Wajcman. 1999. The social shaping of technology – second edition. Buckingham: Open University Press.

Manovich, Lev. 1995. An archaeology of a computer screen. Lev Manovich Essays. http://www.manovich.net/TEXT/digital_nature.html.

102

the kinect effect

Markoff, John. 2004. Contemplating a PC market without I.B.M. The New York Times. December 4. http://query.nytimes.com/gst/fullpage.html?res=9804E0DD1731F937A35751C1A9629C8B63.

Marvin, Carolyn. 1987. Dazzling the multitude: Imagining the electric light as a communications medium. In Imagining tomorrow: History, technology and the American future, ed. Joseph Corn, 202-217. Cambridge, MA: The MIT Press.

Marvin, Carolyn. 1988. When old technologies were new: Thinking about electric communication in the nineteenth century. New York and Oxford: Oxford University Press.

Marx, Leo. 1964. The machine in the garden: Technology and the pastoral idea in America. Oxford and New York: Oxford University Press.

Marx, Leo, and Merritt Roe Smith. 1994. Introduction. In Does technology drive history? The dilemma of technological determinism, eds. Merritt Roe Smith and Leo Marx, ix-xv. Cambridge, MA: The MIT Press.

Microsoft News Center. 2011. Exploring the oceans in our brains with Jaron Lanier. Microsoft News Blog. November 9. http://www.microsoft.com/en-us/news/features/2011/nov11/11-09Jaro nLanier.aspx.

Microsoft News Center. 2012. A career of thriving on “impossible projects” – including Kinect for Xbox 360. Microsoft News Blog. June 12. http://www.microsoft.com/en-us/news/features/2012/jun12/06-12AlexKipman.aspx.

MicrosoftResearch. 2011a. Two decades, many voices. YouTube. September 23. http://www.youtube.com/watch?v=N_qS7jFoIQ 8&feature=plcp.

MicrosoftResearch. 2011b. Celebrating 20 years of research. YouTube. September 26. http://www.youtube.com/watch?v=h5 Zj_Y_lihY&feature=plcp.

MicrosoftResearch. 2011c. Motivation, inspiration, and research. YouTube. September 23. http://www.youtube.com/watch?v=Ht s_6YjpT-k&feature=plcp.

MicrosoftResearch. 2011d. Why research matters. YouTube. September 23. http://www.youtube.com/watch?v=jnVpC8qW

103

references

O1s&feature=plcp.MicrosoftResearch. 2011e. Converging worlds: A new era of

computing. YouTube. October 5. http://www.youtube.com/watch?v=3tNEOQdaAZg&feature=plcp.

MicrosoftResearch. 2011f. Rethinking computing to deliver a new generation of user experiences. YouTube. September 26. http://www.youtube.com/watch?v=5cqdz58Vdxk&feature=plcp.

MicrosoftResearch. 2011g. Transforming user interactions with computers. YouTube. September 23. http://www.youtube.com/watch?v=fLkXAG-ysYw&feature=plcp.

MicrosoftResearch. 2011h. Inventing the future. YouTube. September 23. http://www.youtube.com/watch?v=gl_rjIxdovA &feature=plcp.

MicrosoftResearch. 2011i. Researchers highlight collaborations with Kinect for Xbox 360. YouTube. September 26. http://www.youtube.com/watch?v=r3QhCGncWsM&feature=plcp.

MicrosoftResearch. 2011j. Kinect-based solution for scanning 3-D objects. YouTube. September 22. http://www.youtube.com/watch?v=FYUYzk9FT-Y&feature=plcp.

Mims, Christopher. 2012. The most important new technology since the smart phone arrives December 2012. Technology Review. June 26. http://www.technologyreview.com/view/428350/the-most-important-new-technology-since-the/?ref=rss.

Molen, Brad. 2011. Windows Phone 7.5 Mango review. Engadget. September 27. http://www.engadget.com/2011/09/27/windows -phone-7-5-mango-review/.

Mosher, Dave. 2011. Six months later, Kinect hacks flourish. Wired. May 3. http://www.wired.com/epicenter/2011/05/johnny-lee-kinect-hacking/.

Mundie, Craig. 2011. Craig Mundie: Cleveland clinic. Microsoft News Center. January 5. http://www.microsoft.com/en-us/news/exec/craig/2011/01-05clevelandclinic.aspx.

Nelson, Theodor H. 1974 [2003]. Introduction to Computer lib / Dream machines. In The new media reader, eds. Noah Wardrip-Fruin and Nick Montfort, 301-338. Cambridge, MA: The MIT Press.

Nguyen, Vincent. 2012. Microsoft Surface tablet hands-on [video].

104

the kinect effect

SlashGear. June 18. http://www.slashgear.com/microsoft-sur face-tablet-hands-on-18234512/.

Norman, Donald A. 1999. Affordance, conventions, and design. Interactions 6 (3): 38-42.

NPS. 2009. Nova college tour – Steve Ballmer. Nova college tour. October 11. http://player.omroep.nl/?aflID=10233031.

Officevideos. 2011. Productivity future vision (2011). YouTube. October 25. http://www.youtube.com/watch?v=a6cNdhOKwi 0.

Ovide, Shira, and Ian Sherr. 2012. Microsoft banks on mobile apps. The Wall Street Journal – Technology. April 6. http://online.wsj.com/article/SB10001424052702304750404577321933818976966.html.

Ozzie, Ray. 2010. Dawn of a new day. Ray Ozzie. October 28. http://ozzie.net/docs/dawn-of-a-new-day/.

Paul, Ryan. 2010. Microsoft not amused by open source Kinect driver. ArsTechnica. November 11. http://arstechnica.com/information-technology/2010/11/microsoft-not-amused-by-open-source-kinect-drivers/.

Penny, Simon. 1992. Virtual reality as the end of the Enlightenment project. Simon Penny Texts. http://ace.uci.edu/penny/texts/enlightenment.html.

Pereira, Chris. 2011a. Kinect is the fastest-selling consumer electronics device. 1up. November 3. http://www.1up.com/news/kinect-fastest-selling-consumer-electronics-device.

Pereira, Chris. 2011b. Xbox 360 enjoys its biggest sales week ever with nearly 1m consoles sold. 1up. November 29. http://www.1up.com/news/xbox-36-biggest-sales-week-ever-1m-sold.

Pereira, Chris. 2012. Xbox 360 sales reach 66 million, Kinect 18 million. 1up. January 9. http://www.1up.com/news/xbox-36-sales-66-million.

Peterson, Steve. 2011. Xbox Kinect: One year later. IndustryGamers. November 22. http://www.industrygamers.com/news/micro soft-kinect-one-year-later/.

Pham, Alex. 2009. Microsoft’s Zune, New Year’s Eve don’t play together. Los Angeles Times. January 1. http://articles.latimes.com/2009/jan/01/business/fi-zune1.

105

references

Pogue, David. 2010. Kinect pushes users into a sweaty new dimension. The New York Times. November 4. http://www.nytimes.com/2010/11/04/technology/personaltech/04pogue.html?_r=1.

Pogue, David. 2012. The Surface: Celebrate the competition, question the premise. The New York Times – Pogue’s Posts. June 19. http://pogue.blogs.nytimes.com/2012/06/19/the-surface-a-new-tablet-from-microsoft/.

Reed, Dan. 2009. Extreme computing opens up unlimited possibilities. VentureBeat. October 2009. http://venturebeat.com/2009/10/21/extreme-computing-opens-up-unlimited-possibilities/.

Ricknás, Mikael. 2012. Mistakes, heavy competition forces Nokia to reboot. Computerworld – Mobile and wireless. June 14. http://www.computerworld.com/s/article/9228120/Mistakes_heavy_competition_forces_Nokia_to_reboot?taxonomyId=15&pageNumber=1.

Rieder, Bernhard, and Mirko T. Schäfer. 2008. Beyond engineering: Software design as bridge over the culture/technology dichotomy. In Philosophy and design: From engineering to architecture, eds. Pieter Vermaas et al., 159 – 171. Springer.

Ronell, Avital. 1989. The telephone book: Technology, schizophrenia, electric speech. Lincoln: University of Nebraska Press.

Rosoff, Matt. 2011a. No, Microsoft isn’t in a “growth stall” (at least not yet). Business Insider. May 9. http://www.businessinsider.com/but-microsoft-isnt-in-a-growth-stall-at-least-not-yet-2011 -5.

Rosoff, Matt. 2011b. Anybody who thinks Microsoft is going out of business is nuts – here’s why. Business Insider. July 21. http://www.businessinsider.com/big-business-customers-are-keeping-microsoft-alive-2011-7?op=1.

Schäfer, Mirko T. 2011. Bastard culture! How user participation transforms cultural production. Amsterdam: Amsterdam University Press.

Schäfer, Mirko T. 2012. Unveiling culture. Actors and factors in media practice. New media studies. February 26. http://www.newmediastudies.nl/magazine/unveiling-culture-actors-and-

106

the kinect effect

factors-media-practice.Shaw, Frank. 2011. Where the PC is headed: Plus is the new “post”.

Microsoft TechNet blog. 19 August. http://blogs.technet.com/b/microsoft_blog/archive/2011/08/19/where-the-pc-is-headed-plus-is-the-new-post.aspx.

Sheffield, Brandon. 2010. Opinion: Going through the motions with gesture control. Gamasutra. January 22. http://www.gamasutra.com/view/news/26719/Opinion_Going_Through_The_Motions_With_Gesture_Control.php.

Simon, Bart. 2009. Wii are out of control: Bodies, game screens and the production of gestural excess. Loading 3 (4): 1-17.

Smith, Gina. 2012. Apple founder inventor Steve Wozniak: Why I love my Windows Phone 7.5 Mango. A New Domain. April 26. http://anewdomain.net/2012/04/26/apple-founder-inventor-steve-wozniak-why-i-love-my-windows-phone-7-5-fan/.

Staska. 2011. Apple’s own project Natal – Kinect like 3D controller. Unwired View. December 8. http://www.unwiredview.com/2011/12/08/apples-own-project-natal-kinect-like-3d-controller/.

Sterling, Bruce. 2009. Augmented reality: “The ultimate display” by Ivan Sutherland, 1965. Wired. September 20. http://www.wired.com/beyond_the_beyond/2009/09/augmented-reality-the-ultimate-display-by-ivan-sutherland-1965/.

Takahashi, Dean. 2009a. Microsoft games exec details how Project Natal was born. VentureBeat. June 2. http://venturebeat.com/2009/06/02/microsoft-games-executive-describes-origins-of-project-natal-game-controls/.

Takahashi, Dean. 2009b. Sources confirm Microsoft is buying 3DV Systems. VentureBeat. February 21. http://venturebeat.com/2009/02/21/sources-confirm-microsoft-is-buying-3dv-systems/.

Takahashi, Dean. 2009c. How many vendors does it take to make Microsoft’s Project Natal game system? VentureBeat. September 5. http://venturebeat.com/2009/09/05/how-many-vendors-does-it-take-to-make-microsofts-project-natal-game-control-system/.

Terdiman, Daniel. 2010a. Bounty offered for open-source Kinect

107

references

driver. CNET. November 4. http://news.cnet.com/8301-13772_3 -20021836-52.html.

Terdiman, Daniel. 2010b. Microsoft reverses course, says Kinect left open by “design”. CNET. November 19. http://news.cnet.com/8301-13772_3-20023455-52.html.

Tiemann, Michael. 2007. Who is behind “Shared Source” misinformation campaign. The Open Source Initiative – Michael Tiemann’s blog. November 2. http://opensource.org/node/225.

TIME. 2010. The 2010 TIME 100. TIME Specials. http://www.time.com/time/specials/packages/completelist/0,29569,1984685, 00.html.

Torrone, Phillip. 2011. Things you can’t do with the Microsoft Kinect SDK. MAKE. June 17. http://blog.makezine.com/2011/06/17/things-you-cant-do-with-the-microsoft-kinect-sdk/.

Tu, Janet I. 2011a. As people experiment, they find Kinect is much more than a game. The Seattle Times. October 31. http://seattletimes.nwsource.com/html/businesstechnology/20166 44446_kinect31.html.

Tu, Janet I. 2011b. Microsoft an TechStars launch accelerator incubation program. The Seattle Times. November 21. http://seattletimes.nwsource.com/html/microsoftpri0/2016816806_microsoft_launches_kinect_accelerator_for_startups.html.

Tu, Janet I. 2012. Computing future is name of the game for Microsoft’s Craig Mundie. The Seattle Times. April 2. http://seattletimes.nwsource.com/html/microsoftpri0/2017894821_computing_future_is_name_of_the_game_for_microsoft.html.

Valich, Theo. 2012. Microsoft to bring Kinect UI with Windows Phone 8? VR-Zone. April 10. http://vr-zone.com/articles/microsoft-to-bring-kinect-ui-with-windows-phone-8-/15500.html.

Vries, Imar O. de. 2012. Tantalisingly close: An archaeology of communication desires in discourses of mobile wireless media. Amsterdam: Amsterdam University Press.

Wauters, Robin. 2011. Google buys Motorola Mobility for $12.5B, says “Android will stay open”. Techcrunch. August 15. http://

108

the kinect effect

techcrunch.com/2011/08/15/breaking-google-buys-motorola-for-12-5-billion/.

Weiser, Mark. 1991. The computer for the 21st century. Scientific American, September, 94-104.

Winston, Brain. 1998. Media technology and society. A history: From the telegraph to the Internet. London and New York: Routledge.

Xbox. 2011. The Kinect effect. YouTube. October 27. http://www.youtube.com/watch?v=T_QLguHvACs.

110

Interviewed: Thijs van Doorn (TvD) Interviewer: Eric Alberts (EA)Date: June 25, 2012Location: Headquarters Microsoft Netherlands, SchipholSubject: KinectDuration of interview: 21 minutes and 23 seconds

Note: The original interview is in Dutch. The author has translated and edited the transcript of the recording of the interview to English.

EA: I will introduce myself for formality. My name is Eric and for the past year I’ve worked for Microsoft, part time, and I am currently writing my thesis on Kinect. I would like to ask a few questions about the inception of Kinect and which way things are currently developing. Would you like to introduce yourself as well?

TvD: I am Thijs van Doorn and I am responsible for the marketing of Xbox console and Xbox Live in the Netherlands. And, indeed, this includes all accessories such as Kinect.

EA: All right, well, the first question is about the success of Kinect. I understand the device now officially holds a Guinness World Record for fastest selling hardware device ever. What, do you think, has that success meant for Xbox?

TvD: Well, the funny thing is, actually, that Xbox 360 is

Attachment - Interview with Thijs van Doorn (Microsoft NL)

111

about six years old now and it’s, in a way, quite strange that a device that’s this old is now selling more than the years before. There is clearly an upward trend. Normally, there is kind of a cycle, you know, with all electronic devices. After three years or so you have a peak, then it comes down, and then a new peak arrives.

EA: Yes.

TvD: This cycle is partially due to the fact that the device receives an update every now and then and now, for the second time in its lifespan, the dashboard is completely renewed, which makes it look completely different. People will think that they own a new device while it’s actually the same hardware. Another reason is that the Xbox already had pretty high specifications when it first launched, which makes the device last longer, of course. But especially Kinect has reinvigorated the Xbox. It now focuses on a new target audience, you know? The Xbox - especially the first Xbox - really was seen as a “shooter box”. Think of Halo, shooting games, circular saws, and blood flying around. With the addition of Kinect we really tapped into a completely new audience - more families, I would say, families with children. And gradually you see games emerging - more common games - that are now starting to use Kinect as part of the game. A well-known example, for instance, is a shooter that you operate with the controller - you will probably always need a regular controller when it comes to shooters - and you are able to throw a grenade over your head, like this.

EA: Will we see the rise of a kind of hybrid game, then?

112

the kinect effect

TvD: Yes. Another good example is Forza [Motorsport], for instance, which you operate with the controller or a steering wheel and you are able to look around the corner because of “head tracking”. This means you can look arnound the corner and the screen moves along. So, it really is not just about a game in which you have to jump up and down in a small boat floating down the river. It is also about applications you can use within the game.

EA: I see.

TvD: I think that’s really important for Xbox. Furthermore, it’s also very important for Microsoft, of course, that the technology of Kinect can be used in a Windows environment for things that have nothing to do with gaming.

EA: You mean Kinect for Windows, which was recently released, including the Netherlands.

TvD: Yes, the Netherlands included. There are, for instance, many rehabilitation centres that use this because apparently, well yes, you have to give people certain exercises. But when you receive feedback about whether you are doing these exercises well... that makes it very interesting, of course.

EA: Talking about Kinect for Windows. How are things going in the Netherlands at the moment? Can you say something about that?

TvD: We have only just launched this product so I really cannot say anything about it yet.

113

attachment - interview with thijs van doorn (microsoft nl)

EA: What kinds of companies are involved for instance?

TvD: This product has only just arrived - it is being distributed as we speak. What I can say, though, is that Kinect for Windows is not a consumer product. When you buy this as a consumer it will be rather useless. It is really intended for writing software. There are developers out there who can create all sorts of applications with this product. Many of which have more or less already started on the Xbox. You were able to find stuff on the Internet in order to... kind of hack the device. And you were able to connect it to a PC so there were a lot of people already fooling around with this. Now, with Kinect for Windows, there really is a Kinect sensor specifically for Windows, optimised for the PC. The distance with which you are able to use it, for instance, has been reduced because the distance to a PC screen is much smaller, of course.

EA: This is called “Near Mode” I believe?

TvD: Yes, exactly. Adjustments like this have been made. But this is really a very important innovation, comparable to a touch screen you can operate with your finger. This is the three dimensional version of that.

EA: And the success of the device has obviously weighed in, I guess, on the emergence of Kinect for Windows and the dispersion across multiple platforms such as the PC.

TvD: Yes, and another area where it’s very successful is among the apps on Xbox - not the game apps but the entertainment apps in which you can scroll through a library of movies with

114

the kinect effect

your hand. You can see that only a few months ago in America the magic point has been reached that people spend more time on entertainment, such as music and films, rather than on gaming on the Xbox. So you can rightfully say that it’s no longer a game console but an entertainment console.

EA: And why is that do you think? Because of the addition of video on demand? Netflix?

TvD: Yes because of video on demand but also because of the support of Kinect within that feature. When you look at all studies of video on demand the biggest problem to let it break through lies within the ease of use. The fact that you have all these remotes - which one do I need this time? - and display menus that generally are slow on set-top boxes - Kinect makes it all so accessible. You can simply use your hand, in America even your voice, to order a movie and it’s even easier than turning on your TV. So, this really takes away the threshold and that’s very important for video on demand services.

EA: For that matter, when you talk about interfaces, Kinect, I believe, is really a good example of a natural user interface. That is something Microsoft bets big on.

TvD: Yes, that’s correct.

EA: Do you think Kinect or the technology behind Kinect will play an important role in Microsoft’s plans for the future?

TvD: Well yes, it kind of already does, right? In other

115


words, I do not think we would be this successful with those entertainment apps if we did not have Kinect. Because that’s really the added value and that makes the difference with the things you can do with your set-top box.

EA: And do you think that because of that Kinect will be integrated in, for instance, laptops or phones like Windows Phone?

TvD: Yes that could be true.

EA: All right, let me see. Again, briefly, concerning Kinect’s success and the combination with Xbox. Do you think the success of Kinect has increased Xbox’s lifespan?

TvD: Absolutely, yes.

EA: Will Xbox last for several more years, you think? If so, how many years?

TvD: I do not know. I can’t say anything about that, really. But, yes, I think so when you look at what’s out there. When you buy an Xbox today you don’t buy an out-dated device. You buy a device that can handle the competition with others easily. In a way, you can see the [Nintendo] Wii as the largest competitor because they started with motion technology. Well, they will release the Wii U soon and, from what I understand, I have to hear this from the press too, the specifications of the Wii U are not more powerful than the Xbox. In that respect, despite the fact that the Xbox is six years old, it is still in very good shape. Moreover, indeed, you will not win only on specifications. It’s no longer just

116

the kinect effect

a matter of speed. With HD graphics you can make things all the more faster. That’s important, sure, but what’s more important is addressing a wide audience and removing the threshold - the threshold is the controller. With Kinect you have nothing in your hands and the funny thing is, when you walk into a room with a Kinect active, even when you have an aversion to gaming, you cannot help it but the device will react to your movement and at that moment you are gaming, whether you like it or not. This shows how much Kinect takes away the threshold.

EA: Do you think that the lack of a controller, taking away that technology, has reduced the threshold even more than the Nintendo Wii?

TvD: Yes, I believe so.

EA: Is that the key to Kinect’s success?

TvD: It appears so, yes. People still have to hold a remote in their hands and use buttons. With Kinect you don’t. It’s the same as when you go to a shopping mall and you hear music play - usually very bad music, by the way. You don’t have to do anything to listen to the music, you are affected by the music subconsciously. That is the difference. On first sight, Kinect is very similar to what the Wii and Sony’s Move do but it’s that psychological difference, you know, that you’re holding nothing in your hands and everything goes without saying. That’s the power, that’s the magic.

EA: All right. So, we now have two different products in the market: Kinect for Xbox, which is a consumer product, and

117


Kinect for Windows, which is aimed more at developers. Do you think the two will continue to exist side by side or will the two eventually merge? I mean, for instance, will the Xbox, Kinect included, eventually merge with the PC?

TvD: The reason they exist side by side is because one is aimed at consumers and the other at developers. For a developer it’s much easier to develop on Windows than on the Xbox because in a Windows environment you have all sorts of tools at your disposal. Moreover, it’s also rather obvious that when you want to develop more serious applications you turn to the PC, as you will want to connect to other things you already use within a PC environment. You really have to see Xbox as an entertainment device and it’s more likely to use a PC for more serious applications that include databases and so forth.

EA: Exactly, but I can imagine that you initiate a developer programme with the aim of, eventually, reaping the benefits of such a programme, or...

TvD: Sure, but you should regard Kinect’s technology not as suitable only for gaming applications but also for other applications.

EA: I’ve read something about the open source community playing a large role in opening up Kinect. Kinect was initially positioned as a genuine gaming device. Was opening up Kinect to Windows something Microsoft thought about even before the launch of Kinect for Xbox?

TvD: Well, we entered the market with Kinect specifically

118

the kinect effect

for gaming. That’s the reason why Kinect is developed. At the same time, however, there are people who are smart enough to see the potential of Kinect within other fields. This is something you see happening to gaming anyway. Many games are now becoming educational, right? Games like Sesame Street...

EA: ...serious games...

TvD: ...and things like that. That’s a development you see anyway. Game concepts being applied to more serious affairs.

EA: Indeed. And when you talk about Kinect as natural user interface, do you think Kinect has played a decisive role in the direction in which Microsoft is currently going? Or would Microsoft have gone in that direction anyway, apart from Kinect’s success?

TvD: That’s difficult to say. I think natural user interface is something that not only concerns Microsoft but something many others are currently experimenting with.

EA: I’ve read Samsung has big plans for its televisions.

TvD: Sure, but, in a way, you can say that the Surface table, that large table, is also an example of a natural user interface.

EA: That is something Microsoft is working on for a long time, right?

TvD: That’s true but I find it to be a rather logical step.

119


EA: Last week Microsoft announced the Surface tablet. Will it feature technology from the large Surface table? Is that technology now coming to fruition in Microsoft’s product line?

TvD: I don’t know if the Surface tablet has anything to do with the old Surface, but...

EA: More with Windows 8, perhaps?

TvD: Yes, I think the Surface tablet is very Windows 8 focused.

EA: I’ve also read up on a Kinect 2. There are all kinds of rumours circulating, of course. Is there anything known about this?

TvD: No.

EA: When will a next Kinect be released or are you not at all concerned with that at the moment?

TvD: Not really. I can’t say anything about that.

EA: Very well. Let’s get back to platform convergence for a moment. Windows Phone, Surface, and Windows 8 play a large role in this. Can you say anything about whether Kinect will play a role in this process as well?

TvD: I don’t know if you’ve seen the clips from E3 about SmartGlass? I think here you have a great example of converging screens. We call this the “three screens and a

120

the kinect effect

cloud” philosophy. You could have a scenario, for instance, in which you’re on your way home from work and decide to watch a movie on your mobile phone or tablet. When you arrive home you can then choose to continue watching the movie on a TV screen, right where you left off. Your mobile phone or tablet will then turn into a “second screen” on which you can view additional information about the movie. An explanation from the director about a specific scene, for instance, or a map of the city of New York showing the location of police vehicles. The moment you fast-forward the movie the meta information on your second screen simply moves along. Well, this partially has something to do with Kinect because when you combine this technology with Kinect, which we are currently doing, the experience will become truly magical and fluent. You’ll simply start with your finger on a touch screen and then wave your hand across the living room in front of your TV. I think this is for many people a very natural continuation of the same gesture, the same control. I believe SmartGlass will not be that impressive without Kinect.

EA: Does that mean Kinect for Xbox was a little bit ahead of its time when it first arrived?

TvD: I don’t know if it was ahead of its time. It was, at the time, simply a way of broadening gaming to other applications. And the fact that Xbox is now more an entertainment console than a game console also completely meets our expectations.

EA: You can see that this happened to Zune, too, right? Zune has now become a service for multiple mobile devices and

121


has grown independent of the hardware.

TvD: Yes, that’s true.

EA: Alright. Clear. I’m pretty much done with my questions. Well, I can imagine we will hear much more about Kinect for Windows in the near future?

TvD: Undoubtedly.

EA: And Kinect in combination with laptops and, perhaps, integrated in the Surface tablet?

TvD: Yes, well, the story I find fascinating is the one about two surgeons who use Kinect technology during their operations. When you conduct a medical operation you, obviously, wear gloves. When a surgeon suddenly, in the midst of the operation, needs to watch some x-rays he has take off his gloves, turn on his computer, grab his mouse and keyboard, and search for the necessary files. When you are instead able to leave your gloves on and use gestures to bring up a picture and zoom in and out, well, that’s a very good example of how Kinect can be useful in very serious matters.

EA: I’ve also seen examples of how autistic children completely blossom through the use of Kinect. In that respect it’s truly is a magical device that you can use is multiple fields.

TvD: Indeed. The moment when we were about to launch Kinect we had organised an event for some people of the press. One press member was a representative of Youth

122

the kinect effect

News who had brought along five to ten children - viewers of the programme. Just like all press members, these children had the opportunity to play with Kinect for the first time. We had prepared a talk of about ten minutes to explain how Kinect works, that it tracks your body, that is uses infrared, and that you have to really exaggerate your movements in front of the device. The first journalists were a bit shy and afraid of acting like a lunatic in front of a screen so we had to give them some tips and pointers to make them go all-in. With the children, however, I didn’t even had the chance to finish my ten minute talk. They immediately figured out how operate and interact with Kinect. So, yes, that really is...

EA: Natural.

TvD: Yes, [laughs] exactly. That shows that it’s really natural.

EA: I have one more question. There has been a close collaboration with Microsoft Research in the development of Kinect. How definable has the role of Microsoft Research been? And has the Netherlands played any role, too?

TvD: I find it really difficult to say anything about that because those are things that take place at our headquarters in Redmond. But yes, of course, it has been very important because Kinect is not so much about the hardware. When you look at what’s inside you can easily reproduce the cameras and sensors. Those things are not that special. The true magic occurs through the software. The software interprets all the signals and body movements. So the software is what makes it all so special.

123


EA: My Kinect at home recognises my face. How far will it go because I read that it also recognises clothes. Will Kinect be used within the fashion industry, perhaps?

TvD: Well, you could imagine a scenario in which you go to a store to shop for clothes and instead of waiting in line for the fitting room you can walk up to a mirror of some sort that tells you what to wear. You can turn around in front of this mirror or screen and see yourself with the clothes on that you fancy. And when you can do this in a store then you can do this at home, in your living room, as well. You might not even have to go out to a store anymore. So, I think that this will be a very interesting scenario for an e-commerce kind of application.

EA: Or maybe for video conferencing maybe? Telepresence?

TvD: Yes, it’s already used for that because it has Kinect Video, of course. The fun thing is that Kinect can track you across the room while you walk around. When somebody sits next to you on the couch it can figure out who’s who.

EA: I’ve also read something about using your Xbox avatar in video conferencing. You then get a live conferencing session where everyone’s avatar is in a kind of a virtual space.

TvD: That may well be true, indeed. That’s a potential scenario. Sounds fun.

EA: All right, well, thank you for your time.

TvD: You’re welcome.

the kinect effect

Documents