Open for Open Questions - UX London 2014

Download Open for Open Questions - UX London 2014

Post on 17-Aug-2014




7 download


Many design and usability research methods cater for delving into a focused topic: You set a goal, establish hypotheses, gather data and gain insight to help create the proof, story, a view point, strategy, or whatever you are looking for within the given budget and time. However, there can be situations where your research may focus too much on individual trees that it cannot provide much information about the forest. For instance, what if you have perfect usability test data to prove the effectiveness of your design, but your client may be more interested to know what types of people would buy the product (and get disappointed to hear that you dont know)? What if your polite research participants never want to talk with you about negative things about your design? This talk will share a few anecdotes exemplifying the importance of factoring in the space when exploring broader viewpoints to the user research questions, through informal social encounters, serendipitous interactions, and activities that are designed for cross-examining their results.


  • Its great to stand here a/er spending a year working for one person more or less [maternity leave] even though it was the greatest interac=on experience ever. I am interac=on designer by educa=on and major parts of my career. 1
  • 2
  • But I have done a fair bit of research along the way. Depending on who you talk to there are many terms used to describe the process that we go through to acquire the informa=on and insights on people. So today I would like to talk about what I like to remind myself in planning these types of research. Which I learned from my own experiences in the past, rather than what I learned in the book. 3
  • If you work in a rela=vely big organiza=on, the news about a new research project is analogous to that of a party. While you have the limited =me, money and resources, people are all the more eager to learn from your undertaking. Marke=ng may want to know what is the key marke=ng message about the product should be, the roadmap team wants to know the demographic informa=on of the most likely buyers and rejectors with quan=ta=ve evidences, strategy team wants to know how this product performs over the compe==on in what way, SW team and design teams want to know how to improve the product, and Finance team wants to know the op=mal pricing. 4
  • It is not this bad all the =me, but I did see a project geOng named Crystal ball as there were a lot of expecta=ons from this single study. I am sure people manage such situa=ons in various cunning ways, but when I faced situa=ons like this, I was very stressed partly because I was nave enough to think that I should try to cater for everyones needs. But soon enough you become wise enough that some combina=ons just wont work, or the complexity of the research increases to the extent that it becomes an impossible mission. 5
  • And what about people who par=cipate in your study? This is from a study I ran a couple of years ago for a brand new product we were developing, in prepara=on for the launch. With all things considered, 5 dierent ac=vi=es were needed, including warm-up. We had the 9 dierent features of this new product that we had to demonstrate to the par=cipants, get them to understand, form opinions over them, and ash out their own crea=ve expressions for the product at the end. What does that mean in terms of =me? 6
  • About 6 hours long. Which is more than twice long as the typical focus group sessions. Our EVP sent me a worrying email if this would be a valid method to get people engaged =ll the end. It is true that if you consider people as passive par=cipants answering simple ques=ons that most people wont be able to keep their spirits high for this long, without just going yeah, yeah, yeah. Having used co-crea=on methods several =mes before, I was condent it would work. But this is the point where the research planning becomes an interac=on design challenge: Keeping people engaged and intelligent while trying to cater for the expecta=ons from the various stakeholders. I always ask myself: are we truly learning the balanced view from the research? Is there any beder way to achieve the same? User research results for the early phase of product or concept 7
  • So here are a few points that I consider as reminder for myself in user research planning. First reminder is to ask yourself if you get to understand how people feel in rela=on to your product or service. It may sound like a common sense, but it can be easily overlooked. 8
  • I lived in India for 2 years, leading Nokia Research Center in Bangalore my team worked on a number of projects that were relevant to India. 9
  • One of the topics we picked up was the problem of na=ve languages use in digital media. India is an extremely mul=-lingual country. There are more than 1.2 billion people in India, using more than 122 languages. There are 22 ocial languages in India with scripts. There is no exact sta=s=cs but it is guessed that around 10% of popula=on is literate in English though the number must be on the rise. 10
  • Indian language scripts are very sophis=cated and have a very dierent wri=ng logic. When alphabets are combined to form a syllable, it typically changes the shape. 11
  • So there is a considerable dierence between handwri=ng and digital text input. But in order to be able to type on keyboard, user needs to have a fairly good memory of the alphabe=c table to make the combina=ons for the syllables. 12
  • And you may think that everyone remembers alphabets! I thought so as well. When we gave the task of wri=ng alphabets which is more than 50 more than half of par=cipants could not complete the task. Par=cipants were all educated at least 8 years. 13
  • It was worse on mobile phones one key typically needs to be assigned with 6-8 characters. This lady was our xer in Bareily, where we conducted our usability study. She is from the untouchable cast but university educated and could read and write English. But none of her social network did. And hindi in her phone was impossibly dicult to use. So as the result she never used tex=ng on mobile phone. So the reality that we saw around us was quite clear: People do not use tex=ng in the local language, period. What about Internet in general? If you are not able to read and write English, the world of Internet is a very small place indeed. 14
  • We took on familiarity as the important theme of the design development to minimize the learning curve and lower the barrier to adop=on. And touch screen because it was the future and to avoid the logis=cal and usability problems of physical keymat. 15
  • So we ran various tests with a wide range of par=cipants to improve the design from usability of the onscreen keypad to the language content itself. 16
  • While we were working on this, Nokia launched a product called Nokia C3 in 2010. It sold very well especially in India. However we heard that there were a lot of complaints about Qwerty with Hindi keymat print. 17
  • The product had the Inscript Hindi input, which is the government standard for qwerty keyboard. While the minority of consumers who were able to use Inscript welcomed the product, the majority found it annoying. On one hand, each key became too crowded and made it extra dicult to nd the key you want. But the real underlying reason seemed to be that buyers of this product do not want to be seen that they need Hindi. Implying that they can communicate in English, which is a thing to be proud of. We ended up recalling the product, replacing the keymat with just English. 18
  • Of course, a/er this we got a lot of ques=ons if it is worthy of inves=ng in na=ve language input tool. But as we have been asking our varied par=cipants to the study how this would change their life we were luckily well equipped for the answers, even though it was not a major ques=on when we were planning the usability research. There was a strong sen=ment that its a language that enable communica=on with their most close families, but the need to use it on mobile phone was marginal. We saw it as a chicken and egg problem. Literary professionals saw this giving a great educa=onal value. So we were able to pitch it to the product team that its one of the priority implementa=on for Asha touch product line. But we agreed that there would be no marke=ng around it. 19
  • Second reminder is honesty. Are you allowing people to express their real opinions? It may be honesty, but some=mes it can be about helping par=cipants to express themselves beder. 20
  • We ran a project with 30 farmers in rural India, from 2 separate communi=es. They were progressive farmers who were very open to trying new farming methods developed by the agricultural university. 21
  • We were tes=ng a simple app that connects farmers to a voice message box to ask their ques=ons to experts in the university called Kisan (which means farmer). Experts can access the recorded ques=ons, then publishes the answer in text through the app, which becomes visible to all par=cipants. 22
  • The trial went on for almost two months. There were several interviews along the way by our team and the university researchers. We were planning to host a joint workshop with all the par=cipants at the end to discuss and ideate how we can approach mobile informa=on system for rural communi=es. It was great to hear the posi=ve stories all farmers we talked to shared how they beneted from the Kisan and some suggested addi=onal categories to add. 23
  • I dont know what you imagine when you hear Indian farmers but my rst impression was that they were extremely polite. We were mostly met with a farmer who was in clean, crisp, white clothes. 24
  • When we visit the house the room was always prepared for us. And this farmer 25
  • prepared snacks for us when we visited his home all by himself as his wife and children were away. This put smiles on my face. But maybe it was my distorted personality I started to get worried that I was only hearing posi=ve feedback on the system. It was also because how farmers generally talked. They liked to talk on a very high level, and it was quite dicult to get to the details of the interac=on or any other minor issues. They were happy that they solved quite a few problems with Kisan prototype service that otherwise would have taken much more eorts from them to solve. 26
  • So the day before our workshop day, I made a special prepara=on. This was the venue of the workshop 27
  • Blue badges 28
  • And red badges. Folded and stapled in the hotel room the day before in a hurry. 29
  • Blue team member 30
  • Red team members. 31
  • We made a debate task in the workshop. Red and blue teams had to come up with arguments that supported the statement given in their color. This way, people could poten=ally raise the most nega=ve and bold opinions with necessary details without feeling socially unaccepted. / Each team was given 10 min to construct their argument to present for each statement. 32
  • So it started o fairly quiet. 33
  • But quickly the mood started to heat up 34
  • And the professor from the agricultural university had to come in the middle to mediate. 35
  • As you can see more than one person started to stand up wan=ng to speak 36
  • Our dear professor had to calm a few too excited farmers 37
  • We had very passionate speakers as well while the transla=on was a real challenge for me to catch up what was being said - I was almost feeling like this could be the atmosphere in an elec=on campaign here 38
  • At the end we asked people to vote for their real personal opinions. It was very clear that peoples opinions were quite divided indeed, but the debate ac=vated them to talk about barriers to adop=ons and dicul=es that they feel they will face if the service was real without any social s=gma that they were impolite to us or to university stas. For instance, while people wanted the informa=on system to connect them to a wider geographical area and possibly all over India the issue of languages came out high as barrier to cross-state communica=on. Availability of the mobile phone and the cost of using the service was another hot topic. 39
  • And we ended the day with a small gi/ and leder of apprecia=on giving ceremony. 40
  • Plus the obligatory group photo shoot. I was very happy to see the par=cipants having fun as they should and the day ended in high energy. But having fun, high energy in the session does not always warrant a good learning from the session. Honesty is a dicult one in user research. Your results may be deeply inuenced by it but you would never know. This may sound lame but you need to follow your gut ins=nct in judging the situa=on if you need further ac=ons to either make people feel relaxed about expressing their opinions, or help their expressions. 41
  • Third reminder is that some=mes it is benecial to plan specically to bring in unexpected insights that could fall way outside your radar. I would like to describe it as DIY research for the par=cipants with the minimum interven=on. This approach can be benecial especially when you dont know what you dont know. 42
  • We were studying how people outside the benets of major technology development several years ago before we established the internet to the next billion strategy with very aordable mobile phones. We chose 3 ci=es,
  • And 3 communi=es within those ci=es. There was very lidle known about their technology use among shanty town residents.
  • While the main team was busy doing contextual interviews, our fringe ac=vity had a simple mission: Make an open studio for the design compe==on. The theme was to design your ideal mobile phone. 45
  • So we partnered with local organiza=ons to run a mobile phone design compe==on in the community. Mumbai team came up with a slogan Design a phone, Get a phone. Despite the harsh rainy season, the team went around in all parts of Dharavi to hand out the informa=on.
  • In Mumbai, we hired a photo studio to be the hub
  • And in Rio we worked with a NGO that was doing computer aided design educa=on Rio team came up with a slogan T na hora de criar, seu telefone cellular, which translates in English: its about =me, to create your cellphone.
  • Local team came up with gra= wall paper, logo yers and even a song.
  • In Accra, within the Liberian refugee camp, we worked with an NGO oering computer courses. Buduburam team came up with a slogan Your dream phone, share it with the world. The promo=on was done through banners placed in the key areas of the camp and the radio sta=on adver=sement, which was very eec=ve. The local NGO called MOPGEL oered the space, which was normally used for computer courses. // The rst proposal was Refugees are human, Nokia is interested in their opinions and we had to turn it down as we didnt allow the word Nokia in the slogan.
  • Par=cipa=on was simple people can just go in to the studio space and ll in this entry form. The local team helped those who needed help with wri=ng and interviewed the par=cipants to make sure we understand the idea beyond what was wriden in the form. 51
  • To show you some examples what this brought: This 19-yo student/social worker wants to change the world, especially leading young people to the right direc=on. His idea is that all entertainment systems will be in one phone including a virtual blond psychologist wearing a bikini who can answer any doubts and keeps people updated about cultural events in town. This resonated well with the sen=ment in the community that urged adults to keep their children indoors, away from poten=ally bad inuences and violence on the street. And possibly the blond in bikini.
  • This is a phone that had a split screen so that he can easily no=ce calls from overseas. As a refugee, he relied on funds sent from his remote rela=ves in the US, so he was very keen not to miss any calls from them. 53
  • This idea from Dharavi is about being able to get the weather forecast by simply poin=ng the phone at the sky. It brought up a lot of debates among the jury members the local design students. Some argued that the weather informa=on is available already and there is no innova=on around it. But what won the argument was the fact that this idea was highly relevant for the local residents in the community. They had largely weather-dependent professions, and most people did not know how to access mobile internet to get the weather informa=on. So making the informa=on access as intui=ve as possible would be benecial and its actually technically feasible now.
  • Lots of ideas around mul=ple SIM card use as well. This Golden Mobile has two SIM slots, golden to be no=ced among the crowd, and got plasma charger. 55
  • This one is a star shaped mobile that can host 4 SIM cards. There were no mul=ple SIM card phones in the market except small Chinese vendors. There were repeated evidences like these entries that people wanted mul=ple SIM card phone so that they can save the communica=on cost, and mi=gate the frequent network problems. While we were feeling that it is one of the trends we should highlight, this submission put a nail to that need. 56
  • So we decided to invest a bit more on that topic while we were in Accra, interviewing various people on the topic of mul=ple SIMs. We even found a service that s=tched your SIM cards together. Photo: Those who use more than 1 mobile phone numbers & SIM card combining service operator shop, Buduburam (Ghana), Younghee Jung & Nokia. 2007
  • What was remarkable with a par=cipant like him was the incredible amount of =me they invested in submiOng their ideas. Some visited the studio several =mes, discussed ideas with their friends. It was possible because it was their own ini=a=ve and drive. There was of course rewards promised for winners, but I felt that the real mo=va=on was the recogni=on of their ability and the opportunity to be heard.
  • We planned the Open Studios to be a complementary research source as we were faced with communi=es that we did not have much informa=on on beforehand. Running parallel ac=vi=es while we were on the eld allowed us to have a broader view than what we ini=ally set to nd out but also allows us to cross reference informa=on as we were learning. And it is cri=cal to work with the real local people to make street surveys and Open Studios type of methods work. I like to call this type of open-ended ac=vi=es as scou=ng project. Its not to answer any ques=ons within your hypothesis, but to broaden your eyes and minds. And if you are lucky, you get to meet really interes=ng people to bring to deep dives or contextual interviews. 59
  • Fourth reminder is to always try to look at the forest over trees. I am currently working in a new func=on of product marke=ng that works with design and technical teams to conceptualize new products. So we are o/en in the posi=on to run the market research to test if the consumer value proposi=on holds in various markets. 60
  • To confess, the 6 hour long session was one of the research projects I ran for the marke=ng team. As it involved a real product to launch, there was a lot of pressure to answer very specic ques=ons such as coming up with consumer value proposi=ons with priori=zed list of key selling points, insights on pricing, and improvements to the product. 61
  • Par=cipants go through various ways to think about the product and its features seeing the demo 62
  • Various demonstra=ons 63
  • And making their own presenta=on about it 64
  • And we ask them to keep a workbook so that they can keep track of what they have gone through and we can keep track of what they have been thinking and how they put it in their own words. 65
  • And there is always a tempta=on to get a stat within the room with a simple method like ranking the votes. The challenge here is that ideas were introduced to people all at once with lidle real experiences to really understand what it means to them. It means that people can change their minds throughout the session. We try to probe it in various ways like individual ra=ng, ranking exercise, or just observing how people ask ques=ons and summarize the concept. But where do we put more weight on? How do we analyze what resonated most for the par=cipants? One of my favorite methods in such situa=ons is to employ an improvisa=onal ac=ng task. We asked the par=cipant that they could choose anyone in the room to be their imaginary friend. 66
  • (And of course I was chosen rst so beware that you may need to act as well!) 67
  • I brought here an example video. This ac=ng ac=vity was done a/er they went through the series of exercises to understand and rate the individual features of the product. This 20 year old university student chose our local colleague from Nokia oce to be her imaginary friend. --- The girls ra=ng of the experiences had very lidle to do with what she was saying in this video to our local colleague (who turned to be her temporary boyfriend in ac=on). Dual SIM and instant social updates were her selling points. My colleague tries to push and push but her two key points were very clear. Why did she not score these highly in the earlier exercises? And how do you conclude on the result? This is I say the reason why we are worthy of keeping our job as its not so bleeding obvious. Its a call of the researcher to be ready to analyze and extract insights from these seeming inconsistent outcome. If it were for marke=ng communica=on, you may infer the communica=on challenge to the brand new features that people are not yet familiar with how to name the experiences. And instead of following any of the numbers that seemed to have been produced from the study, I would follow your strategy and invest in how to make it more memorable. 68
  • Looking back at my early days of working as interac=on designer, I considered usability research and user research more like a scien=c work that needs to follow a strict protocol and make the condi=ons as equal as possible among par=cipants. But over the years I realized that it is an art AND a design challenge itself how you engage people. Also you need to be prepared to answer various ques=ons a/erwards that can fall outside your ini=al hypothesis. Ironically I dont always advocate user research. On the contrary, I have seen so many instances where what the teams needed was a strong vision and leader rather than a consumer research data to make the decision for them. So ask yourself if you really need user research, or more design explora=ons, asking your colleagues opinions, or a belief. 69
  • If used wisely user research is a good tool like doctors stethoscope. Its ul=mately up to you to decide what course of treatments will be needed. If you are a good, experienced doctor, you probably dont rely on this tool too much, and are able to tell a lot about the pa=ents symptom by just looking or talking to him or her. Thats the ul=mate art we acquire as we increase our experience in the domain. I have always worked in a big corporate environment so I do some=mes get the comment that we are lucky to be able to run user research at all. I know I have been lucky to choose certain research topics purely for explora=on, but we were never exempt from the =me and budget constraints either. User research does not have to be a big project that you set up formally with par=cipants you pay for. Some=mes I ask my colleagues, or their friends and family to come and try out our product or designs. If your users are not some thousands miles away speaking dierent languages, you have no excuse if your project really needs people to give feedback and inspire you. Its your aOtude to care, not a budget or =me that determines how much insights from people you can infuse your design with. I hope those of you who will be involved in user/design/consumer/market research in any way will make their journey a lidle more enjoyable and insigh{ul by being ready for open ques=ons. 70
  • 71