topic: design research

description; ?>

View this presentation on SlideShare.View more presentations from Arvind Venkataramani.

Pecha Kucha presentation given at the Ethnographic Praxis in Industry Conference (EPIC 2011).

Abstract: I explore opportunities for reframing research methods & practice by borrowing approaches from other disciplines. Along the way, I share examples of recent work that straddles boundaries, and compare existing ways of doing research that are tied into product development/organizational processes with the kinds suggested by the works.

Further, I raise the question of what the role of research in innovation is, approaching Don Norman's recent "design research doesn't lead to innovation" question from a completely different angle - one that is more hopeful and less a fallback to an explanation from technology. I use the conclusions to point towards reasons for collaboration with other disciplines in our professional orbit.

Notes & best practices from the discussion: thanks to everyone who participated and contributed - I merely collected and summed them up. Please share freely.

Problems & Dilemmas

  • How do we put more effort up-front, before the project ever begins?
  • There's usually a one-way flow of requests (needs/feature statements) from Marketing / Market Research to Product Development.
  • Should I be a product manager or a researcher? What's the balancing act? Is it possible to be both? ("Experience Engineer" as one possible resolution, at Intel ESP because "engineer" provides status equality with "Software Engineer"). What should the relationship between a researcher and a PM be?

Best practices

  1. Target your insights to a particular person
  2. Mine your client for information about their work, their challenges, their colleagues
  3. Make the stories defendable (to other disciplines; go beyond just providing qualitative evidence that is sufficient for our standards)
  4. Design trackability of research outputs into the design process; find ways to tie insights into performance metrics as a way to prove ROI of research
  5. Find the extra cubicle: developing temporary embeddedness to establish relationships with your client teams. Live with them, become one of them.
  6. Identity engineering: Shape your identity carefully based on who your client is (or who you're talking with). "Research" might be too closed a label, and it doesn't help people relate to what you know.
  7. Be humble enough to make the idea someone else's (when other people use your words without realising they're yours, that's how you know you've succeeded).
  8. Speak like your client, think like your client. (How? See #2 )
  9. Know your customer's cadence (their cycles of product development; know when to interject and involve yourself, and how)
  10. After presenting, talk about your client's work. How do the insights impact their work? (Take time to socialise the research).
  11. Identify the decision makers (both of research budgets as well as product management). Target your insights to them.
  12. Understand how the clients define success. (How? This is still a sticky problem that generates vague answers. Probably something you constantly should do throughout a research project)
  13. You don't make the company successful, you make the person who hired you successful. Know what they're trying to achieve.
  14. Create two kinds of outputs: a. the dog & pony show for impressing the client's clients, and b. actionable/immediate/business-relevant for the client.
  15. Find about client's work lives (but be an informed blank slate, so you can quickly ramp up the discussion if they think you should know those things already)
  16. When pitching / interviewing, look up the other person on LinkedIn.

   * More discussions at www.journalofbusinessanthropology.com (an Open Access journal).

Also posted on Google+.

Thanks to everyone who participated!

graph showing the drop-off in images uploaded to flickr taken with the iPhone 3g

From the Flickr camera finder, a graph showing the drop-off in the iPhone 3G as a source of images. Note for a while that the 3GS - the next newer model - paralleled but never exceeded the number of members uploading from the 3G, while the 4 exhibited a delayed but steady rise to where it's becoming the most popular model. It would appear that the 3G is being rapidly replaced in the Flickr user population.

Note that this data doesn't tell us anything about the larger population of smartphone users (and the subset of picture uploaders) - it's possible that images from these Apple and non-Apple phones are being uploaded as often, but to other services, or that if you don't own an Apple smartphone, you just don't upload to Flickr.

Still, as a way to track the death of a product, it's an interesting indicator. Just goes to show that there's lots of opportunities for web-services to exploit the informating capabilities of technology.

Next: find out where all those discarded smartphones are going (China? Poorer populations who just haven't discovered Flickr yet?) and look for evidence of use in corresponding web services (Kaixin/QQ? Myspace?)

By the way, one of my favorite questions to get the ball rolling (at least in the US) is: “what strikes you most about American life these days?” Try it out!

Nan Bress, in an email to the anthrodesign list.

The true/LOVE experiment was quite successful: we appear to have tapped into a vein of interest & emotion that we didn't realise existed. Apparently, when you ask people - in so many words - to share what they love about a place, they respond heartily. We were swamped by people practically the whole time (from 8.00pm to 12.30 am).

I'm still analysing the actual map that resulted, but the video above is a good discussion of why this sort of participatory, site-specific engagement works, and what design principles went into it.

More considered, analytic, and reflexive commentary to follow.

 

Can design research methods be used outside the confines of a project, to throw light on the design of technological/information systems? What can be learnt from an open-ended exploration, sampling events & sites instead of people?

Come by for an experiment: art as commentary on crowd sourcing, web 2.0, and measures of community interest. All of this without a single computer involved.

[with apologies to Victor Margolin]

Don Norman recently wrote an essay claiming that “design research is great when it comes to improving existing product categories but essentially useless when it comes to new, innovative breakthroughs”. Towards the end of the essay, he says "The inventors will invent, for that is what inventors do."

The central thesis of the essay is a list of inventions that changed human life: the airplane, the automobile, the telephone, the radio, the television, the computer, the personal computer, the internet, sms text messaging, the cellphone. Norman claims that the creation of these innovations (note the linguistic sleight-of-hand - we will return to it later) were not influenced in any way by design research or market research.

Big Straw Men

This is a straw man, for two reasons. The simpler one is that there wasn't any design research around at the time, and it is even now a discipline that's in its infancy and is learning its place in the world and in industry. The second and more important one is that that has never been design research's claim! Design research does not invent technologies: it merely points the way towards opportunities for doing, creating, serving, and making things. It does not even create the conditions for inventing new technologies, because innovation is a social process, firmly embedded in the exigencies of the corporation's structure, organizational culture, power struggles, competencies and finances. Design research is not a secret sauce for product success (even if some design researchers claim it is).

In addition, as an objective statement on the nature of product development, as Nicolas Nova points out, it is simply not true.

So the point of this essay is that only technologists can invent technologies that change the world, because inventing technologies is what technologists do? Let us, instead of focusing on the truth of this tautology, take a look at the subtext of the essay and the discussion surrounding it.

A Theory of Consumption

Consider Norman's statement about how the automobile changed human society. It did indeed, but how: cars&car-based ways of life destroy landscapes, create landfills, increase distance, decrease sociality, pollute, help bring about global warming, and are the most dangerous consumer technology invented, killing more people per year than anything else. Were automobiles brought about by design research? Nope. Did cars bring about important changes to human mobility? Sure. Could paying more attention to people's lives & the consequences of the proliferation of cars have changed the way this technology worked for the better? You bet.

Underlying this is a theory of needs & consumption. Norman says "Consider the cycle. First comes a new technology..." and later "... the technology launched the products. The products discovered needs. People slowly adopted them, leading to more changes in the products." Naturally, this perspective leads him to believe:

Where does design research fit into this cycle? Design research has many definitions, but within the product cycle, it consists of studies aiming to understand the activities, desires, and needs of the people for whom a product or service is desired. Design researchers use a wide variety of methods, but all of them, whether it be ethnographic observations, systematic probes, or even surveys, questionnaires, and focus groups aim at one thing: to determine those hidden, unspoken needs that will lead to a novel innovation and then to great success in the marketplace.

This is old hat. We, as a community, have matured beyond this perspective a while ago. (Norman appears to have missed the debate around "implications for design".) By framing design research in this narrow manner, he ignores all of the other ways design research can impact business: by reframing worldviews, discovering problems unrelated to product development & design that nevertheless impact those domains of people's lives, informing branding & messaging, and most importantly, changing corporate culture. (I'm certain I'm leaving out many other things).

This leads Norman to cite an example that actually counters his argument:

Why did the Macintosh almost fail? Was the world ready for the concept? Not really. Apple didn't help with its advertising campaign that snubbed business as dull, dreary, and not worthy of a Macintosh, yet business should not only have been Apple's biggest customer base, but families wanted to buy their children the same computer they would be using in business. As a result, a far inferior computer, the IBM PC, running a command-line, baroque operating system (MS-DOS), swept the market. Within Apple itself, the Macintosh caused huge internal disruption between the Lisa, Macintosh, and the Apple II groups. The Apple II was where Apple was making its money: the other groups were losing money. Internal politics? Massive. Interdivisional rivalry? Yup.

That is, one reason that the Mac almost failed was that it had a misguided marketing campaign that applied the wrong meanings to the product. (This is precisely the sort of thing that good market research or formative design research can uncover.)

The other explanation here is the 'ahead of their time' notion: which essentially functions as a euphemism for a good idea implemented insufficiently well (usually because the available materials were not good enough). While this is not the fault of the people who made the various products Norman cites as having failed due to this reason, it does not detract from the fact that they were either simply not good enough (read usable) or could not be turned into a successful business. If, as Norman says, the Apple Newton was ahead of its time, why didn't Apple simply start making the Newtons again soon as pen-based devices started appearing in the market?

Built on Abstractions

In fact, there are some other fundamental assumptions in his piece:
- Innovation is a new thing, an object, or a technology. (This is a rather narrow perspective, to put it mildly. Recall the linguistic switch between innovation & invention: this is what makes it possible.)
- If you do design research and find opportunities, innovation must happen. Innovation doesn't happen so often, so the design research claim must be false. (Completely ignoring the sociological truths of product development)
- The impact of a technological invention comes solely from the invention itself. (See 'technology giveth and technology taketh away' for one examination of the tortuous relationship between media technology & society. Or just learn a little bit about Twitter and how it's evolving.)

All of this paints a picture of the relationship between design, design research and technology based largely on a set of abstractions instead of the messy complexity of real life (much like Roger Martin claiming that businesspeople don't engage in abductive reasoning). To sum it up, it looks like this:

  • all the technologies that had massive impact were driven by 'technologists'
  • when technologies succeed hugely, it's because of their inherent qualities
  • when innovative technologies fail, it is because the world wasn't ready for them
  • 'needs' are created by technologies. (or, consumers are created by products)

This is a remarkably techno-centric worldview. The reactions to it seem to largely sidestep this issue, either trying to make a case for a seat at the table, pointing to the complexity of the process, or showing how all good invention uses the very activities of design research (while possibly stretching the definition of the term somewhat).

Upsets & Reframings

Contrast this with Norman's closing sashay “Technologists will... get the grand ideas running, but their implications are apt to be complex, overwhelming, and just plain horrid. Horrid applications? Yes, but that's good news: we will forever be indispensible.” Mr Norman wishes to be a purveyor of commodities. He has given up hopes of power: he wants to be a fixer, a second-class citizen in the glorious country of makers. And, by claiming to speak for the design research community, he endangers the community's ambition to bigger things. This is upsetting, naturally.

Or is it? As we have seen, there is little in this essay that is substantive (provocativeness alone does not a good argument make). Other than chorusing "We can, too!", there is little to be gained from responding to it, other than to perhaps limit misunderstanding arising out of extreme statements made by famous people.

What seems to me to the greater issue is of addressing the underlying techno-centricism of this worldview. In a year which has seen much havoc & pain caused by misbehaving social institutions (which seem to have arisen out of a similar pattern of belief in financial technologies), it is ironic to encounter writing and discussion around issues of participation in world-making that completely ignores everything else but the possibility of telling people what to make next. (I strongly suspect Don Norman still lives in the world of rockstar designers). Aren't there other ways to contribute? After all, as Jon Kolko argues, we're in this to make the world a better place.

Here, for instance are some questions to consider about our practice:

how do we participate in the life of the things we help create beyond their release cycles?
what are ways to tell clients not to make things?
how do we move from being product-focused to being organization-focused?
how do we increase our presence in industries that don't currently employ us?
how do we enable systemic changes?
how do we enable endeavours that need to happen across organizations for systemic changes to succeed?

design research 101: you don't have to look at all the data. given a compromise between examining and categorising every transcript, photo, answer or looking at the data in different ways, pick manipulation & representation over organization every time.

Having lots of data points doesn't mean you have greater understanding: it simply means you have lots of data points and more chances to prove a statement true or false. Having fully organized data is great if you want to test conclusions, but it's easier to check hypotheses than to find a new, more powerful, more insightful way of looking at the phenomenon being studied. Finding a good explanation for behavior usually involves multiple rounds of trying to make sense of something. That takes time.

If you're short on time, forget trying to organize everything and spend more time interpreting and deriving conclusions from the data. After all, that's what you were hired for, isn't it?

a newspaper dispenser for the Columbus Dispatch, outside a Subway a newspaper dispenser for the Columbus Dispatch, outside a Subway

Here's an example of one my favourite ethnographic analytic moves: 'unpacking'. Unpacking is one kind of interpretation, where an observation is examined for unstated (or implicit) assumptions, consequences, and meanings. Here's how it works.

Consider the above image: a newspaper dispenser outside a subway. Here's what we're going to assume about it:

  1. It is intended to sell newspapers
  2. It is designed to support this intention

Now, notice the statement on the bottom of the dispenser: "We capture history every day". Unpacking this involves finding premises that give this statement meaning:

  1. "History" happens every day
  2. Some things are Historic and some things are not
  3. History can be identified as it happens
  4. "Capturing" history - by identifying what is Historic and what is not - is a non-trivial task
  5. the Dispatch performs this non-trivial task every day

Taken together, the premises make an argument for how value inheres in the Dispatch, and why you should buy it. Note that in order to accept the statement, you have to accept the premises, but in order to unpack, you only have to notice something and wonder what makes that thing work. What, one asks, has to be true to make this true, to make this believable? The insight is in the unpacking, the discovery of assumptions.

The insights here have to do with the claims being made about the nature of history: if one wanted, say, to counter-advertise, one could devise an advertising strategy designed around these claims. For instance, one could reverse the time-orientation and claim value in reporting and making sense of the present, and pointing towards the future.

But unpacking doesn't happen in a vacuum: in order to perform this particular instance of unpacking, I had to have cultural knowledge - (that the statement is an advertisement, that advertisements are stories that help sell something, and so on, that advertisements can appear on newspaper dispenser). This is what makes unpacking a particularly ethnographic move: after all, that cultural knowledge had to be acquired somehow. In this instance, advertisements & newspapers are common enough and shared enough that I could do this analysis without having to do research (although you could count my living 5 years in the US as a process of gaining cultural sensitivity; just growing up as a city-bred person counts too, I guess).

In cases where the things being noticed are in less public or familiar contexts - a McDonald's in India, or on the shop floor of an oil rig, or in a hospital - cultural, contextual, and procedural knowledge has to be learned, and often the quality of the learning depends on the amount of exposure one has to that context (what we call 'research'). This is why anthropologists and ethnographers prefer longitudinal participation - the longer one stays in a place, the more one learns, the more things one can unpack and find the premises of, the more powerful the insights. This is also why the ethnographic method has little to do with mere observation alone - you have to do a lot more than that to interpret and find meaning.

Personas are composite constructions that serve two distinct purposes: to act as aids for design decisions, and to generate empathy. The two can easily get in each other's way: people looking for information that they can act on might consider all the additional texture of people's lives "fluff", whereas people for whom a persona doesn't represent a specific set of information but, rather, a set of contexts and behaviours might be disappointed if the persona had "just the facts, ma'am".

Within corporations, personas are an intensely political artifact: claims on what they should contain are made by many different groups of people: marketing, product planning/management, designers, researchers, engineers.. Each has their own set of questions they want a persona to answer, and each has their own claims on what the persona means. Our poor hardworking persona, then, has to act both as an embodiment of consensus and as a coordinating artifact - something that various kinds of people can look to for guidance as they go about their work.

Regardless, personas are usually composites: rarely in research are personas constructed from one perfect respondent - they are amalgams of many stories and many situations. This is (hopefully) not the case for personas that are entirely behavioural, and who are being constructed for a highly specific activity or around a particular product; the necessity for composition arises when generating personas for a wide variety of activities or products, and especially the case when personas are created as part of generative research, which tend to include attitudinal and aspirational data (to use marketing terminology) in addition to behavioural data. The point is to create a person who embodies the entire range of behaviours and attitudes for that class of people - so that the resulting design covers as much range as possible.

It is at this point that something like the following occurs:

"Does he have a car?"
"Uh, I don't know. I mean, he has an iPod.. isn't this the persona who doesn't like CDs because he wants access to his music all the time?"
"So, ok. He has a car, and he listens to his iPod in it and his FM connector keeps getting interference but his car stereo is old so he has no other way of connecting it"
"Yeah, that sounds about right"

If you're a ux/design researcher you've probably had a conversation very much like this. This is the act of composition: when the essential stories & features have been picked, and they have to be woven into a single story. In this sense, personas are a collection of fictional facts: the stories come from different places, but they have to be made part of this person's life, reflecting their desires, their interests, the conditions of their life and so on - and in the process are given a fictional form. At all times the data speak, but through the voice of this character. In this sense, defining personas seems remarkably similar to writing a novel...

To see this, we turn to the most excellent Umberto Eco (and it is worth reading in full):

What I mean is that to tell a story you must first of all construct a world, furnished as much as possible, down to the slightest details. If I were to construct a river, I would need two banks; and if on the left bank I put a fisherman, and if I were to give this fishermana wrathful character and a police record, then I could start writing, translating into words everything that would inevitably happen. What does a fisherman do? He fishes (and thence a whole sequence of actions, more or less obligatory). And then what happens? Either the fish are biting or they are not. If they bite, the fisherman catches and then goes home happy. End of story. If there are no fish, since he is a wrathful type he will perhaps become angry. Perhaps he will break his fishing rod. This is not much; still, it is already a sketch. But there is an Indian proverb that goes, "Sit on the bank of a river and wait; your enemy's corpse will soon float by." And what if were a corpse were to come down the stream – since this possibility is inherent in an intertextual area like a river? We must also bear in mind that my fisherman has a police record. Will he want to risk trouble? What will he do? Will he run away and pretend not to have seen the corpse? Will he feel vulnerable, because this, after all, is the corpse of the man he hated? Wrathful as he is, will he fly into a rage because he was not able to wreak personally his longed-for vengeance? As you see, as soon as one's invented world has been furnished just a little, there is already the beginning of a story.

Eco, U. (1994). The novel as cosmological event. In The Name of the Rose (1st ed., pp. 512-515). San Diego: Harcourt Brace.

Exactly. Of course, if you're creating a persona you're working off of data, and the construction of the world is not random but is informed by your data (though there may be a few judgement calls along the way for clarity's sake, or to include perspectives). Importantly, the more constraints (user stories, situations) you add, the more formed your story (persona) gets.

Eco continues:

The problem is to construct the world: the words will practically come on their own. Rem tene, verba sequentur: grasp the subject, and the words will follow

Indeed.

Syndicate content