dan russell: good afternoon,everybody. welcome to this episodeof authors at google. i'm dan russell. and i'm the host for our guesttoday, jeff johnson. and i know jeff because back inthe mid '80s we were both at an institutionknown a xerox. i was at parc and jeff was atoffice information systems. that means, he got to work onthe xerox star which if you go down to the computer historymuseum, you can find.
it was a fantastic,groundbreaking system that commercialized a lot of theideas that later showed up in things like, oh, themacintosh, and other kinds of systems. you've actually used-- may or may not know it-- you'veactually used a lot of the work that jeff hasdone over the years. he's also not just a userexperience person, but he programs in java, mesa-- doesanybody remember mesa--
c++, even things like apl. so he's a remarkable guy who'sprobably best known for his bloopers series of books. and you can come and check outsome of them up here later. the one i first saw was "guibloopers." but then there were "web bloopers," and "webbloopers 2.0." which are sort of catalogs of all the mistakesthat we make and we should learn from. so these are really interestingbooks if you are
all interested in userexperience design. so jeff's talk is a part of thegrowing user experience education effort that'sbeing led by the infrastructure team. and we're going to be havingmore tech talks in this series so please stay tuned. if you like this one, we'regoing to have more. and you'll enjoy yourparticipation in that. also notice that we'recoordinating--
well cindy, who's sitting backover there with her hand up is coordinating a workshopand tutorial here-- well somewhere hereat google-- on february 7 where jeff andaustin henderson will be talking about returning thenotion of the conceptual design models. that will be open forregistration on glearn i guess in a week or so,real soon now. and there's spacefor 50 googlers.
so sign up soon ifyou like that. jeff's book will be availablein the back, $15, cash or credit card. so if you like his talk, you'regoing to love his book. so please join me in welcomingjeff johnson to talk to us about how to design withminds in mind. jeff. jeff johnson: thank you. so thanks to cindy yepez androbin jeffries for arranging
this talk, and to danfor hosting it. how many of you designuser interfaces? good number. goo. well those of you who do that,how many of you ever looked at or seen user interface designguidelines like the ones i'm displaying now? these were developed by andwritten, published by ben shneiderman in 1987.
and all subsequent issues of hisbook up to now remain much more or less the same. how many of you have seenguidelines like these when you're designing userinterfaces? ok so now we'll look at somesimilar guidelines that were published by nielsonand molich. they were intending these to beused for website design and website evaluation. now i'm not going to go throughthe guidelines in
detail, because that's notthe subject of this talk. but i will point out thatthere's similarity between these guidelines and those thathave been put forward by ben shneiderman. so the question is, did nielsonand molich plagiarize the earlier guidelinesby ben schneiderman? and then subsequently, diddebbie stone and her colleagues-- in the book that they publishedin 2005, has a
similar set of guidelines-- did they plagiarize? what was the reason that theseguidelines were much the same in these differentpublications? well i'm going to argue that thereason is not plagiarism. the reason is that userinterface guidelines are based on us. they're based on how we work. that's where those guidelinescome from.
now i used to be in theguidelines business, or actually anti guidelines. so when i wrote "gui bloopers"and its sequels, i was sort of in the, don't do this. do this instead. but i didn't really explainit in those books why. and i found that user interfacedesigners typically are not satisfied with justbeing told, thou shalt. or even, thou shalt not.
they like to knowwhy thou shalt. why shalt thou? and so the other problem is thatit's difficult to follow those guidelines ifyou're a designer. let's go back and lookat some of those guidelines for a second. first step to goalshould be clear. what does that mean? that's pretty vague.
first step is fairly clear. but what does it mean for thefirst step should be clear? what is clear? what makes something clear? when is something not clear? so the point is that guidelinesare not rote recipes that you canjust follow without any background knowledge. applying user interfaceguidelines effectively
requires understanding theirscientific basis. understanding where the rulescame from so that you can balance the trade offsthat occur when there are competing rules. if somebody says toyou, make it fast. make it high-res. well maybe you can't doboth of those things. so you have to choose oryou have to find some kind of trade off.
it helps to understand where therules came from in order to be able to apply them. so i wrote this book thatwas about that. that's basically about how thehuman perceptual cognitive processor works, or i shouldsay processors-- i'll get to that later. how it works and where do thesedesign rules come from, that's what the book "designingwith the mind in min" is about.
books have chapters. so some of the chaptersdeal with perception. i'm listing here on this slidethe perception chapters. and then there are somechapters on cognition. i'm not going to talk abouteverything that's in the book. it takes about two days to givea tutorial that covers the entire book. and we have an hour. so i'm just going to cover a fewof the facts about human
perception and cognitionin this talk, and show you some examples. so the first one is that weperceive what we expect. what you are perceivingright now is not what is really here. what you're perceivingright now is biased heavily by many things. the main things that i'm goingto talk about right that bias your perception are the past,the present, and the future.
the past is your experience. the present is the currentcontext, everything else that's going on right nowin this situation. and the future is your goals. what are you planning to do? what are you trying to do? all of those things heavilybias your perception. your perception is almost in noway reflective of what is actually out therein the world.
i'll put it that strongly. so i'm going to showyou some examples. how many of you arefamiliar with the art of jackson pollock? so here's jackson pollock. he's landed a great gig. he gets money for throwingpaint onto canvases, splattering paint on canvases. what a great gig, i wish i couldget a gig like that.
ok so i'm going to show youa jackson pollock image, a jackson pollock painting. everyone see the splatterpainting? ok so now i'm going to showyou something else. i'm going to show youa dog sniffing the ground next to a tree. it's a dalmatian. does everyone seethe dalmatian? so there's the dog.
there's the tree. he's sniffing the ground. his tail's in the air, hindfeet, front feet. this is actually a famous imageby a guy named rc james. i could basically tell you whatyou were going to see by showing you pictures of jacksonpollock painting and things like that. and then telling youyou were going to see a splatter painting.
so basically the past influencedyour perception. now i've told you it wasa dalmatian, and then i showed it to you. and how many see the dalmatianby the way? it's about everyone. and now the interesting thingis, if you ever see this picture again you'll never notsee the dalmatian, ever again. so what does this have to dowith computer system design? well let's assume that we havea multi-page dialogue box,
otherwise known as a wizard. so it's a multi-stepdialogue box. it doesn't matter what'son the pages. so if you watch someone using awizard and a usability test, what you'll see is fairlypredictable. so basically they read what'son page one, whatever it is. they fill out what whateverfields are there. and they click next. and it goes to page two.
and so then they read onwhatever's on page two. and they fill out any fields. and they click next andit goes to page three. and then they fill it out, andthen they go to page four. and then they clickthe back button. and it goes backto page three. and they say, why did it dothat, i clicked next? and you say, no i was watchingyou, you clicked back. and they say, no i didn'ti clicked next.
and you say no, i was payingattention to what you were doing and you clickedthe back button. and then you takethe page four. and they, oh, the back button isin the opposite place than it was on all theother buttons. of course they didn't noticethat because they weren't paying attention to the positionof the buttons. they were paying attention totheir task of whatever it was, buying an airline ticket,reserving a hotel room,
whatever it was. so their perception was based ontheir past experience, not only with this dialog box, pagesone, two, and three, but with all previous wizards thatthey've used before. so that biased theirperception. now let's talk about currentcontext biasing perception. here we have two symbols. they're identical. they're the same symbol.
but depending on what context iput them in, your eyes will perceive them, or moreaccurately your brain, will perceive them as an h that'skind of crooked or an a that's open at the top,automatically. or let's move away from thedomain of letters to the domain of graphics. so we have a line here. this is called the muller-lyerillusion. so we can take that lineand duplicate it.
so now there are two of them,exactly the same line. and then we put these on themand now even though you know that it's the same line, youreye sees the top one as longer than the bottom one. and you can't actuallystop yourself from seeing it that way. so this is context affectingperception. now here's the interesting thingis that the brain is multi modal.
as you know you havemany senses. and the context canbe cross model. the content influence, thebiasing of perception by sensory input can be from othersensory modalities. so there's a famous effectcalled the mcgurk effect, which i'm just now goingto show you. which is that based on whatyou're looking at-- by the way, let me ask you thisquestion, how many of you can lip read?
well in fact you all can lipread as you're about to see. now i want you to listen towhat this man is saying. and i have to give credit tosome unidentified blogger on youtube who put thisvideo on youtube. it's probably the bestdemonstration of this that i've seen and it'sunidentified. so i don't know who to credit. but i want you to watchthis video. male speaker: bah bah.
bah bah. jeff johnson: he's sayingthe same sound. jeff johnson: you hear adifferent sound depending on which guy you look at all. jeff johnson: that's an exampleof your hearing being biased by what you're seeing. you can actually all lip readand you didn't know it. and as i said earlier, thefuture also biases your perception, your goals, whatyou're intending to do.
people, especially adults, tendnot to notice things that are unrelated to their goals. our mind is designed to noticethe things that are related to what we're trying to do. so i'm going to show you thecontents of a toolbox. and i want you to tell me ifthere are scissors in the toolbox, ok? were there scissors? ok.
was there a wrench? some people saw the wrench. so there is a game youcan play in your house if you have guests. i'll describe it quickly. which is if you're sitting inyour living room and your guests are with you in theliving room, you can send them into the kitchen. and probably in your kitchensomewhere you have a drawer
that has random kitchenstuff in it. i know i do. and you can tell them,go to that drawer. it's the third one down fromthe left next to the refrigerator. and get me the turkey baster. and bring it back. and then when they come back,you say, was there a meat thermometer in the drawer?
and they will have no idea. most adults will have no idea. a six-year-old actually willprobably have a pretty good idea of all the other stuff inthe drawer if you can get them to come back in the first placebecause of all the cool stuff that was in the drawer. but that's because their brainisn't quite as goal directed as the adult's is. so in terms of website designfor example, if i put you on
the home page of theuniversity of canterbury's website. and i say i want you to find mesomewhere in this website a map of the campus that showsthe computer science department and where it is, yourbehavior at that point is very predictable. what will happen is, your eyeswill start moving randomly around the screenvery quickly. your eyes move 10 times asecond, tenths of a second per
saccadic eye movement. your eyes are moving veryquickly all the time in almost random motion. but because i gave you the goalof looking for a map of the computer science departmenton this website, your eyes will spend a lot oftime over these places, where it says departments, the worddepartments, the campus map. and maybe also your eyes willzero in on the search box because you're starting tothink about maybe typing
something in there and lookingfor search box. but you will probably exit thepage without ever noticing that you've been randomlyselected to win $100. because nobody asked youto look for that. now i'm going to talkabout color vision. the human color vision systemis very unlike a camera. a digital camera is depictedby this graph on the right. so on the right essentially whatwe have is the spectrum across the bottomof the graph.
and what we have is,in a camera-- we have on the plate at theback of the camera. we have a bunch of photosensitive unit. cells you can call them. and a pixel is madeup of three cells. one that you would calla red sensor. one you call a green sensor. and one you calla blue sensor. the red sensors are sensitiveto light in a certain range.
the green sensors are sensitiveto light in a certain range. and the blue havea certain range. and notice those ranges overlapa little bit and the amplitudes or the maximumsensitivity of the three kinds of sensors are approximatelythe same. it's designed that way. that isn't how the human eyeworks at all in terms of color perception.
the human eye has-- well let me ask youthis question. how many of you remember fromhigh school or college that we have rods and cones and thecones are for our color perception and the rods are forblack and white and sort of brightness? how many of you remember that? well you can sortof forget it. because it's only partly true.
the rods really we don't usethem very much in modern industrial society. they were designed foruse when we lived mostly in the dark. and let me ask you this, howlong ago was it that people lived their lives mostlyin the dark unless it was daytime? about 100 years ago. it was only about100 years ago.
so our ancestors, and all ofthe animal ancestors before them, basically had these rodswhich were designed to help us see in situationsof low light. but in the modern world in whichwe live, where it's lit most of the time even atnight, our rods are completely maxed out. so they're screaming all thetime providing very much little information. you rods in your head rightnow, in your eyes, are
probably just going, ahhhh, likethat and not giving you much useful information. so let's forget about them forthe rest of this talk. basically what happensis you've got cones. they're called red, green,and blue by biologists. but that's actually sortof misleading. because the red sensors aresensitive over almost the entire range of what wecall visible light. the green ones are alsosensitive over almost the
entire range, but not assensitive to light overall as the red ones. and the blue ones are hardlysensitive to light at all, for those people whohave blue ones. some people havedefective ones. so the human visual systemdoes not work the way a camera works. a camera visual systemessentially works by addition. every pixel is the sum of thered, green, and blue response
to the light that's hittingthat pixel. that's the rgb value thatyou're familiar with. the human visual systemcan't work that way. because how could it work thatway if the sensitivity ranges of the three kinds of conesoverlap so much? it could not work by addition. so it doesn't. it works by subtraction. so there are basically threechannels of information coming
from your cones to your head. one of them is yellow minusscreen, or green minus yellow. i can't rememberexactly which. and the other oneis blue minus-- sorry one is red minusgreen and the other one's blue minus yellow. and then there's a third one,which is such a complicated function of additions andsubtractions that produces brightness from the cones.
so the cones are producing thebrightness information as well as the color information. and they are doing it bysubtraction not addition. which makes our visual systemalmost insensitive to absolute brightness levels. so if i take this illustrationwhich a college classmate of mine edward adelson produced. and you see the checkerboard. can you see the there arecertain squares in the
checkerboard marked a and b? can you see that? what if i were to tell you thatthe a square is the same shade of gray as the b square? it's the same shade. so i'll demonstrate that to youby taking a piece out of each one with photoshopand putting it over there on the right. now but you still don'tbelieve me.
you think i cheated. so i'll take the image. blow it up. and then i'll cover up that. cover up that piece, that piece,that piece, that piece, that piece, and that piece. so the human eye is really notdesigned for detecting absolute brightness levels. it's designed for detectingdifferences, edges, changes.
that's what it's designed for. so what that means is thathumans have trouble discriminating certainkinds of color pairs. so for example if two colors areboth pale, people may not be able to see the differencebetween them depending on exactly how theircones function. and also by the way on the kindof display that they're looking at. like for example i'mlooking at this on
my macintosh screen. and you're looking at it ona projection display. and then there's alsothese monitors here. and all of those can influencehow you see the two colors and whether you tell the differencebetween them. so all you color designers outthere, you graphic designers who think you're in totalcontrol of the color that your users are seeing, forget it. you are not in control.
their eyes are in control. and the displays that they'reusing are in control. similarly, people have troublediscriminating color patches that are small. so if you have two color patchesthat are very small and the colors are close to eachother people aren't going to be able to tellthe difference. but if you make the same colorpatches larger, it'll be easier for people totell the difference
between those two colors. so what does that say if you'remaking a graph and you're putting the legend overon the right or underneath the graph with little dots that showyou the different colors of the different lines? make those legend dots big. don't make them littletiny squares. because people won'tbe able see the difference between them.
comparing colors is alsoeasier when patches are together than when they'reseparated, especially if eye movement is involved. because if i'm eye movement isinvolved in the comparison, then memory is involvedin the comparison. so if i move these two togetherit's easier to tell that they're different thanif they're separated. so if we look at itn.net,which is a website for ordering airplane tickets andhotel reservations, and we see
that they have said that they'reshowing me what step of their process they're on. and they're trying to show methat they're on step one. but they're using pale yellowto mark the steps. so we're on step one. now we're on step two. can you see that change? some of you can probablysee it and some of you probably can't.
and it also will depend on thedisplay that you're using. some people have colorblindness. so about 8% of males and abouta 0.5% of females have some kind of color visiondeficiency. now color blindness does notmean people can't see colors. the term color blindnessis misleading. what color blindness means isthat there are certain pairs of colors that people cannotjust tell apart. so for example a friend of minewho's red green color
blind cannot tellthe difference between these two lines. to him they look the same. now just out of curiosity isthere anyone in this room for whom these two lineslook the same? we're looking for the8% number here. ok so there's one. now think about that. think about that for a second.
if you create a graphin which different-- let's say it's stock values overtime, apple, google, and various other stocks-- and the lines cross. it's not just that the personis not going to be able to tell which one is which. wherever they cross they won'teven know which one goes on from that point. so it's really a seriousissue for people
who are color blind. so you really haveto be careful. my friend who is colorblind alsocan't tell the difference now he's classified asred green colorblind. but this line i wouldcall blue. the other one i wouldcall purple. but he still can't tellthe difference and this one, one is blackor dark brown and the other one is red.
he can't tell the differencebetween either of these two pairs. i don't want you to go out ofthis room and say, jeff johnson says don't use colorin our user interfaces. because that wouldbe pretty boring. what i'm saying is don't relysolely on color to convey information. if you're conveying informationwith color, use other cues that are redundantwith that.
so for example, i would reallyimprove itn.net's display by in addition to using a brighter,more saturated yellow to mark the current step,i would also bold the box and the wordand the number. now how they improved it withoutactually asking me. they never consulted withme unfortunately. but that's fine. they improved it this way,by making the step-- sort of inverting its color andputting the sort of blue
area around it. now that's just as good. so basically people are goingto be able to, regardless of what their color vision problemis or what kind of monitor they're using, they'regoing to be able to tell what step they're on. now peripheral vision. human peripheral visionis very bad. it's very poor.
there's an error messageon this screen. does everyone see it? it's right there. this is an actual screen, alogin screen, of a web app that a client companyof mine created. this is an actual screen. and that's where the errormessages were displayed. and so in usability testinghere's what we saw. user types in id.
they type in pin number. and in some cases we gave themincorrect ids and pin numbers, so that they would getan error message. and what happens is the usertypes the id, the pin number. and they hit login. and the screen re-displays. and the fields are blank. and the user goes, what? why did the screen re-display?
why are the fields blank? i don't get it. i must have hit cancel. so they fill in the id again,which is still wrong and the pin number or one ofthe two is wrong. and the screen re-displaysagain. and they go, huh? am i nuts? that time i was payingattention.
that time i know i hitthe login button. what is going on here? oh, why didn't i see thaterror message before? here's why. there are actually three reasonswhy they didn't see that error message before. and they all combined to makeperipheral vision of humans, and actually almost allmammals, horrible. reason number one is that whatwe're seeing now in this graph
is the distribution of rods andcones across the retina. now as i said earlier,we can pretty much forget about the rods. but you can see that there'snon in the middle. there are no rodsin the middle. and then a little bit awayfrom the middle, they are pretty dense. and then the density falls offas you get to the periphery. cones on the other hand, arevery dense in the middle.
that is to say there are 158,000cone cells per square millimeter in your fovea,in the center of your visual field. now to give you an idea how biga visual field is i want everyone to hold their armout at arm's length and hold your thumb up. and look at your thumbnail. look directly atyour thumbnail. your thumbnail spans the size ofyour fovea on your retina.
and the rest is periphery. everything else is periphery. so your thumbnail is 1%. we've heard a lot recentlyin the news about the 99% and the 1%. here's another one. the 1% of your visual field inthe middle when you're looking directly at something. the 1% is the fovea.
and that's high-res. everything else is low-res. and there are three reasons why everything else is low-res. reason one is the density of thecone cells in the middle is 158,000 per squaremillimeter. and in the periphery, not veryfar from the center, it falls off to 9,000 per squaremillimeter. now 9,000 is a lot persquare millimeter,
but it's not 158,000. that make sense? that's reason one, high densitypixels in the middle, low density pixelseverywhere else. the other reason is that in yourretina, every cone cell in the fovea sends a fiber outto the optic nerve that goes back to your visual cortex. every nerve cell in thefovea sends one fiber. the optic nerve that leavesyour eye that goes back to
your visual cortex is a cablewith millions of fibers in it. every cell in the middle ofyour visual field, in that middle 1% sends a fiber out. in the periphery, which iseverything else on the other 99% of your retina, three orfour cone cells combine to send one fiber. so what does that mean incomputer geek speak? it means the data coming fromthe middle is uncompressed. and it's compressed with dataloss everywhere else.
so you can think of itas you got a tiff in that middle 1%, high-res. and you've got a jpeg everywhereelse, low-res jpeg. and reason number three is thatwhen you get back to the visual cortex at the back of thebrain, which is called the occipital cortex,there's a map. the occipital cortex formsa map of the retina. but half of that map is devotedto processing from the 1% fovea area and the otherhalf is for the rest.
so think about it that way. your brain, your eyes-- by the way biologists consideryour eyes to be part of your brain, but anyway. your visual system is setup unlike a camera in almost every way. because a camera has its pixelsdistributed evenly across its plate. the human eye has highresolution in the middle and
low resolution almosteverywhere else. so let me tell you, theresolution of the human eye at the fovea is approximately 300dots per inch at arm's length. if i print something on yourthumbnail at 300 dots per inch, if you have normal vision,you can see the dots. what is the resolution of yourvisual field at the edge? guesses? how many dots per inch? it's measured indots per foot.
the answer is three. the effective size of a pixelat the edge of your visual field is approximatelythe size of a cabbage at arm's length. now everyone is rolling theireyes and looking around the room and saying i see everythingin high-res. your eyes move threetimes a second. you can't control it. it takes a tenth of a secondfor an eye movement.
and by the way your visionshuts off during those saccadic eye movements. but you don't see theworld in blinks. here's an interesting exerciseyou can do later. which is, stand in frontof a mirror. put your face righton the mirror. and look at your left eye. then look at your right eye. then look at your left eye.
you will never seeyour eyes move. but somebody standing therewatching you will see your eyes move back and forth. but you will not. what you will see is, you'relooking at your left eye. you're looking at right eye withno time in the middle. that's because during saccadiceye movements the brain shuts down. but it also is takes those twopieces of video and stitches
them together in time. so you never notice the littleblack intervals. so if we look at this websiteof airborne.com, this login page, one of the things we seeis that there's an error message here in red overon the upper left underneath the title. but it turns out people alsohave trouble seeing that error message, because they presseda login button. well now we know why.
because if we show what theeye sees when a person has clicked the login button beforethey move their eye at all, before the eye moves. this is what they see. this is what the eye sees at themoment the login button is clicked, before the eyemoves anywhere else. and remember eye movementis somewhat random. it is actually guided by goalsand many other things. but one of the things that canmake it move somewhere is
movement or change. and the problem is there wassome red stuff in the upper left before and now there'sstill some red stuff. so the eye doesn't reallysee a change. so there's no reason for theeye to actually move to the upper left. therefore it's not going tosee that error message. or at least there's aprobability that the eye will not see that error message.
i think that the probabilitythat the eye will see this error message is higher than itwas in that previous one. but it's still not guaranteed. and of course this is why youget in usability test-- programmers in the back roomwatching the usability test and saying, what'swrong with you? don't you see it? it's right thereon the screen. no because they're human.
they don't see it. because their eye was notgiven a reason to move in that direction. so in terms of user interfacedesign, how do you make sure that error messagescan be seen, or information can be seen? well put it where usersare looking if you can predict that. put it near the errorif there's an error.
use red for errors. or use an error symbol, likeone of these error symbols. now here's an example of that. so for example at aol.com, ifyou sign up basically for their service there'sa long form. i haven't shown youthe whole form. you go down to fill out allthe stuff on the form. and then you clicked done. and if you've got a passwordthat they don't like for some
reason, they willsend you back. they will scroll you back tothis place on the page. they put an errorsymbol there. they tell you how goodyour password was. they put an error message,which is in red. they highlight thefield you're in. and they put the cursor there. you are not goingto miss that. now there is some heavyartillery that you can use to
make sure people are going tosee error messages or other kinds of information. but you have to useit carefully. you can pop up things anderror dialog boxes in front of the user. some people hate that. other people turn off popups in their browser. so there's certain situationsin which that's not a useful thing to do.
there's audio you can beep. but of course imagine a roomlike this where everyone's got their computer and they'reall beeping. whose is beeping? i can't tell. so audio is limited. then there's flashing orwiggling brief briefly but not continuously. if you flash or wiggleconstantly, some people will
get epileptic seizures and otherpeople will not look at it because they assumeit's a ad. so for example, this issomething you're going to see more and more of in the future,and you are starting to see more and more ofit in various places. which is if you want to getsomebody's attention with an error message you wiggle it. that error message went onepixel up, one pixel down, one pick left, one pixel right,and then stopped.
stopping is very important. that will attract your eye. why? because it might be a leopard. right? your eye does not knowwhat that is. it just knows that there'smovement in the periphery. so it yanks the fovea rightover there because it might be a leopard.
so blinking is a poorsubstitution for wiggling if you can't wiggle. but it's better than nothing. but again the importantthing is to stop. ok i'm going to talk aboutcognition now. i'm going to switch gears andtalk about cognition. short term versus longterm memory. when i was in graduate schoolcognitive psychologists believed there were places inthe brain where they was short
term memory and there was otherplaces the brain where there was long term memory. and different brain structuresaccounted for them. and they had models of shortterm memory and models of long term memory. that's not the view anymore, nowthat people can actually study the brain inoperation using magnetic resonance imagery. now we know that short termand long term memory are
actually just twocharacteristics of the same memory system operating. and short term memory is nowmuch more closely tied and interpreted as being tied toperception and attention than it was in my graduateschool days. so short term memory is arepresentation of your conscious mind. it's what you're attendingto right now. so short term memoryis not a place.
it's not a place where things gofrom perception and things are hauled in fromlong term memory. it's not like an accumulatorin a computer. that's not what shortterm memory is. short term memoryis the stuff-- think of it this way. your memory system isthis huge warehouse. and everything in thereis old and dusty and covered with cobwebs.
and occasionally somethingnew is shoved in through the doors. and while it's being shovedinto the doors it's illuminated becausethe door is open. and then the door slams shutand the stuff comes in. and it goes dark just likeeverything else. except that there are foursearchlights in the ceiling. and they can look at fourthings at once. four, not seven.
how many remember the magicalnumber seven plus or minus two from your school days. that is now known tobe an overestimate. the actual capacity ofshort term memory-- and when i say capacity idon't mean it's a place. it's not a bucket. but you have four searchlightsthat can look at four things at the same time. some people have five.
but most people it's four. four, plus or minus one. for dogs it's one, plusor minus one. and so those four things thatthose four searchlights can be looking at at any time arethings like goals, numbers, words, objects. if a search light is pulled awayfrom what it was looking at then the whole thing is lostfrom short term memory. so short term memory is again--it's not a place.
it is what you're attendingto right now. so that's why you can have thesituation of sitting in your living room reading and thenhearing the cat meow. and saying, oh i haveto feed the cat. and getting up to go andto feed the cat. and then having the phone ringand then having your brother talk to you on the phone. then you finish talking to yourbrother on the phone and go sit down and goback reading.
then the cat meows againand you go oh, i forgot to feed the cat. because your brother's callpulled one of the search lights off your goalof feeding the cat. so here's a short termmemory test. memorize those numbers. say your phone numberbackwards. go ahead. you do it?
ok, what were the numbers? anyone know? ok, memorize these numbers. now, if you notice somethingabout those numbers namely that they are the first sevendigits of pi, it now is not seven things to remember. it's one thing to remember. and your brain can doit much more easily. memorize these numbers.
that's probably three bits. it's like odd numbers startingat, ending at. memorize those words. what are they? not bad. same number of words. same number of words,one chunk. so if we go to this websiteand we look at what it's showing us, it's showingus search results.
so somebody has done asearch and they're looking at the results. and here are the resultsdown here. and the website is nice becauseit's giving us a way to fill in advanced searchoptions if we want to adjust our search. but there's something importantit's not showing us. what is that? what did we search for?
and of course i'm going todefend the programmer now and i'm going to say, well you onlyput it in 10 seconds ago, what's wrong with you? can't you remember somethingfor 10 seconds? no, because i'm human. i have four searchlightsin my brain. maybe some of you are luckyand you have five. but got a certain numberof searchlights. and they were totally pulledaway from my search terms by
the search results. because all my attention isfocused there trying to figure out, are those resultsrelevant to what i was looking for? now long term memory isthat warehouse i was talking about earlier. and the memories in their-- memories are also tied toperception in the current view of cognitive psychologists.
think of it this way. if i look at his face, let'ssay i look at dan's face. what happens in my visual systemis starting at the visual cortex and moving forwardin my brain, millions of neurons fire. a huge pattern ofneurons fire. and that pattern is myrecognition of dan's face. that's my memoryof dan's face. and if i look at ben, adifferent set of neurons fire.
it's very much similar becausehe's human and he's human and he's you know about thesame age i guess. but they're not the same. and so i'm not going to-- the thing is the human braincan tell the difference between this pattern of amillion neurons and that pattern of a new millionneurons firing. and if i run into him-- if i runinto dan on the street in san francisco, another setof neurons will fire.
and it will be very similarbecause he's the same guy. but it won't be the same becausewe're in san francisco and not in this room. and remember how we learnedearlier how context impacts and so i might recognize danon the street in francisco. and i might not. especially let's say ifhe shaved his beard. or what if i run into dan's twinbrother on the streets of san francisco.
and i say, whoa, hi dan. and his brother goes, what areyou some kind of pervert? get away from me. so here's the important thingis that a huge pattern of neurons fires. that is recognition. that's what recognition is. and so experiences triggerpatterns corresponding to features in your brain.
your brain is a featurerecognition machine. that's what your brain is. and so similar experiencestrigger the same pattern. that's recognition. recall, in contrast torecognition, is something completely different. imagine what recall is. recall is a particular patternin your brain of millions of neurons becoming activated inthe absence of the stimulus
that produce it inthe first place. that's hard. your brain is actually notdesigned for that. it's designed for recognition. getting a particular patternof neurons to fire again, evoking a memory in the absenceof the stimulus that originally producedit, is difficult. that's why recognition iseasy and recall is hard. and that's also why the factthat these patterns are big
collections of features is whylong term memory is error prone, impressionist, free associative, and easily biased. so for example, featuresmay get dropped from a particular memory. so you may have seena whale shark when you're out on the trip. and 20 years from now, you andyour brother argue about whether you saw a whaleor a shark.
because features got droppedfrom that memory. so here's a long termmemory test. was there a roll of tapein the toolbox? good. what was your lastphone number? the one before you have now. anyone remember it? you do? oh that's good.
was this a pollock paintingor a dalmatian? you'll never eversee the pollock painting in this again. so you shouldn't beburdening long term memory as a designer. so for example here's aninstruction that a designer gave to a user. which is kind of funny becausethe designer knew that they were asking the user to dosomething that users can't do.
change your pin to a numberthat is easy to remember. that's part one. part two. a pin can be 6 to 10 digits andcannot start with zero. your pin must be numeric. they have just givenyou instructions that you cannot follow. and they know it. because at the bottom itsays, remember please
write down your pin. put it on a yellow stickyunderneath your computer so that any guy can comein and steal it. having your brain keep trackof features, keeping those searchlights trained on certainobjects in your in your memory is work. it actually takes calories. and so your brain does itas little as possible. so your brain tracks onlyfeatures crucial to the task.
so that causes something calledchange blindness. so if i show you that thispicture and then i change it, i want you to tell me ifthere's any change. what has changed? what changed? ok, let me change it in adifferent way so that i remove the pause in between. so your brain is going tofocus on the people. because you're human, your brainwill focus on the people
and their food and thebeer probably. and so people say things like,well he had a mustache in the first one and he didn't have amustache in the second one, or something like that. but no, it's things that areirrelevant to what you're interested in that you aren'tgoing to notice. so i'm getting closeto the end. i'm like two slidesfrom the end. so as i said earlier,recognition is easy.
recall is hard. we recognize thingsextremely quickly. it doesn't take you very long,especially if you're out in the wilderness, to noticesomething like that. because otherwise you wouldn'tbe passing your genes on. i took this pictureby the way. talk about getting intothe vehicle quickly. so lucy walking around on theafrican savanna 4 million years ago had to be able torecognize objects very quickly
otherwise we wouldn't be here. it doesn't take you verylong to recognize this face, these faces. and even more tellingly, itdoesn't take you very long to recognize that you don'trecognize these faces. and what this shows is that wheni was in graduate school, people thought of thebrain as a computer. and they were trying to figureout what are the algorithms that the brain uses todo face recognition.
and so they said back inthose days, it can't be, is it this one? no. is it this one? can't be that becauseit's way too fast. so they said, it has to besomething like, is it in this half of the database or is it inthis half of the database? it's in this half. take that and splitthat in two.
is it in this half orthat half of the database, binary search. it's way too fast for that. face recognition isway too fast. so they came up with all thesealgorithms back when i was in graduate school abouthow the face recognition can be so fast. and now we know whyit's so fast. it's because the braindoes not search.
the brain is a contentaddressable holographic memory device. it's a recognition machine. like i said, those bigpatterns go off. so it's basically facerecognition and also recognition that you don'trecognize a face is almost instantaneous. it's not searching. there is no search.
similarly people can recognizevery sophisticated patterns like for example, kasparovversus karpov, 1986. if you're a chess masteryou'll recognize that. if you're not, you won't. finally, i guess i'll talkabout performing learned routines versus new behaviors. performing learned routines iseasy, like riding a bicycle after months of practice,driving to the same workplace after many years, and usinga touch pad after a
few years of practice. now the reason that's easyis it's automatic. cognitive psychologists make adistinction between what are called automatic processingand controlled processing. automatic processing useslots of different parts of your brain. it doesn't use up shortterm memory. so it's not using up thosefour searchlights. it's what we computer scientistswould call compiled
mode or parallel processing. there are many processors. you can multitask automaticthings. so one way i think of it is thatevery part of the brain is a brain. and so things that are automaticand running in compiled mode essentially,you can do many of them at the same time. that's why i can walk and chewgum at the same time, or i can
be whistling a tune that i'vevery familiar with. but try to do that withsomething new, following a new cooking recipe, drivingsomewhere you've never been, writing with your non-dominanthand, switching from mac to windows pc or vice versa. that's controlled. that consumes short termmemory and attention. it's what we computer scientistswould call interpreted mode orserial processing.
there's only one processorthere. it's called the attention. there's only one processorand you cannot multitask. so you can only do one novelthing at a time. you can multitask a novelthing with an automatic things, any number of them. so let's just do a test. recite the alphabet a tom. go ahead recite the alphabet a to m. easy.
recite backwards, m to a. that'scontrolled, requires short term memory, requiresattention. hum the first measure of"twinkle twinkle little star." go ahead, do it. hum it backwards. control processing requiresshort term memory, requires attention, full attention. so all of the things thati showed you is why user interface design is a skill notsomething anyone can do by
following user interfaceguidelines. and that's why knowing thecognitive basis helps us prioritize and recognizewhich rules to follow in each situation. now i'll take any questionsthat anyone has for as long as you want. [applause] jeff johnson: thanks. yes?
male speaker: so this model ofshort term and long term memory that you were discussing,how is this consistent with the fact thatthere's a phenomenon where people go unconscious inaccidents or concussions lost memory of the four or fiveminutes preceding the accident, people take versedand not lay down the memory as well? so that model seemed, or thatclose result seemed to [inaudible] the old model ofit being a different short
term memory [inaudible]. jeff johnson: yeah one of thethings that had been noticed when i was in graduate schoolwas that people come back from combat situations with certainparts of their brain shot away or auto accidents or whatever. some of them can remembernew things. and others can't. and another phenomenon likethe ones you described. and so that's why they sort ofthought there had to do these
certain places where there wasshort term memory and other places where there waslong term memory. and what they now know or nowbelieve is that there are certain areas of the brain thatare essentially gateways. they're sort of bottlenecksfor information. and if those are shot away ordamaged, then the ability for the brain to sort of generateand compare these patterns becomes destroyed. so that's basically it.
it's now thought of as insteadof that short term memory here, let's say inthe hippocampus. because that's where they usedto think it was when i was in graduate school. is that the hippocampusis somehow-- well remember i said earlierthat the human brain can compare patterns and cantell whether it's seen a pattern before. they don't know why that is, butmaybe it's possible that
the hippocampus is involved inthat, in that comparison. so i guess what i'm saying isthat now the understanding of short term memory versus longterm memory is not as cut and dry as it was back in the '70s,when they thought that short term memory is in thehippocampus and long term memory is in the cerebral cortexor something like that. i don't know if that answersyour question. male speaker: well it doesn'texplain those phenomena, but maybe i have to lookdeeper for that.
i understand that. jeff johnson: so say exactlywhat the phenomena was. male speaker: the phenomenathat somebody in the car accident doesn't remember threeminutes before the car accident, which normally wouldhave been laid down in short term memory or long term,whatever you want to call it. jeff johnson: right and so thereis some process by which stuff comes in from perceptionand gets put into long term memory.
i mean one of the things thatwe know now which is kind of interesting. if you're a musician, they knownow that it's better to practice for 15 minutes and thensleep than to practice for eight hours. because the sleep has somethingto do with the stuff you practice gettinglaid down. so i don't know if i can answeryour question fully. male speaker: peripheralvision-- sometimes people talk
about athletes like proquarterbacks having great peripheral vision. has that been tested[inaudible]? jeff johnson: well i wouldn'tdoubt it since there's all sorts of genetic variationand there's a lot of randomness in our genes. and so that's why for examplethose people 4 million years ago who could actually see theleopard coming at them are the ones who survived.
and those who didn't seethe leopard, maybe the neanderthals, are not. [interposing voices] jeff johnson: not that i knowof but one thing that-- have ever heard ofthe "door" study? go to youtube and typein the "door" study. so that's a situation whereit's not really peripheral vision, it's more has to dowith change blindness. but basically if you get someonein a situation where
you change out a personin front of. they're interacting with aperson and you change the person out. a lot of people don't notice. but some people do. and they don't know yetwhether there's any systematic-ness to that orwhether it's just random. male speaker: i was wondering,do you recommend-- should some user interfaces ortypes of interactions rely
more on long term memory andothers have to be designed for a situation where someone hasto keep track of something right then? how do we apply thisin that way? jeff johnson: ok so it'scertainly the case that some user interfaces are designedto be walk up and use. or at least to be used in asituation where it's been a year since you walked upand used it before. and so you're not goingto remember details.
like for example, recoveringfiles from a backup system, how often are you goingto have to do that? so that user interface isprobably going to have to be designed differently from theuser interface for handling your email, which you doevery morning for half an hour or an hour. so people who design walk up anduse interfaces really have a different sort of mindset,design mindset, than the people who are designing userinterfaces that are going to
be used by which peoplewho have a lot of experience and training. i'm not sure that exactly getson your question which was-- male speaker: yeah, i thinkthat ties into it. jeff johnson: yeah,the thing is air traffic control for example. intense information overload allthe time, and also extreme danger if you screw up. so that user interface has toreally support making sure
that you can see things. so peripheral visionfor example. if there are planes over here,but you're looking over here, they have to somehowget your attention. actually a lot of people thesedays are thinking about user interfaces for security guardsat airports and places like that, security checkpoints. because think about that job. nothing happens for months oreven years at a time and then
something happens. and you have to be attentive. but it's hard to be attentivewhen you've been bored for three years. so how do you design a userinterface to make sure that someone notices something ishappening now when nothing has happened for the lastthree years? so there's a lot of attentionbeing paid to that particular problem.
ok, thank you.