http://www.flickr.com/photos/kaptainkobold/

In one of my courses this year I had students talk about the kinds of human factors problems they had run across when using computers. There were a number of great anecdotes, but one thing that interested me was the difficulty they had discerning the difference between a software bug and a human factors problem. For example, one student complained that if they clicked on a certain button in a program, it crashed the computer.

It was interesting trying to explain the difference. I basically said that a bug is an unintentional effect while a usability problem is due to some misunderstanding of how humans could or should use a system. This isn’t perfect;  some human factors problems come from “bugs” that went undiagnosed during development. Generally, I think bugs are not representative of any misunderstanding of human abilities or limitations.

The difference between human factors problems and bugs is about to become very important for Toyota. These are excerpts from a recent ABC news story:

Refusing to accept the explanation of Toyota and the federal government, hundreds of Toyota owners are in rebellion after a series of accidents caused by what they call “runaway cars.” Safety analysts found an estimated 2000 cases in which owners of Toyota cars including Camry, Prius and Lexus, reported that their cars surged without warning up to speeds of 100 miles per hour.

…

Toyota says the incidents are caused by floor mats becoming stuck under gas pedals, but owners say that’s not what happened to them.

…

“I’m absolutely certain that in my situation, it was not the floor mats,” Elizabeth James told ABC News. She was driving her Toyota Prius outside Denver, CO when she says it suddenly shot up to 90 miles an hour, even though her foot was on the brake and not the gas pedal. “I kept going faster and faster,” James said. “And all of a sudden& my foot was pressing on the brake super, super hard and I wasn’t slowing down.” James and some other Toyota owners suspect the accidents have been caused by some kind of glitch in the electronic computer system used in Toyotas that controls the throttle.

…

The National Highway Traffic Safety Administration has done six separate investigations of such acceleration surges in Toyotas since 2003 and found no defect in Toyota’s electronics.

…

“Toyota has announced a safety recall involving 3.8 million vehicles in which the accelerator pedal may become stuck at high vehicle speeds due to interference by the driver’s side floor mat, which is obviously a very dangerous situation.

Here is a link to a picture of what a stuck floormat looks like.

On the human factors side, we have a known issue with the floormats (indeed, I’ve had to pull my Matrix floormat back for years as it creeps forward.) We also have prior incidents of persons mistaking the accelerator for the brake. On the bug side we have a car with a great deal of control given to electronics. Frankly, Toyota loses either way.

The interesting part is how the drivers believe it would be more Toyota’s fault if there is a “bug” in the electronics than if the problem has to do with human factors design. We appear to feel more control over a human factors issue, even when it is beyond our control, than we do with a software bug.

This attitude can be seen over in a Consumerist post of a 911 call by a family with a floormat-stuck gas pedal. (I don’t suggest listening to the call .) Read the comments instead, they include:

“Shouldn’t a CHP officer know that ? Tragic but completely avoidable.”

“Maybe I’m missing something, but couldn’t they just have turned the engine off?”

“It is sad but the driver had the ability to put the car in neutral or turn off the engine off at any point.”

“maybe it was the way I was raised, but I understand the concept of putting a car in neutral, or stepping on the clutch, or shutting the engine off… I’m not trying to be crass, but I feel like everyone should understand these concepts before driving.”

and (wow) -

“I’m gonna come out and say it: It was his fault. His failure, as the driver, to keep his head about him in an unexpected emergency was the #1 cause of the death of his family. The fact that he called 911 is inexplicable, and the fact that some posters on this board call that an example of “having his wits” really makes me question their basic intelligence level. The fact that this story will tarnish Toyota, an otherwise crap car maker, is sad. Sometimes, shit happens. You can place blame, sue, complain, etc. – but if you aren’t sharp enough and alert enough at the time to do the right thing, it doesn’t matter who’s to blame.”

We still have a long way to go in educating the public.

Similar Posts (auto-generated):

• Electronic Books–A Human Factors perspective
• 1960’s Human Factors : The Titan II Missiles
• John Wayne, United Airways, and Human Factors
• Driving and Talking in California

During the conference I had a very personal experience with the effects of automation reliability on trust and subsequent behaviors.  First, a bit of background.  There is a large body of research examining how humans interact with automated systems (Global positioning systems, for example).  Human-automation interaction is quite complex; being affected by many factors.

Julian Sanchez (of MITRE) presented a poster at the conference summarizing the literature; presenting how the many variables of human-automation interaction relate to each other (figure 1).  One factor being extensively investigated is the issue of how much the user/operator (you & me) trusts the automation.

Figure 1. Conceptual Model of Human-Automation Interaction

I have used Google Maps on my phone extensively; and in the many cities I’ve used it, it has been a reliable tool for directions.  Since the phone includes a GPS chip, it can track my movements as I walk showing me my distance to my destination.  However, it failed miserably in San Antonio…twice.  First, I tried to find a restaurant near the River Walk and following the directions led me to go almost in the complete opposite direction.  I was so confident in Google we spent 20 minutes walking around until we asked a local policeman for directions. One reason I was confident was that as we walked toward our destination, the phone confirmed that our position was nearing the GPS destination.

The second failure was when we tried to find the Cowboy bar.  Automation researchers would say that I was complacent–I over-relied on the automation which indicted that my trust was not calibrated correctly.  My high level of trust came from thinking, “San Antonio is a big city, it must be fully and accurately mapped…” as well as past successful navigation attempts. This is one consequence of ultra-high reliability systems: the effect they have on users expectations and trust. Ever since my return, I’ve needed to use Google Maps (on the web or phone) and I have found myself very uncertain of the stated locations and directions offered by Google Maps.  I confirm Gmaps using the competing service (Bing Maps).

Sanchez, J.  (2009).  Conceptual model of human-automation interaction.  Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting.

Similar Posts (auto-generated):

• Trust in Automation
• HFES Conference in San Antonio, Part 1
• HFES Conference in San Antonio, Part 3 – Health/Internet…and ROBOTS!
• HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures

http://www.flickr.com/photos/banlon1964/ / CC BY-NC-ND 2.0

The paper described in this post was part of the Aging Technical Group sessions at HFES.

Hearing Levels Affect Higher-Order Cognitive Performance – Carryl L. Baldwin, George Mason University

Perhaps I was excited by this talk because I could see how the information could be used in the book Rich and I are working on. This presentation was a fascinating exploration of the types of trouble adults over sixty-five might have with auditory interfaces. These problems are not necessarily related to the function of the ear: in general, older adults may have more poor hearing, but it is due to environmental exposure rather than aging of the ear. Many older adults show no detectable hearing loss, yet still have trouble with auditory interfaces, as found by Carryl’s experiment.

An important contribution of this paper was the connection found between decline in a sensory ability (hearing) and decline in cognitive ability, even on tests that had no auditory elements. Carryl addressed this years ago in her article Designing in-vehicle technologies for older drivers: application of sensory-cognitive interaction theory. Essentially, when most of us study cognitive aging, we either omit or control for sensory ability. For example, in my work, all older adults must have corrected vision of 20/40 or better and if there is any auditory component, must meet hearing level requirements.  Sensory ability may well predict their task performance, but I do not study it.

Carryl pointed out in her talk that even older adults with no measurable hearing loss showed worse working memory capacity as stimuli got harder to hear. This was true for younger adults as well, but the older listeners were harmed differentially worse as the stimuli dB levels decreased.  It isn’t hard to see why telephone menus and other auditory interfaces can be so frustrating: what requires more working memory than a softly spoken voice menu with 9 options? Eek.

Take home messages:

“The observation that scores on an assessment of working memory capacity decreased in young listeners indicates that hearing level, irrespective of age, can impact performance on aurally presented working memory tests.” (p.124)

“…functional hearing level may play a substantial role in the performance of older adults.  Subclinical hearing loss may result in the need to expend greater effort to process test stimuli – thus compromising performance in higher order stages.” (p.124)

Primary Sources

Baldwin, C. (2009). Hearing Levels Affect Higher-Order Cognitive Performance.
Baldwin, C. (2002). Designing in-vehicle technologies for older drivers: application of sensory-cognitive interaction theory. Theoretical Issues in Ergonomic Science, 3, 307-329.

Similar Posts (auto-generated):

• HFES Conference in San Antonio, Part 3 – Health/Internet…and ROBOTS!
• HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures
• Two HF Driving News Articles
• HFES Conference in San Antonio, Part 1

One of my major interests at the moment is in the use of technological tools (primarily the Web) in the management of health.  So it was with great pleasure that there was so much research on this topic (I will mention more in future posts).

The first was presented in the Aging session (where Anne was program chair).  Jessie Chin and her co-authors were interested in a cognitive dilemma faced by older adults.  With increasing age, fluid cognitive abilities (those used in rapidly changing situations like working memory) decline with age.  These abilities seem particularly crucial when using the web as well as other tasks.  However, it is known that older adults often compensate for fluid ability declines by capitalizing on pre-existing knowledge (so called crystallized knowledge) which increases with age. (For an adequate and publicly available elaboration of the fluid/crystallized distinction, see the Wikipedia entry).

The researchers examined the relationships between fluid and crystallized intelligence on illness knowledge in older adults (hypertension knowledge).  Consistent with their hypothesis, they critically found that illness knowledge (a form of crystallized intelligence gained through time) was a significant predictor of illness knowledge and that this knowledge may moderate the reduced fluid abilities.

Incidentally, I just published a forthcoming paper in the Journal of Medical Internet Research examining health information searching, aging, and fluid/crystallized abilities.  The take-home message is that tag-based interfaces are best for older adults searching for health information on the web because it capitalizes on their crystallized knowledge.

Jessie’s paper was also the recipient of the Arnold Small Memorial Award for Best Student Paper.

Cognition and Illness Experience are Associated with Illness Knowledge Among Older Adults with Hypertension. Jessie Chin¹, Laura D’Andrea¹, Dan Morrow¹, Elizabeth A. L. Stine-Morrow¹, Thembi Conner-Garcia², & James Graumlich².  ¹University of Illinois at Urbana-Champaign, ²U of I College of Medicine.

Two other papers in this same session were related to human perceptions of virtual assistants and robots.  Jenay Beer (at Georgia Tech) and her colleagues examined older adult’s ability to recognize facial expressions in robot/virtual assistants.  This could be useful because humans are “hard wired” to recognize faces and so can be a useful channel for systems to quickly relay information to users (compared to, for example, a text warning).  However, it is unknown if artificially generated facial emotions can be accurately recognized by users.  The aging angle comes in because home-care robots may become more frequent helpers and companions for older adults living alone.

Emotion recognition of virtual agents’ facial expressions: The effects of age and emotion intensity. Jenay M. Beer, Arthur D. Fisk, & Wendy A. Rogers.  Georgia Institute of Technology.

The other paper, by Neta Ezer, examined younger and older adult’s attitudes about robots in the home.  It examined how people feel about having robots perform a range of home tasks (ranging from vacuuming to emergency response).  To grossly summarize, although people still have doubts about the need or utility of home robots, they can quickly see benefits in very specific circumstances.

More than a servant:  Self-reported willingness of younger and older adults to having a robot perform interactive and critical tasks in the home.  Neta Ezer, Arthur D. Fisk, & Wendy A. Rogers.  Georgia Institute of Technology.

Lastly, Caryl Baldwin (George Mason U) presented a paper on the relationship between hearing ability and higher order cognitive performance.  Anne will discuss in an upcoming post.

Similar Posts (auto-generated):

• HFES Conference in San Antonio, Part 4 – Hearing and Understanding
• Tag-based interfaces and Aging
• HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures
• Older adults prefer less choice than younger adults

I interrupt this HFES re-cap to express my slight dismay at my new computer mouse.  I purchased a portable Bluetooth-enabled mouse for my laptop (I hate using those mouse nubs).  The mouse is pretty nice because instead of a scroll wheel, it uses a full trackball to scroll.

Depending on my mood or environmental conditions, I can switch the mouse between regular mouse + roller scroll ball and full trackball–useful for small spaces or surfaces where the mouse will not work.

Turning the mouse on or switching modes requires you to press the mode button.  Can you find it?

Ok, not a big problem.  The button location is unusual but what turns this minor annoyance into a big human factors problem are two things:  powering on/off the mouse and changing mouse modes requires different sequences, and more damning is that visual feedback of success is obscured by my finger over the button:

In the image above, I am only using the very edge of the button so I can see the confirmation feedback.  Is this a deal-breaker?  No, the merits of the mouse (size, Bluetooth, trackball) still make me glad I got it but it’s little things like this that break the user experience or make it unpleasant.

Similar Posts (auto-generated):

• The Tactile Thinkpad: More Laptop Redesign from Lenovo
• Bad Astronomy on Usability
• Human Factors vs. Bugs – Somebody stop my Toyota!
• Educational (low-priced) laptops and cross-cultural Human Factors

I‘d like to highlight some of the talks I enjoyed last week and point our readers to their research.

First up, we have:

The Influence of Rating Method on Knowledge Structures.
Chad C. Tossell, Rice U.; Brent A. Smith, U.S. Air Force Academy; Roger W. Schvaneveldt, Arizona State U., Polytechnic

This talk was a great introduction to understanding how we organize information in the brain as we change from novices to experts in a task. This isn’t something I’ve done in my work, so I learned a great deal in the 15 minute presentation.

In a nutshell, one can use a technique called Pathfinder to see how people link concepts associated with a job or task. These links can be analyzed to see how people change the organization of the knowledge as they become experts. For example, a novice would have a very different understanding of how the parts of an engine depend on one another, or at best only show topical links between parts of the system. An expert mechanic would organize the parts of the engine differently. Usually the knowledge structure of one expert is very similar to that of other experts (while novices vary greatly.)

Unfortunately, using the Pathfinder technique is time consuming and arduous. The current talk focused on simplifying the technique while preserving the integrity of the findings.

How Pathfinder Works

Each concept is presented with every other possible concept in a task or system and the participant judges the relatedness of the two concepts.

This creates a map. Below is an example of what one of these maps might look like, taken from the NON-Pathfinder program (and addictive web game) Funny Farm.

As you can see, related concepts are linked together in multiple ways. (In the game you try to guess the unrevealed related concepts, shown above in orange.)

Chad was nice enough to let me show the comparison tool (called “Pairwise”) they used in the experiment, which was compared to the Pathfinder technique.

Here, instead of going through each comparison one by one, the person can drag all the boxes to their degree of relatedness to the core concept shown (coordinated flight).

Results

They did not find as much of a time savings as they anticipated using this method, but came up with some good changes to make that should result in a more efficient technique. However, it did result in novice and expert knowledge maps that were not different from a Pathfinder analysis, which is a start for creating the alternative technique.

I enjoyed the creativity of the interface and look forward to seeing how Pairwise develops!

Primary Source

Tossell, C., Smith, A., & Schvaneveldt, R. (2009).  Influence of Rating Method on Knowledge Structures.   Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.

Similar Posts (auto-generated):

• HFES Conference in San Antonio, Part 1
• HFES Conference in San Antonio, Part 3 – Health/Internet…and ROBOTS!
• HFES Conference in San Antonio, Part 4 – Hearing and Understanding
• HFES Conference Part 5: Automation & Trust & Google Maps

Anne and I just got back from the Human Factors and Ergonomics Society conference held in San Antonio.  We plan on posting some snippets of posters/talks that we found interesting in an upcoming post.  But in the mean time, here is a panorama of the view from our hotel.

Being in San Antonio, TX, we also visited a Cowboy bar complete with a mechanical bull.  Being human factors geeks, we had to take a look at the surprisingly simple controls used to create such complex movements.  And no, we did not analyze the controls or interview the operator

Similar Posts (auto-generated):

• HFES Conference Part 5: Automation & Trust & Google Maps
• HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures
• HFES Conference in San Antonio, Part 3 – Health/Internet…and ROBOTS!
• HFES Conference in San Antonio, Part 4 – Hearing and Understanding

http://www.flickr.com/photos/kayakaya/ / CC BY-SA 2.0

Not much detail but the LATimes is reporting that:

“There was a misunderstanding about an embedded default setting applied by the machine . . . ,” officials at the renowned Los Angeles hospital said in a written statement that provided no other details about how the error occurred. “As a result, the use of this protocol resulted in a higher than expected amount of radiation.”

The article ends with an interesting footnote:

Dr. Thomas Dehn, a radiologist and chief medical officer for National Imaging Associates Inc., which manages health plans for private insurers, said he believes that overrides are more likely to occur at large, state-of-the-art hospitals.

“At a small hospital, you are not going to try and out-think GE,” he said.

“You have to be pretty confident to think you know more than the guys who designed the equipment.”

This incident reminds me of an incident documented in the book Set Phasers on Stun.

Similar Posts (auto-generated):

• Usability Affects Use: Police make fewer arrests due to struggling with software
• More medical errors–Operating on the wrong side of the patient’s brain!
• Developing the “science of error measurement”
• Response to “Paper Kills”

The Kindle e-book reader has great promise, especially for students.  Who wouldn’t want to trade in a bunch of heavy books for a slim electronic device?  Amazon partnered with Princeton to see how students would interact with the device.  The results are not good.  The student’s comments sound vaguely familiar to my own experiences.  The Kindle is great as a pleasure book reader, but not so good for academic use.  One student in the trial eloquently commented that:

Much of my learning comes from a physical interaction with the text: bookmarks, highlights, page-tearing, sticky notes and other marks representing the importance of certain passages — not to mention margin notes, where most of my paper ideas come from and interaction with the material occurs,” he explained. “All these things have been lost, and if not lost they’re too slow to keep up with my thinking, and the ‘features’ have been rendered useless.

One of the loudest complaints seem to center on the inability to easily take notes (which was one of my major gripes).  The human factors lesson?  Task analysis:  learn how the user does it now, and then replicate or improve it, but don’t interfere with it.  I can’t beat up on the Kindle too much (I really *want* to like it) because presumably it was not designed for academic use.

[The Daily Princetonian via Engadget]

Similar Posts (auto-generated):

• Electronic Books–A Human Factors perspective
• HF/Design/Usability Potpourri from around the Web
• “Internet radio so simple that even an old, stupid idiot can use it”
• How does one get into Human Factors?

I‘m glad that open-source software is taking usability seriously.  I think that not having a good user experience may be one of the biggest hurdles to more open-source adoption (e.g., compare GIMP to Photoshop).  Shuttleworth (of Ubuntu) has a great plan: the STFU protocol.

During his keynote, he extended an invitation to any open source application to submit their software for testing by user-experience experts. The sessions would be recorded for posterity, and the developer would not be able to interact with the user.

“If the developer is in the room, they have to say nothing. It’s the shut the f— up protocol,” Shuttleworth said. “You sit and watch someone struggle with the software that you’ve so lovingly produced.”

[Internet News via Slashdot]

Similar Posts (auto-generated):

• Mac-based usability testing software: Silverback
• “Science of fun”; User testing the game Spore
• Quick note: Usability of Electronic Health Records
• Crowdsourced Usability Testing?

Electric cars are utterly silent making them hazardous when they sneak up on you at low speeds.  Nissan is thinking about having their Leaf electric car emit the whine reminiscent of the flying cars in Bladerunner.  It’s one of my favorite movies so I approve!

“We decided that if we’re going to do this, if we have to make sound, then we’re going to make it beautiful and futuristic,” Tabata said.

The company consulted Japanese composers of film scores. What Tabata and his six-member team came up with is a high- pitched sound reminiscent of the flying cars in “Blade Runner,” the 1982 film directed by Ridley Scott portraying his dystopian vision of 2019.

[Bloomberg]

Similar Posts (auto-generated):

• The VP Debates: An Adventure in Clutter
• In 2019 I will be 44…
• Design out, Guard, then Warn
• The New Field of Unmanned Aircraft

Time.com is reporting that part of the economic recession may have been caused by Warren Buffet not being able to check his voice mail:

as Buffett was rushing out to a social engagement in Edmonton, Alberta, he got a call from Bob Diamond, the head of Barclays Capital…[ed. Diamond was creating a plan to save an investment bank and needed money from Buffett]…

Fast forward 10 months. Buffett, who admits he never has really learned the basics of his cell phone, asked his daughter Susan about a little indicator he had noticed on the screen: “Can you figure out what’s on there?” It turned out to be the message from Diamond that he had been waiting for that night.

Similar Posts (auto-generated):

• Senior-friendly mobile phone
• Can you hear me now? Why yes, I can!
• Unusually quiet morning radio show
• Parking garage ticket machine warning

The NYTimes has an interesting OpEd where they asked various designers to re-imagine the homeland advisory system.  It’s a multimedia presentation with narration from the graphic designers.  Not much warnings research but interesting.  Here is what it looks like now:

and here is one proposed redesign that, according to the designer, takes advantage of our ability read emotions from eyes:

Similar Posts (auto-generated):

• Homeland security warnings
• Getting Ready for November – Ballot Designs
• Trust in Automation
• Electric Scooters and their Warnings: A Guest Post by Kim Wolfinbarger

Gordon Bell, a Microsoft Researcher, is recording his life in excruciating detail in a project dubbed MyLifeBits:

Web sites he’s visited (221,173), photos taken (56,282), emails sent and received (156,041), docs written and read (18,883), phone conversations had (2,000), photos snapped by the SenseCam hanging around his neck (66,000), songs listened to (7,139), and videos taken by him (2,164).

Why is he doing this?  He sees some appeal in the ability to always remember:

By using e-memory as a surrogate for meat-based memory, he argues, we free our minds to engage in more creativity, learning, and innovation (sort of like Getting Things Done without all those darn Post-its).

In a work context, this is true.  A large part of my time is spent looking for files or trying to remember.

A whole slew of interesting human factors and usability questions are elephants in the room:

• Currently, a portion of the recording is done manually.  How and what should be automated?
• How does one efficiently search/browse through potentially petabytes of lifedata?  I don’t think a search engine would suffice (not all material would be textual).
• This seems to solve the “encoding” problem in memory.  But it wreaks havoc with the “retrieval” portion.  You still need a good retrieval cue.
• What are the implications of off-loading so much memory?  How will it change the way we currently learn/work?
• As a type of automation, what will happen when it fails or is unreliable?
• What are the privacy implications of recording this much data (especially the sensecam)?

His book outlining this idea comes out September 17th (Amazon link).

Similar Posts (auto-generated):

• The VP Debates: An Adventure in Clutter
• Driving and Talking in California
• In Praise of a Good Interface
• Interface aid for hyper-milers

I woke up this morning to the People’s Pharmacy on NPR and an interview with Peter Pronovost (of checklist fame in a previous post) and David Newman-Toker. These two M.D.s hope to inspire research into accurate error measurement as an essential to developing systems that avoid errors in medical diagnosis. The goal of their commentary, published in JAMA, is to bring attention to the field of diagnosis.

The essence of the problem is that diagnosis errors are prevalent and not only result in ineffective treatment for a condition that is not present, but that either the lack of treatment of the real condition or the side-effects of treatment of the incorrectly diagnosed condition result in actual harm. This harm is one of the things that we do not yet measure well.

Once we fully understand the errors, we can develop effective decision aids that function and are accepted in the medical context.

How to reduce diagnostic errors (from the article):

• Develop Systems Solutions to Cognitive Problems
• Create Actionable Categories of Errors Based on Context Rather Than Cause
• Emphasize Misdiagnosis-Related Harm Rather Than Diagnostic Error
• Build Workflow-Sensitive Solutions
• Focus on Comparative and Cost-effectiveness

Here are my favorite excerpts from the article that point to potential human factors solutions.

• Look to other areas for information

“Parallels in medication safety offer an alternative view. If the problem is illegible physician handwriting in medical prescriptions, the most efficient solution is probably not handwriting retraining but computer-based prescription writing. Likewise, if the problem is a cognitive bias such as a tendency to overestimate the probability of a rare diagnosis recently encountered, the most efficient solution might not be cognitive debiasing training for all physicians but computer-based decision support systems that provide accurate estimates of disease probability.”

• Collect data

“Systematically recording key clinical inputs (symptoms/signs/tests) and outputs (morbidity/mortality/costs) would also offer a platform for continuous quality improvement through structured feedback…”

Link to the primary source:

Newman-Toker, D.E. & Provanost, P. (2009). Diagnostic errors: The next frontier for patient safety. JAMA, 301 (10), 1060-1062.

Similar Posts (auto-generated):

• Patient record mistakes
• “From the Doctor’s Brain to the Patient’s Vein”
• Response to “Paper Kills”
• Error Leads to Radiation Overdose