Dr. P patted the middle aged patient on the back, helped him off the elevated exam table and guided him to the chair by the sink. He picked up the chart and using the exam table as his desk he flipped through the chart, pulling out several pieces of paper, spreading them to his right, while making small talk with his patient. He reached into his pocket and pulled out a battered silver recorder and without any warning started dictating: “Mr. H is a 60 year old mildly obese gentleman presenting with…..“.
He had a pen now in his right hand, and as he was talking into his recorder, shuffling the various papers in front of him, he was also writing orders and prescriptions as fast as he was dictating. “….follow up in two weeks” was the last thing he said. He didn’t write that one down, but turned around, handed the patient a bunch of scripts, told him to stop by the front desk and make an appointment two weeks out and stop by the lab on the fourth floor to pick up a container for the urine test. Two minutes, tops, including the small talk. It was my turn now and I was sweating bullets because I knew exactly what he is about to say. “Can I do this in the EMR?”
EHR usability has finally arrived to Washington as the guest of honor at the most recent ONC HIT Policy Committee hearing. ONC seems to be considering the regulation and certification of EHR usability. NIST has created a testing procedure and just like its Meaningful Use testing procedures, it is superficial and doesn’t really test anything of any consequence. Those who represented “providers” and patients argued for the need to improve usability and those who represented academia and grant funded research argued for more funded research. Predictably, usability experts, argued for hiring more usability experts. Large vendors eloquently stated their objections to government mandating what EHRs should look like and small vendors argued that the more mandates, the better, since this will automatically remove the built-in competitive advantage of those with larger budgets and larger usability departments. As is customary, EHRs were compared to ATM machines, cars, iPhones, Google and a variety of “other industries” that are all so much more advanced than health care when it comes to usability.
When usability, or lack thereof, is discussed, most actual users of EHRs (oddly, there was only one of those at the ONC hearing) think about too many clicks, too many screen changes, convoluted workflows, stilted terminology, finding needles in haystacks, slow and freezing software, crashed servers, disappearing information, mind numbing alerts and lack of functionality. But wait, there is more… There should be out-of-the-box interoperability, ability to customize everything, thousands of templates, no bugs, no need for training, no need to document all that crazy billing stuff, and it wouldn’t hurt if it looked pretty and colorful (as opposed to “dull”), and it should work on the iPhone, iPad, Blackberry, Android, Mac and Windows 98 too.
There are two questions facing all involved: How to measure usability, and who should define and measure usability. The ONC committee is presumably exploring whether government should be the answer to both questions. Before you cheerfully agree that government should indeed regulate EHRs through an FDA approval process, let’s take a minute and explore what it is that we want government to regulate. No doubt, we want government to ensure safety of patients. Since EHRs are part of the clinical process, the FDA has, in my opinion, a clear and definitive role in making sure that EHRs do not endanger people’s lives. Usability, however, is a much larger aspect of a product than safety. To use the completely inappropriate analogy to automobiles (more on that later), it is pretty obvious that government should mandate that cars come with airbags and seatbelts, but it is less clear that government should mandate that all vehicles come with heated seats or automatic transmission, even if manual transmission and freezing bottoms may be tied to some types of accidents, for particular types of users, in particular circumstances, at particular times of day. And here is a trickier question: should the government fund and engage in the design of a preferred seatbelt, and then require that all automobile manufacturers use the exact same design?
Back to the more general question of usability and how it should be measured. ONC is funding projects and the government is paying for contracted work to provide an answer to this question. The initial outcomes as presented at this hearing consist of a rather strange standard form for assessing effectiveness (success/failure), efficiency (time to completion) and satisfaction (subjective) for several use cases based on narrow Meaningful Use criteria as defined by NIST testing procedures for certifying EHRs for Meaningful Use incentives. For example, an evaluator would be asked to prescribe a statin for a patient, or record vital signs, or execute a similarly granular sub-step of real life clinical scenarios. I don’t think I need to belabor why this exceedingly simplistic approach provides no indication for evaluating usability of the EHR. However, as one participant stated during the hearing, it seems that it is better to measure something than nothing. If you are reading this and you are a physician, this way of measuring things out of context, just because we can, would be akin to measuring the percent of patients sitting in your waiting room at a random date and time with a blood pressure under 130/70, and deciding that you are a good doctor if they all do, or a bad one if they don’t, whether you are a pediatrician, a geriatrician, or if you practice in a posh suburb, or tending mainly to indigent and homeless folks, or if it just so happens that this is the time when you do sports physicals for the local boys’ lacrosse team.
To continue on this path to oversimplification, there is a much circulated drawing in the circles of EHR usability experts (created by a former colleague of mine, Eric Burke), depicting three screens: the first shows an Apple screen with one word on it – “touch”, the second shows Google’s famous home page with nothing but a Search button, and the third is a cluttered data entry screen supposedly belonging to an EHR. This drawing is supposed to impress upon us how horrific EHR designs are by comparison to “other industries” and other software products we use in our daily lives. I’m not totally sure what the Apple screen is supposed to symbolize since touching a blank screen does nothing for me (sorry, Eric). I do understand the Google search screen and I agree that if you only want to do one thing, you should only have one button. When you want to do many things, many business and enterprise type things, it would be more meaningful to compare an EHR screen to say, SAP, or Siebel, or Epicor, or Photoshop, or any serious CAD application. The results of such comparison may surprise some usability experts, who seem to have all the answers. EHRs are not leisure applications for consumers and EHRs are not gaming platforms. To use the automobile example one last time, EHR is to iPhone and Facebook what a Ford F-150 is to a Little Tykes Cozy Coupe.
In conclusion, I would like to leave you with a screenshot of a widely used EHR. It indeed defies almost every single usability expert generated opinion on what good design should look like. However, if you look very carefully at the top-left of the screen, you will see that this is a screenshot from VistA, the VA EHR, designed and built by clinicians for clinicians. I have not met a single doctor who used VistA and did not really, really like it.
So let me ask again, who do you think should decide what a good and usable EHR should look like, a Government usability expert or Dr. P?
Margalit Gur-Arie was COO at GenesysMD (Purkinje), an HIT company focusing on web based EHR/PMS and billing services for physicians. Prior to GenesysMD, Margalit was Director of Product Management at Essence/Purkinje and HIT Consultant for SSM Healthcare, a large non-profit hospital organization. She shares her thoughts about HIT topics and issues at her blog, On Healthcare Technology.
Categories: Uncategorized
Many EHRs really do suck, as ZDoggMD described in his very popular parody video ‘EHR state of Mind’. But without some nudge towards an improved user experience many of the “less than optimal” EHRs will only get worse as they grow and Engineering-centric developers add more and more features on top of a poorly designed information architecture.
Vehicles are not allowed to be sold in the USA unless they have meet strict safety standards, why isn’t this the same for EHRs?
Don’t like 30 clicks to order Ambien? How about 50!
Usability in healthcare is critical to patient safety, but way too many vendors have short-cut their “Safety-enhanced Design” (ONC’s euphemism for Usability) in order to get their clients Meaningful Use funding.
Is it just greed? Hhy do EHR vendors choose to ignore the research and develop a product that doesn’t match the mental model of their users? Or are they just uninformed? Do they choose to be uninformed? Is ignorance really bliss?
We’ve had the tag line “Usability starts with you” for a while–and remember when a senior developer wrote back “that it ends in “Y.”” In healthcare the why is obvious – it saves lives.
Don’t believe me? See the Joint commission alert 54
Stephanie Warner
Director of Business Development
The Usability People, LLC
4000 Legato Road, Suite 1100
Fairfax, VA 22033
703-687-6300 x323
http://www.TheUsabilityPeople.com
Please visit our Thought Leadership at
http://www.theusabilitypeople.com/thought-leadership
So I was active most of the people who were suffering from the same problem and have trusted and
tried the same product to stay fit and toned.
But it will help make you more susceptible to cancer.
The odds of a good candidate for weight loss? You want your weight loss to be coupled with
no sugar. The scientific approach of this e-book aiding to lose weight without Dieting Vegetables.
Margarit is among the best on topic. Keep in mind she is not even a native speaker of English. I usually agree with her, but she is somewhat off in this posting.
I just wanted to thank you for the article, it was certainly thought provoking. I think we are all in a very interesting time where UX, the lack thereof, and how to measure it are mere forerunners in the landslide of questions coming about EHRs, how to use them and how to use them “meaningfully”. As technology evolves to address these concerns, new concerns will continue to crop up. The key is keeping your health care IT professionals and vendors as educated as possible and providing tremendous room for R&D, without which the EHR will never reach it’s full potential. It’s an exciting time to be in the world of informatics, and especially as it relates to something as far-reaching as our health care.
http://about.me/rook
“QA, you are mixing up concepts here. You are again quoting examples of things made in factories, and you are again talking about safety.
I am sure you understand that usability is a tad larger subject than safety.”
Hmmm…last time I checked displays in nuclear power plants, aircraft gauges and flight monitoring displays, medical device software and many other systems regulated by the federal government weren’t made in factories…any more than say EHRs are “made in factories”. Unless you’re arguing that human factors engineering shouldn’t (or can’t?) be applied to software (or maybe just health IT software)?
There are many dimensions for evaluating usability. Safety (“use error”) is the most important dimension…and the most appropriate place to start.
If you can point to existing EHR industry led protocols or metrics for assessing the “use error” in their product, please post them here. And examples of collaboration among leading vendors in this area.
User-centered design doesn’t mean that users (e.g. doctors) should design and come up with metrics for measuring the usability of systems. Unless you’re just making stuff up.
The common denominator to your examples is that they are all tangible goods. There is a big difference between, say, a display in a cockpit or some other gauge and pure software, which is just bits and bytes. And as Dr. Levin explained above, the way EHR software is used is indeed different than anything else out there. By definition.
User error is not always associated with safety issues, and safety issues are not always due to user error.
Use centered design is most definitely not requiring users to become software engineers. However, they should define the metrics, i.e. what should be measured (not how), based on what is important to them. How do you know what to measure, if you don’t know what your customers deem important? And I am not talking about basic coding standards, such as font sizes and types, scrolling, button location and uniformity of controls type and placement. This in my opinion is just good workmanship on the part of the software folks and should not require anything out of the ordinary.
This is a very immature and competitive industry with an undue number of products, most of which will eventually cease to exist. This is also an industry where plagiarism is rampant and largely nonpunishable. It would be incredibly naive to expect industry leaders to “shar” intellectual property which gives them a distinct competitive edge over new entrants.
There is a great need for innovation in healthcare especially in data collection side. It makes no sense that a physician who is a highly specialized professional should be doing data entry. Most of the information physicians enter is collected from the patient anyway. Why shouldn’t the patient enter this information. We saw this application and decided to not deal with entering mundane data again (http://123diagnosis.com)
There will be no savings created in healthcare unless more technology is put in place. Things like self-directed lab analysis, increased over the counter medicine (or behind the counter/non presecription medicine) will lead to a lot more savings than the limit to access that the health care reform is trying to put in place.
Margalit said:
“it is mighty presumptuous to assume that any one Government can decide what is good technology and what is bad technology”
So I guess we should throw out the human factors evaluation / certification programs at the Nuclear Regulatory Commission, the Federal Aviation Administration, the National Transportation Safety Board, the Food and Drug Administration, the Consumer Product Safety Commission, NASA and the military?
Gary Levin said:
“The end user ultimately determines usability, and frequently one cannot pre-determine that factor until the system is used. Setting a standard for usability will be a meaningless exercise in more governmental monetary waste.”
You’re basically arguing that the user-centered design process is worthless. And that we should just ask doctors to assess and design systems. These conclusions, I’m sure, are based on your expertise in human factors engineering from medical school.
If I may jump in here with a clarification from my point of view and what I stated in an earlier comment.
I don’t think I or Margalit are arguing that the” user-centered design process is worthless”. I have had extensive experience designing software (and I believe Margalit also has similar experience) and I am a firm believer in the value of good software design in making systems easy to use and to help prevent errors. My point is that this is not something that can be codified in government regulations. I know good design when I see it but I don’t think this can be specified as simple rules. (BTW, my medical school training did not have any course material that could be considered human factors engineering although my engineering training did.)
I believe that the government should define standards for data (metadata), interoperability, security, privacy, etc. but I just don’t think it can do the same for “usability”.
Couldn’t have said better….
QA, you are mixing up concepts here. You are again quoting examples of things made in factories, and you are again talking about safety.
I am sure you understand that usability is a tad larger subject than safety.
No one here is arguing against imposing patient safety regulations on EHRs and no one is debating the merits of human factors evaluations.
However user-centered design is just that: user-centered. And the users are doctors, and yes, you should ask doctors to assess and design systems.
You should ask them before you start the development process, continuously during the process and after the software is complete and accept their opinion in a virtuous feedback loop, or you will not have customers. Unless of course the government is delivering them to you on a silver platter.
Margalit, thanks for a fine thought-provoking post. Many replies have occurred since I read it, and I haven’t had chance to read them all yet.
I have just one quibble with your statement that “Large vendors eloquently stated their objections to government mandating what EHRs should look like and small vendors argued that the more mandates, the better, since this will automatically remove the built-in competitive advantage of those with larger budgets and larger usability departments.” That implies that the testimony for/against government mandates was split among EHR vendors, but the written testimony indicated that only the rep from CureMD argued in favor of mandates, so your summary seems to be a strange extrapolation from a sample of 1. “Large” does not mean “right” but it doesn’t necessarily mean “wrong” either: How many physician practices and end users have expressed their usability requirements to the larger vendors, compared to those who have interacted w. CureMD? Just asking…
But overall, thought your post was quite good.
David
David, I might have been unclear, and I should have stated the actual numbers of various representatives.
I do not think that large means bad or good, or that small has any of those meanings either. I do however think that it is infinitely easier for large vendors to engage in quality usability research and implementation, than it is for a smaller vendor who may only have a handful of resources and a handful of dollars. Large vendors are much better positioned to do effective usability work, exactly because they have larger install bases and because usability done right is expensive, and I can see how a smaller vendor would prefer that government prescribes the details, which will significantly lower the barrier to entry, particularly for “me too” type of products. I just don’t think this is in the best interest of users.
I somehow suspect that we do not disagree….. 🙂
QA, I am not certain from which you are coming with your comments about Margalit, whether you are from the health space or another IT space. From a physician perspective and from one who has used multiple EMRs, including VISTA, the DOD EMR and private EMRs, the healthcare space is vastly different than industrial or banking software applications. The most obvious difference are the variable tasks performed by each users. In a banking or industrial space the users are frequently assigned or use one particular feature of an application, assigned to their particular niche in the organization. Not so in medicine. Physicians wear many hats throughout the day, and also must use multiple EMRs in office, hospitals (frequently more than one). Emergency room usage is vastly different from other locales, and Even in one setting of a clinic visits differ enormously, some very brief and others complex and lengthy. Many EMRS require entry of meaningless information without regard to the complexity of the visit. Many of these are for some arcane reasoning on the part of administration, insurers, payors, CMS and have become engrained in the system at one point or another, may be obsolete and/or useless. Besides money as Margalit says another precious resource (and perhaps the most important one, is time) Simply stated physicians are becoming much less patient by external factors imposing on patient-physician face time. The end user ultimately determines usability, and frequently one cannot pre-determine that factor until the system is used. Setting a standard for usability will be a meaningless exercise in more governmental monetary waste.
This should not take a series of committee meetings by unknown people. It should take a group of physicians from many specialties working through this equation. Many EMR vendors in the past have sat down with the doctors asking and also following the doctors throughout several days of work observing their work flow and then designing the software to comply with the workflow,not the opposite. A series of onsite observations would do far better than a group sitting in a meeting room developing more useless vector diagrams. Margalit is owed an apology, and recognition for her many observations and articulate articles on health.
QA writes:
Dr. Lowry’s testimony, of course, stated the opposite:
“Let me be clear: the usability evaluation program (EUP) is focused on helping developers of systems assess and demonstrate that their software is free from design-induced user-error and will not dictate particular user interface designs.”
If that’s true, then NIST seems to be taking on some of the safety issues that the FDA should be handling, and not addressing usability issues, as Dr. P and I would define them, at all.
Check out the recent posts on “Health Care Renewal” about the lack of usability of the CPOE system at Memorial-Sloan Kettering.
Ms. Gur-Arie says:
” And to use a common platitude from the general health care debate, in a world of “finite resources”, we should be mindful of how we spend tax payer money.”
Agree completely. I can think of lots of ways that the billions we’ve spent on HIT so far, with precious little to show, could have been better spent (I’d start with pre-natal care and childhood immunizations).
One thing people must realize – there are the user aspects of EHRs and the actual medical implications of EHRs. Being able to retrieve X-rays or immunizations or transfer information is a different issue than what the computer means in terms of patient interface and medical decision-making.
There are far more people who can address the former than the latter.
http://www.guardian.co.uk/commentisfree/cifamerica/2011/apr/23/nhs-health
Thanks for the excellent link Cory. I would like to quote a couple of sentences from this reference:
“Good technology is adopted. Bad technology isn’t – it’s always that way. All the government money and training in the world doesn’t change that. Spend it on patient care.”
I would add for QA’s benefit, that it is mighty presumptuous to assume that any one Government can decide what is good technology and what is bad technology, as the NHS seems to be discovering as it goes along. And to use a common platitude from the general health care debate, in a world of “finite resources”, we should be mindful of how we spend tax payer money.
Margalit is quickly becoming the nation’s single best source of mis-information on EHR usability.
The CIF (Common Industry Format) that you poo-poo was actually developed by the IUSR working group of usability experts from purchasing and supplying companies, including companies such as IBM, Microsoft, Hewlett-Packard, Boeing, US West and Kodak.. It was based on collating best practices from the different companies, and aligning this with ISO 9241-11. It became an ANSI/ISO standard in 2006. The idea that it isn’t useful in the context of EHRs is ludicrous.
Margalit speculates that EHR Evaluatiojn Protocol that NIST is developing will dictate user interface design when Dr. Lowry’s testimony, of course, stated the opposite:
“Let me be clear: the usability evaluation program (EUP) is focused on helping developers of systems assess and demonstrate that their software is free from design-induced user-error and will not dictate particular user interface designs.”
Margalit casts doubt on the usefullness of myriad of existing federal usability evaluation / certification programs…because they are being used on products that aren’t EHRs…you know completely different products like medical device software!?!
Before you spout more ignorance, please, please, please speak with someone who actually works in human factors. Or at least listen to Ben Schiederman’s testimony from the ONC Policy Committee hearing.
Mr/Ms QA, thank you for the dubious compliment. First and foremost, I would like to clarify that I am not opposed to usability testing and/or human factors analysis and/or UCD for EHRs or anything else. Quite the opposite. The only question here is who should do the actual work, and who should make the decisions. I believe in a quasi free market, it should be the sellers and the buyers.
The CIF is a format for reporting usability tests results and the ISO standard you are quoting is a definition of “how” to measure and test, not so much “what” to do in each case. I have absolutely no problem with NIST creating helpful documentation and guidelines for vendors to improve whatever they are currently doing (or not) in the usability domain.
I do object, however, to NIST creating test procedures (as the one outlined in the bibliography to the hearing), and passing them down to an ATCB to test or score EHR product usability, for two reasons: I am not convinced that NIST has enough expertise to do so and I am not convinced that ATCBs can implement those in a way that is meaningful to users.
I have listened to the portion of the hearing you are suggesting and if my memory is correct (the transcript is not available just yet), there were a few problems pointed out by other panelists (I am trying to be gentle here). If you are referring to the written testimony and its bibliography, then again, although some general ideas may be the same, manufacturing things with screens, is not the same as writing EHR software. And again, educating vendors and users is a good thing. Mandating what and how and who, is not.
So hobbling the doctor is OK as long as the staff might have future benefits and the patient might have future benefits. You are OK with decreasing productivity by 150%. This will be useable how?
No, it’s not OK, unless the doctors chooses to “hobble” his/her performance for ancillary benefits and this is why folks need to be honest upfront.
Now, the amount of hobbling is largely dependent on how a doctor documents on paper. If he is fast and efficient like the doc described here, he will be hobbled. If he is in the habit of writing extensive documentation for each visit, he may be less, or not at all, hobbled.
Here is an interesting thought, though. I suspect that the hobbled party may be the patient. In successful EHR implementations, physicians are able to return to previous levels of productivity in 3 to 6 months (some take 12 months).
The ones that take the computer into the exam room, are taking away at least 2 – 3 minutes per visit to deal with “computer stuff”, and this is true even for those who share computer interaction with their patients. Since doctor-patient time is 10 to 15 minutes tops, this is a large chunk of time that formerly belonged to the patient, and this may explain why some patients feel left out.
The docs that don’t take the computer into the room, are probably taking work home, or shorten the visit by just a little bit each time, to achieve par with pre-EHR productivity.
I know this sounds pretty bad, but sometimes I wonder if an optimally used EHR with optimally displayed information and decent connectivity, is not actually compensating for the lost time, as far as ultimate quality is concerned. I wonder if we are witnessing a paradigm shift where patient care is moving away from a high touch activity, to a technology enabled impersonal activity, just like social media “friends” replaced or augmented one’s “real” friends. If that’s the case, EHR is just the tip of the iceberg.
I don’t know if this is good or bad. I admit it makes me uncomfortable, but I seriously doubt that any of us can alter the course at this point, and I don’t know that we should. Who is to say that our ways are better than what’s to come…..
I do agree with Leonard, and to QA’s point, I understand that usability is quickly becoming a science and rightfully so. Human Factors Engineering has been around for decades, and since my previous life was spent in the Aerospace industry, I know how valuable HFE can be.
However, I have a hard time accepting that ideas and thought process developed for manufacturing of products is easily ported to enterprise software. Also from the NIST testimony:
“In developing our guidance to measure the usability of EHR systems, we are adopting existing best practices, such as the FDA human factors evaluation process; the Army MANPRINT process; Navy Human-System Integration; FAA flight deck certification process; and potentially others, adopted by federal agencies with oversight on system usability.”
As Mark, wrote, I am sure the folks at NIST have good intentions but, what I’ve seen so far is not looking very promising.
http://www.nist.gov/customcf/get_pdf.cfm?pub_id=907312
I also think that even if we assume that NIST or the Government can indeed do a decent job at coming up with something that is useful, like the Microsoft CUI, it should still refrain from imposing that on the market. The MS CUI would not look too good on an iPad for example and this is a fast changing industry.
Unless the Government decides to build an EHR and provide it for free to everybody (which is a valid approach), these decisions are best left to the marketplace, with the caveat of appropriate safety standards and oversight. It seems, Leonard, that this is the main driver in the AMA position anyway.
[tried to post all the links pertinent, but it won’t accept them, so I’ll do it one at a time, as we comment]
First some house keeping. Not sure why, but the hyperlinks are not in the post, so here they are:
ONC Hearing – http://healthit.hhs.gov/portal/server.pt?open=512&objID=1473&&PageID=17117&mode=2&in_hi_userid=11673&cached=true
NIST Form – http://www.nist.gov/customcf/get_pdf.cfm?pub_id=907312
UI drawing – http://www.nist.gov/customcf/get_pdf.cfm?pub_id=907316
Eric Burke – http://stuffthathappens.com/blog/2008/03/page/3/
I do agree with Leonard, and to QA’s point, I understand that usability is quickly becoming a science and rightfully so. Human Factors Engineering has been around for decades, and since my previous life was spent in the Aerospace industry, I know how valuable HFE can be.
However, I have a hard time accepting that ideas and thought process developed for manufacturing of products is easily ported to enterprise software. Also from the NIST testimony:
“In developing our guidance to measure the usability of EHR systems, we are adopting existing best practices, such as the FDA human factors evaluation process; the Army MANPRINT process; Navy Human-System Integration; FAA flight deck certification process; and potentially others, adopted by federal agencies with oversight on system usability.”
As Mark, wrote, I am sure the folks at NIST have good intentions but, what I’ve seen so far is not looking very promising. I also think that even if we assume that NIST or the Government can indeed do a decent job at coming up with something that is useful, like the Microsoft CUI, it should still refrain from imposing that on the market. The MS CUI would not look too good on an iPad for example and this is a fast changing industry.
Unless the Government decides to build an EHR and provide it for free to everybody (which is a valid approach), these decisions are best left to the marketplace, with the caveat of appropriate safety standards and oversight. It seems, Leonard, that this is the main driver in the AMA position anyway.
Puzzling thought the government deciding what good EHR/EMR design looks like, and what UX will mean to the physicians actually using it. Shall I hint to the proverbial camel being a horse designed by a committee?
That said, I do believe that EHR design is possible; however, physicians would also do well, keeping and open mind and accepting that most things aren’t perfect when first released. One commenter mentioned Apple, and as much as I am an apple fan, I must admit, that even Apple sometimes fails to deliver, iTunes and AppleTV being pretty good examples.
What’s likely driving the ONC: http://www.ama-assn.org/amednews/2011/04/25/gvse0429.htm
Usability must be defined by users. Although we may have some idea of Apple or other products that get high marks for usability or user experience (UX), that measure doesn’t come from some outside body. It comes from people who use the product. (BTW, I give Apple high marks for Mac, iPhone and iPad, their hardware and OSs, but I despise AppleTV and iTunes, because the rarely allow me to get the job done I want done).
The best UX books (I highly recommend effectiveUI’s book: http://oreilly.com/catalog/9780596154790) out there describe a process for delivering what users want, and it doesn’t have anything to do with simple or shiny per se (altthough some users want that). It has to do with finding the most effective method for getting users where they want to go without hesitation or confusion. Different users will want different things.
Needless to say, I don’t think government can define this, but I do think they could take a sampling from users and here what they have to say. Either way, though, probably a well-intentioned bad idea. Whatever’s measured, it likely won’t be usability, and then measuring something is much worse than measuring nothing because it wastes a lot of time and effort without getting anyone closer to the goal of better usability.
A better solution would be to find out how to drive the right environment for UX by preventing lock-in. Software will have better UX when it can compete on UX.
“NIST has created a testing procedure and just like its Meaningful Use testing procedures, it is superficial and doesn’t really test anything of any consequence.”
…from NIST testimony at ONC policy committee hearing:
“In response to this challenge, NIST is developing an EHR Usability Protocol or EUP that provides the detailed specification of an objective, repeatable procedure for measuring and evaluating the usability of HIT systems. The goal is to establish formal technical methods for measuring usability that will uncover critical usability issues that may impact performance and errors in EHR use based upon known human factors principles, processes and best practices. Simply put, applications should be designed to eliminate the causes of error – including human error. On a practical level, this means that we deliver a process that describes guidelines on usability factors associated with critical safety issues, and technical evaluation protocols for ensuring EHR developers have a framework to understand how usability affects critical safety issues in patient care.”
Nothing of consequence, eh?
This is a noble effort to measure and evaluate usability. However, it is an impossible task since it involves the intersection of human perception, behavior and computer software. I wish them good luck but I don’t think they will come up with anything useful. (Any they may very well come up with something incredibly stupid.)
Over the past few years I have moved from Windows to Linux to Mac computers. I found the most difficulty with the Apple Mac interface. It may be more “intuitive” to some but it’s been quite a learning curve. Linux is a lot like Windows so was very easy. Macs “think different” and are difficult to learn if you are used to Windows and Linux. The point is that it’s hard to measure UI “objectively”.
Sounds like this definition of usability could have the exact opposite effect toward what most people would define as usability.
I think you laid out the issues and a potential answer to Dr. P’s question quite well in this thorough discussion.
I think the government should regulate core elements (data definition and interoperability standards as well as security and reliability by establishing standards). However, as you point out, it is difficult to assess UI design so it would be best to leave this to the market.
Your car analogy is actually good (everything should be able to be reduced to a car analogy). The government mandates safety systems such as seat belts, lights, air bags, etc.) and manufacturers must certify that they meet these standards. However, the design of the car is left to the manufacturer. The government spot tests cars to see that they meet the standards but not every model of every car.
Thanks for this insightful article.
Another excellent post, but I want to know how you responded to Dr. P’s question!
pcp,
I simply answered “no” to Dr. P. There is no way today to be able to dictate directly into an EMR and review results and write orders or scripts simultaneously. I also told him that what he did in less than 2 minutes will probably take more like 5 minutes in a good EMR. Since his next question was why should I do this then, I went into a lengthy explanation of the current and future benefits to his staff, patients and to a lesser degree himself.
Just to clarify, he wasn’t shopping for an EHR. He is getting a “free” one from the hospital and just wanted to know what the future holds.