New AHRQ-Funded Report Provides Snapshot of Electronic Health Record (EHR) Vendor Usability Processes and Practices

Picture 4

The usability of EHR systems, while recognized as a critical factor in the successful adoption, safe and   effective use of these systems, has not historically received the same level of attention as software features, functions and technical requirements. In recognition of the importance of this issue, the Agency for Healthcare Research and Quality (AHRQ) initiated a series of research activities focused on assessing and improving the state of usability in Electronic Health Record (EHR) systems.

Based on a research gap identified by a multi-disciplinary expert panel formed to recommend and prioritize research and policy actions in this area, AHRQ funded a follow-on project to gain insight into the processes and practices that certified EHR vendors employ during different phases of the product development to make their products usable.

Specifically, AHRQ contracted with James Bell Associates and the Altarum Institute to conduct a series of structured discussions with selected certified EHR vendors to understand their processes and practices with regard to:

  • The existence and use of standards and “best practices” in designing, developing, and deploying products
  • Testing and evaluating usability throughout the product life cycle
  • Supporting post-deployment monitoring to ensure patient safety and effective use
  • Vendors perspectives on the role of certification in evaluating and improving usability

While all vendors interviewed for the report expressed a deep commitment to the development and provision of usable EHR product(s) to the market, vendors described an array of usability engineering processes and the use of end users throughout the product life cycle, practices such as formal usability testing, the use of user-centered design processes, and specific resource personnel with expertise in usability engineering are not common.

Because nearly all vendors view usability as their chief competitive differentiator, collaboration among vendors with regard to usability is almost nonexistent. Specific best practices and standards of design, testing, and monitoring of the usability of EHR products are not readily available. Vendors reported use of general (software) and proprietary industry guidelines and best practices to support usability. Reported perspectives on critical issues such as allowable level of customization by customers varied dramatically.

Surprisingly, many vendors did not initially address potential negative impacts or harms caused by their products as a priority design issue. Vendors reported a variety of formal and informal processes for identifying, tracking, and addressing patient safety issues related to the usability of their products. Most vendors reported that they collect, but do not share, lists of incidents related to usability as a subset of user-reported “bugs” and product-enhancement requests. While all vendors described a process, procedures to classify and report usability issues of EHR products are not standardized across the industry.

No vendors reported placing specific contractual restrictions on disclosures by system users of patient safety incidents that were potentially related to their products.

Disagreement exists among vendors as to the ideal method for ensuring usability standards, and best practices are evaluated and communicated across the industry as well as to customers. Many view the inclusion of usability as part of product certification as part of a larger “game” for staying competitive, but also as potentially too complex or something that will “stifle innovation” in this area. To overcome competitive pressures, many vendors expressed interest in an independent body guiding the development of voluntary usability standards for EHRs. This body could build on existing models of vendor collaboration, which are currently focused predominantly on issues of interoperability.

Key recommendations from the project’s expert panel echo and build on their policy recommendations from the initial AHRQ-funded project in this area

First, the panel recommends establishment of usability / information design of EHRs as an essential part of the certification requirements for EHRs, basing certification on a practical and fair process of usability evaluation, and designing certification programs for EHR usability in a way that focuses on objective and important aspects of system usability.

Findings also reinforced the prior recommendation to require/strongly recommend that vendors establish and document their programs for testing the usability of their systems (people and processes), including evaluating potential impacts on quality and safety by finding ways to encourage vendors to address key shortcomings that exist in current processes and practices related to the usability of their products. Most critical among these are lack of adherence to formal user-design processes, dependence on post-deployment review supporting usability assessments, and a lack of diversity in end users involved in the testing and evaluation process. Potentially under-sampled populations include end users with limited past experience with health information technology and those with disabilities (only one vendor mentioned efforts to make their products accessible for healthcare professionals with disabilities).

The lack of voluntary collaboration among vendors supports the panel’s prior recommendation for a “National EHR Usability Laboratory” or independent body to support public-private efforts to overcome market forces that discourage collaboration, development of best practices, and standards harmonization in this area. Such a body could also support development of best practices around EHR usability / information design customization during system deployment and use.

Finally, the panel recommended the development of tools and processes to support evaluation of the usability and information design of EHR products and their implementations. AHRQ is working closely with the National Institute of Standards and Technology (NIST) and the Office of the National Coordinator (ONC) to address this and other recommendations identified in its research. In June, AHRQ plans to award a follow-on project for the development, testing and dissemination of an easy-to-use, objective and evidence-based toolkit that healthcare organizations can use to evaluate critical aspects of their EHR systems’ usability, accessibility and information design. In addition, NIST is currently seeking applications for development of an EHR usability evaluation framework and ONC, AHRQ NIST and other agencies are collaborating on a joint conference in July to promote collaboration in health IT usability among government, industry, and academia to prioritize, align and coordinate short, medium, and long term strategies and tactics to improve the usability of EHRs.

Categories: Uncategorized

15 replies »

  1. This diet pill can help you lose 2-5lbs a week, depending on the amount of work you put in. This physical activity range from walking to running to any outdoor game and others. Hence, if you’re looking for amazing taste and health benefits at the same time, you could always eat the fruit as a whole.

  2. Also the introduction of REC’s through the HITECH act. is a great way to avail of quality EHR solutions at competitive prices. The stiff competition among not only these REC’s but also among EHR vendors ( to become a preferred vendor of a given REC) will result in lot of positives to medical practioners.
    Looking the funding provided to the REC’s, the staggered grant allocation system also promises to be an unbiased way of allocating funds. It will also help in the concept of REC’s helping out each with their own unique business models. It can be one of the possible answers to the
    ’safe vendor challenge’ as discussed by many critics.

  3. @Truth Seeker, MD
    Like the majority of those posting, I am a bit skeptical of how an NIST conference to discuss the issue will foster real change in usability. That said, I will be speaking at the July 13th meeting and sharing some of my experiences implementing an EHR in a primary care practice. I hope I am not the only clinician speaking that day.

  4. Hi,
    There are lot of great resources on this topic of ARRA or getting Federal incentives through proper implementation of EHR’s or putting EHR’s to meaningful use (http://www.waitingroomsolutions.com/wrs/arra-stimulus-money-44k-arra-emr-stimulus-bill-arra-ehr-stimulus-incentives#meaningful_use_EHR).
    Also I found a great link http://www.waitingroomsolutions.com/wrs/arra-stimulus-money-44k-arra-emr-stimulus-bill-arra-ehr-stimulus-incentives#Certification_Criteria_EHR….on genral guidelines that need to be followed in EHR implementation.
    Do post replies, if and when you need more information

  5. It is close to criminal that the federal government didn’t take the information archetecture in the VISTA (VA) system, make it platform neutral, and establish it as a core option for a (defacto) standard health information technology infrastructure, say a dozen years ago. There could have been plenty of aftermarket opportunity to build front and back-end products to work with this core, but a viable core information structure for health information interchange would have been established.
    Now we have this tower of HIT Babble, and the government is too gun shy to even identify the bad apples.

  6. We’re on a path in healthcare IT that will perpetuate the very systems that don’t work instead of nurturing new systems that doctors and patients want and need. And many well-intentioned government agencies are major contributors to this problem rather than to the solution. In my opinion, this study exemplifies this danger.
    I think it’s fair to say that everyone agrees physicians and hospitals need and want more comprehensive, better organized, more accessible, and easier to use information about their patients.
    It’s also fair to say that physicians and hospitals have been telling the vendors of EMR systems for years that the systems they have been offering — and are continuing to offer — don’t cut it. They simply have refused to buy what the vendors have been selling.
    How much clearer a message do we need? When only 15% of docs and 30% of hospitals have adopted the systems the vendors have been selling for years, it means the products aren’t meeting the needs of their “market,” namely the docs and hospitals!
    Simply put, the “market” has spoken and is continuing to speak but no one seems to be listening. They clearly want systems that are simpler, easier to implement and lower cost than are available today.
    But now, out of frustration fueled by spiraling healthcare costs and inconsistent outcomes, what are we doing? We’re trying to force physicians and hospitals to adopt the very systems they have been rejecting for years as inadequate. And who is doing the forcing? The government prodded by well-intentioned-but-misguided industry observers. (The voices of physicians and patients clearly are not being listened to or the programs the government is promoting would have been DOA.)
    Instead of seeking and nurturing innovative approaches to medical record systems that will meet the needs and desires of care providers and patients, they are, by their very acts of seeking standardization, regulation and now funding (under HITECH), entrenching the vendors who haven’t gotten it for years and still don’t get it today. They are perpetuating the problem, not solving it.
    What should government do? Stop funding unnecessary studies and direct funds, including some Stimulus HITECH funds, to help develop new approaches to healthcare IT. Give modest, outright “venture” grants to those who have promising new ideas. And do it fast. Evaluate them as investment opportunities not as bureaucratic grants. Help them develop and test their approaches in the marketplace. Then let the market – the physicians, hospitals and patients – determine the winners and losers. In short, stoke the inventive genius of entrepreneurs. They’ll solve our healthcare IT problems practically, quickly and cheaply.

  7. The idea of a NIST conference on usability that has a list of speakers who do not use the crap systems which endanger patients every day is a hair brained folly. Who are they kidding? The NIST is going along with the government providing a facade to protect from accusations on the waste of $ billions of dollars.

  8. Agree w. Margalit. This is not the right place for certification/regulation. Having said that, the argument that usability is a key differentiator among these systems strikes me as bullshit. It is FAR below “features and functions” and specific clinical applications (e.g. pharmacy, labs, PACS, etc.) in importance to the vendors. These vendors concentrate most of their marketing energy on the “walled garden” approach: trapping their clients with huge “installed base” switching costs. Once they’ve trapped the client, they make money pushing out new and costly updates and system enhancements.
    Usability of these products is TERRIBLE. Ask any of the docs in this blog, or anywhere else. EHR interfaces are more than a decade old, like the systems themselves, and date from Windows 95 style templates. Data is entombed in EHR’s, many clicks below the initial screens. None of these companies have seriously addressed how to enable providers to visualize data more easily. And because no one vendor has made quantum advances in usability an engineering priority, there hasn’t been any real competitive pressure for any individual firm to improve it.
    It’s a scandal, but not something that is going to be assisted materially by a clumsy, inevitably consensus driven regulatory model. This is where the “tyranny of low expectations” on the part of the clients themselves has really worked to the vendors’ advantage.

  9. It is evident from his dreams that Shereef has never had to use these crap CPOE systems. When time critical therapy and interventions are needed, these devices result in death, a secret that the manufacturers go to great effort to hide.

  10. These efforts to standardize and define usability will no doubt shed light on the equivocal evidence that EHR adoption actually improves the quality of health care provision. Intuitively, an easily-accessible, efficient EHR system should reduce medical errors by various mechanisms. We are still waiting to see these results reflected in clinical trials, as they are currently quite modest: http://content.healthaffairs.org/cgi/content/abstract/29/4/639.
    I applaud the efforts of the ONC and AHRQ in their attempts to better define “usability” in EHR systems, now that we know that EHR “rollout” in itself is not necessarily effective.
    Please visit my blog on the business of health care in America:

  11. You are shutting the gate after the horses escaped. This should have been done 10 years ago.
    Despite this, you are delusional if you think that usability equates to safety.
    The certification process is a sham, usability is meaningfully absent, and the devices are of unknown safety and efficacy. The country is being sold a pig in a poke. There are too many of influence who have travelled between government and HIT industry. The truth about adverse events has been covered up.
    Clancy and others of AHRQ are too cozy with those from HIMSS.
    BUT, I like your smile and your tie is on straight.

  12. I don’t think I understand the premise here.
    These are private software companies, competing in the same market space. Why would anybody expect collaboration any more than you’d expect Google to collaborate with Bing?
    As to the usability issue, I am not aware of any other software market where usability per se is subject to government certification, or any other vetting process. Usually, as the vendors indicated, usability is part of a vendor’s competitive advantage, or lack thereof. I don’t see why an official agency should undertake dictating development processes and UI elements for an entire market. I am not aware of any precedent to this and I am not aware of any particular competencies available at a government level which are not available to any software company willing to pay for them.
    If the current crop of software vendors is indeed incompetent, the market will drive them out of business eventually. I’m not convinced that we need to spend even more taxpayer money to help private vendors stay in business. If this is a free market, then let the best software vendor win…. and let the others perish.
    However, we shouldn’t confuse usability with patient safety. I fully support FDA oversight, or something lighter, in order to ensure that software doesn’t harm patients, but that is not necessarily related to usability. Data and display bugs are more likely to affect safety than any usability shortcomings.

  13. Hi Vince,
    A key goal of this report was to gain an understanding of industry processes and practices around usability (vs. the merits and de-merits of processes and practices of individual vendors) to inform systematic recommendations for this industry.
    Vendors made senior product-management and product-development staff available during this extremely busy time on the condition that the report not attribute responses to any particular vendor.
    We would not have had the (voluntary and often candid) participation that we did if we had “named names”…

  14. Matt, When will EHR Usability reports start naming names?
    How would we react to a Consumer Reports article that says “Car A gets great fuel economy” and “Car B rolls over”…but doesn’t identify the cars. What good is that?
    My question isn’t asked antagonistically and certainly isn’t meant to single out you or AHRQ.
    Is this gag clauses at work? Very peculiar.