Uncategorized

EHR Usability

A few days ago, I wrote about Innovation, a term being overused in the EHR industry to the point where it lost all meaning. Here is another such term: Usability

Just like Innovation, Usability is the weapon du jour against the large and/or established EHR vendors. After all, it is common knowledge that these “legacy” products all look like old Windows applications and lack usability to the point of endangering patients’ lives. On the other hand, the new and innovative EHRs, anticipated to make their debut any day now, will have so much usability that users will intuitively know how to use them before even laying their eyes on the actual product. With this new generation of EHR technology, users will be up and running their medical practice in 5 minutes and everybody in the office will be able to complete their tasks in a fraction of the time it took with the clunky, legacy EMRs built in the 90s. And all this because the new EHRs have Usability, not functionality, a.k.a. bloat, not analytical business intelligence and definitely not massive integration, a.k.a. monolithic. No, this is the minimalist age of EHR haiku. Less is better, as long as it has Usability.

Usability, according to the Usability Professionals Association, is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ISO 9241-11]”. Based on this definition, it stands to reason that any EHR prospective buyer should want a product with lots of Usability. Everybody wants to be effective, efficient and satisfied. So how does one go about finding such EHR?

Well, as always, CCHIT picked up the glove, and as always, CCHIT will be criticized for doing so. The 2011 Ambulatory EHR Certification includes Usability Ratings from 1 to 5 stars. The ratings are based on a Usability Testing Guide. Jurors are instructed to assess Usability of the product during and after the certification testing based on three criteria: Effectiveness, Efficiency and the subjective Satisfaction, as required by the ISO standard.  The tools for this assessment consist of 3 types of questionnaires:

  • After Scenario Questionnaire (ASQ) –jurors rate perceived efficiency (time and effort), learnability, and confidence after viewing scenarios

4 questions after each scenario –16 overall

  • Perceived Usability Questionnaire (PERUSE)–jurors rate screen-level design attributes based on reasonably observable characteristics

20 questions divided among each of the scenarios;

  • System Usability Survey (SUS) –jurors rate the assessment of usability, and satisfaction with the application

10 questions after all four scenarios have been demonstrated

The questions range from general subjective assessments in the ASQ, to very specific inquiries in PERUSE, like whether table headers are clearly indicative of the table columns content. Following the certification testing, results from all jurors are combined and weighted with more weight to specific answers and less to subjective overall impressions. The final result is the star rating, ranging from 1 to 5 Usability stars.

As of this writing, 19 Ambulatory EHRs have obtained CCHIT 2011 certification and all of them have been rated for Usability presumably according to the model described above. Of those, 12 achieved 5 stars, 6 have 4 stars and 1 has 3 stars. Amongst the 5 stars winners, one can find such “legacy” products as Epic, Allscripts and NextGen. The 4 and 3 stars awardees are rather obscure. So what can we learn from these results?

The futuristic EHR movement will probably dismiss these rankings as the usual CCHIT bias towards large vendors. Having gone through a full CCHIT certification process a couple of years ago, I can attest that the only large vendor bias I observed was in the functionality criteria, which seemed tailored to large products. Big problem. However, the testing and the jurors seemed very fair and competent. Looking at the CCHIT Usability Testing Guide, I cannot detect any bias towards any type of software. I would encourage folks to read the guide and form their own unbiased opinions. Are we then to assume that the 5 Stars EHRs have high Usability and therefore will provide satisfaction?

I don’t have a clear answer to this question. Obviously these EHRs have all their buttons and labels and text conforming to the Usability industry standards, and obviously a handful of jurors watching a vendor representative go through a bunch of preset tasks on a Webex screen felt comfortable that they understand and could use the system themselves without too much trouble. Many physicians feel the same way during vendor sales demos. However, efficiency and effectiveness can only be measured by repetitive use of the software in real life settings, for long periods of time and by a variety of users. Measuring satisfaction, the third pillar of Usability, is a different story altogether. There isn’t much satisfaction about anything in the physician community nowadays and when one is overwhelmed with patients, contemplating pay cuts every 30 days or so and bracing for unwelcome intrusion of regulators into one’s business, it’s hard to find joy in a piece of software, no matter how  well aligned the checkboxes are.

The bottom line for doctors looking for EHRs remains unchanged: caveat emptor. The footnote is that the bigger EHRs are as usable as the Usability standards dictate, just like they are as meaningful as the Meaningful Use standards dictate and when all is said and done it is still up to the individual physician user to pick the best EHR for his/her own Satisfaction.

Margalit Gur-Arie is COO at GenesysMD (Purkinje), an HIT company focusing on web based EHR/PMS and billing services for physicians. Prior to GenesysMD, Margalit was Director of Product Management at Essence/Purkinje and HIT Consultant for SSM Healthcare, a large non-profit hospital organization.

Livongo’s Post Ad Banner 728*90

Categories: Uncategorized

Tagged as: ,

41
Leave a Reply

41 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
21 Comment authors
Richard Castropcpsmart patientJay Rosen, MDbev M.D. Recent comment authors
newest oldest most voted
Margalit Gur-Arie
Guest

You are obviously correct, Richard. there is no turning back and we shouldn’t look for a way back either. We just need a good and responsible way to move forward.
The more folks get on the software, the faster it will improve because there is nothing more effective than customer input and customer complaints.
However, since other than inconveniencing early adopters, these software products can adversely affect patient care, we also need a well regulated way of moving forward.

Richard Castro
Guest
Richard Castro

Margalit, You made some very intereting points in your blog. What most people don’t realize is that EMR software has been around for more than a decade, but that most healthcare professionals have not used it as of yet. Now that EMR software is becoming mainstream, healthcare professionals are beginning to analize,and critize it every chance that they get. Like always no one likes a change, especially healthcare professionals, and they most realize that this change is inevitable. They should learn to use the software that their facility has to its fullest, and realize that better software (updates, etc) will… Read more »

Margalit Gur-Arie
Guest

I’m working my way through the supplement from the bottom up, but one sentence caught my eye: “Future PCMH recognition and certification processes should focus more on patient-centered attributes and the proven, valuable key features of primary care than on the features of disease management and information technology.” There have been studies before that showed indication of what they called “high touch” having higher correlation with outcomes than high tech had. I never really understood why the PCMH criteria were so heavily tilted to EHRs. I do understand the basic need for having a comprehensive record stored at the MH,… Read more »

pcp
Guest
pcp

Margalit:
Yes, there is value in data collection and population management, but those are very expensive activities and cannot/should not be allowed to interfere with one-on-one patient care.
Did you see that the AAFP just released the independent evaluation of the medical home National Demonstration Project? In a word: disaster.

Margalit Gur-Arie
Guest

pcp,
Your description of what passes as CDS is indeed useless and unfortunately all too prevalent.
If we assume, just for the sake of this discussion, that reimbursement is available for both just talking to the patient and data collection, would you see any value in population management? Would registries help in managing a large panel of folks with multiple chronic conditions?
What do you think about Dr. Wagner’s chronic care model?

tsuris
Guest
tsuris

The devices are impediments to safe cost effective care.
They are clinical disrupters with inexpicable idiosynchracies that impede comparison of multiple data points at the speed with which the clinicians’ mind operates. They are disruptive to the creativity that is required to evaluate and manage multi-organ disease in patients on multiple medications. They are good for sandbox care, eg a simple one problem surgical situation, but otherwise, rigidly require workflow adjustments that are counter intuitive to effective care. They facilitate errors.

pcp
Guest
pcp

Margalit: “However, I am looking at all the requirements for “meaningful use” and most things in there seem to go to patient care, such as registries, CDS, sharing information with patients and other providers of care.” When I say patient care, I mean me sitting in the exam room, face to face with the patient. I see nothing in the meaningful use standards that would improve that interaction. All I see is a tremendous administrative load that means I need to see more patients to cover the overhead! To be more specific, my day typically begins with seeing five or… Read more »

bev M.D.
Guest
bev M.D.

I don’t think any physician who gets his patients’ lab results and radiology reports off the hospital computer (which is most physicians these days) can claim EHR’s are mostly made for billing. But I worked mostly in the lab, so what is it you clinicians really dislike – is it CPOE? Having to document your notes in the computer? What? Just come out and say it. Now physician office systems are another issue and one I know nothing about. But again, I would ask the doc commenters to be specific about what they don’t like. EHR’s have multiple different components.… Read more »

Margalit Gur-Arie
Guest

pcp, you are making a point frequently made by many docs (EHRs are designed for optimizing billings, not patient care).
However, I am looking at all the requirements for “meaningful use” and most things in there seem to go to patient care, such as registries, CDS, sharing information with patients and other providers of care. Even recording of orders and receipt of electronic results can be viewed as an attempt to improve patient care.
So what would you, or anyone else that cares to comment, consider a worthwhile feature in an EHR, a feature that will improve patient care?

Dr. Seker
Guest
Dr. Seker

Tsuris and pcp are spot on. These devices should be recalled until there is proof of efficacy and safety. The data at present prove no better outcomes and no cost savings.
Equipment is being sold illegally. Best not to purchase.

pcp
Guest
pcp

The problem with EHRs is that they’re not designed to improve patient care: they’re designed primarily to deal with coding, billing, documentation, insurance issues, data collection, and so on.
Good products sell themselves. Bad products are subsidized and mandated.

tsuris
Guest
tsuris

In response to your questions, Margalit, “So, tsuris, let’s take this a bit further. How do you propose the FDA test these software tools?” HIT caused cases of failed care and lost specimen, lost results, hidden results, death and other, as reported on MAUDE and in Florida are frequent and give the equipment a rightfully deserved “black eye”. The user is not in error, but is, instead, bewildered and ostracized by the unusable equipment. There needs to be candid in situ evaluations of CPOE and EMR equipment spanning the training, implementation, complications, defects, flaws, cognitive disturbance, usability, and chronic efficacy… Read more »

Margalit Gur-Arie
Guest

Dr. Rosen, What I find most disturbing about the Florida incident is the following quote from Dr. Rosenberg: “This year we have seen as many if not more medical records violations from electronic medical records as we saw from hand-written records violations.” As to good EMRs, there are several small EMRs, some written by their users, some commercial, that consistently manage to satisfy customers. I just reviewed one on my blog http://bit.ly/d5D1NG and I was fairly surprised by the scarcity of complaints. Maybe because these small software packages have no pretenses of saving the world or reducing the US deficit.… Read more »

smart patient
Guest
smart patient

If this is true–“I also think that the FDA has a role in ensuring patient safety.”
Margalit, why have the HIT industry’s devices avoided any and all scrutiny?
Were the authorites duped by the C$HIT certification or were there greenbacks being exchanged. The government is supposed to protect the patients, but the government has been commandeered by the self interested “intellectuals” from HIMSS and C$HIT amd others with a revolving door between ONCHIT, HHS and elsewhere. Is not N. deParle tagged to Cerner?

Jay Rosen, MD
Guest
Jay Rosen, MD

Margalit:
You said: “I don’t think all EMRs out there are bad, some are actually pretty good, but this indiscriminate forced shopping spree will inevitably create large numbers of dissatisfied customers.”
What proof do you have for this statement?
Was the EMR involved in the missed uterine cancer case in Florida a “pretty good” EMR? See HIStalk report on this.