Everyone, including this blog writer, has been touting the virtues of the vast troves of data already or soon to be available in the electronic health record (EHR), which will usher in the learning healthcare system [1, 2]. There is sometimes unbridled enthusiasm that the data captured in clinical systems, perhaps combined with research data such as gene sequencing, will effortlessly provide us knowledge of what works in healthcare and how new treatments can be developed [3, 4]. The data is unstructured? No problem, just apply natural language processing .
I honestly share in this enthusiasm, but I also realize that it needs to be tempered, or at least given a dose of reality. In particular, we must remember that our great data analytics and algorithms will only get us so far. If we have poor underlying data, the analyses may end up misleading us. We must be careful for problems of data incompleteness and incorrectness.
There are all sorts of reasons for inadequate data in EHR systems. Probably the main one is that those who enter data, i.e., physicians and other clinicians, are usually doing so for reasons other than data analysis. I have often said that clinical documentation can be what stands between a busy clinician and going home for dinner, i.e., he or she has to finish charting before ending the work day.
I also know of many clinicians whose enthusiasm for entering correct and complete data is tempered by their view of the entry of it as a data blackhole. That is, they enter data in but never derive out its benefits. I like to think that most clinicians would relish the opportunity to look at aggregate views of their patients in their practices and/or be able to identify patients who are outliers in one measure or another. Yet a common complaint I hear from clinicians is that data capture priorities are more driven by the hospital or clinic trying to maximize their reimbursement than to aid clinicians in providing better patient care.
Continue reading “The Data Entry Paradox”
Filed Under: THCB
Tagged: Data Entry, documentation, EHR, Informatics, Usability, William Hersh, Workflow
Oct 4, 2012
Dr. P patted the middle aged patient on the back, helped him off the elevated exam table and guided him to the chair by the sink. He picked up the chart and using the exam table as his desk he flipped through the chart, pulling out several pieces of paper, spreading them to his right, while making small talk with his patient. He reached into his pocket and pulled out a battered silver recorder and without any warning started dictating: “Mr. H is a 60 year old mildly obese gentleman presenting with…..“.
He had a pen now in his right hand, and as he was talking into his recorder, shuffling the various papers in front of him, he was also writing orders and prescriptions as fast as he was dictating. “….follow up in two weeks” was the last thing he said. He didn’t write that one down, but turned around, handed the patient a bunch of scripts, told him to stop by the front desk and make an appointment two weeks out and stop by the lab on the fourth floor to pick up a container for the urine test. Two minutes, tops, including the small talk. It was my turn now and I was sweating bullets because I knew exactly what he is about to say. “Can I do this in the EMR?”
EHR usability has finally arrived to Washington as the guest of honor at the most recent ONC HIT Policy Committee hearing. ONC seems to be considering the regulation and certification of EHR usability. NIST has created a testing procedure and just like its Meaningful Use testing procedures, it is superficial and doesn’t really test anything of any consequence. Those who represented “providers” and patients argued for the need to improve usability and those who represented academia and grant funded research argued for more funded research. Predictably, usability experts, argued for hiring more usability experts. Large vendors eloquently stated their objections to government mandating what EHRs should look like and small vendors argued that the more mandates, the better, since this will automatically remove the built-in competitive advantage of those with larger budgets and larger usability departments. As is customary, EHRs were compared to ATM machines, cars, iPhones, Google and a variety of “other industries” that are all so much more advanced than health care when it comes to usability. Continue reading “(Over)Simplifying EHR Usability”
Filed Under: Electronic Health Records, Physicians
Tagged: EHR, Margalit Gur-Arie, Usability, VistA
May 2, 2011
A few days ago, I wrote about Innovation, a term being overused in the EHR industry to the point where it lost all meaning. Here is another such term: Usability
Just like Innovation, Usability is the weapon du jour against the large and/or established EHR vendors. After all, it is common knowledge that these “legacy” products all look like old Windows applications and lack usability to the point of endangering patients’ lives. On the other hand, the new and innovative EHRs, anticipated to make their debut any day now, will have so much usability that users will intuitively know how to use them before even laying their eyes on the actual product. With this new generation of EHR technology, users will be up and running their medical practice in 5 minutes and everybody in the office will be able to complete their tasks in a fraction of the time it took with the clunky, legacy EMRs built in the 90s. And all this because the new EHRs have Usability, not functionality, a.k.a. bloat, not analytical business intelligence and definitely not massive integration, a.k.a. monolithic. No, this is the minimalist age of EHR haiku. Less is better, as long as it has Usability.
Usability, according to the Usability Professionals Association, is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ISO 9241-11]“. Based on this definition, it stands to reason that any EHR prospective buyer should want a product with lots of Usability. Everybody wants to be effective, efficient and satisfied. So how does one go about finding such EHR?
Well, as always, CCHIT picked up the glove, and as always, CCHIT will be criticized for doing so. The 2011 Ambulatory EHR Certification includes Usability Ratings from 1 to 5 stars. The ratings are based on a Usability Testing Guide. Jurors are instructed to assess Usability of the product during and after the certification testing based on three criteria: Effectiveness, Efficiency and the subjective Satisfaction, as required by the ISO standard. The tools for this assessment consist of 3 types of questionnaires:
- After Scenario Questionnaire (ASQ) –jurors rate perceived efficiency (time and effort), learnability, and confidence after viewing scenarios
4 questions after each scenario –16 overall
- Perceived Usability Questionnaire (PERUSE)–jurors rate screen-level design attributes based on reasonably observable characteristics
20 questions divided among each of the scenarios;
- System Usability Survey (SUS) –jurors rate the assessment of usability, and satisfaction with the application
10 questions after all four scenarios have been demonstrated
The questions range from general subjective assessments in the ASQ, to very specific inquiries in PERUSE, like whether table headers are clearly indicative of the table columns content. Following the certification testing, results from all jurors are combined and weighted with more weight to specific answers and less to subjective overall impressions. The final result is the star rating, ranging from 1 to 5 Usability stars.
As of this writing, 19 Ambulatory EHRs have obtained CCHIT 2011 certification and all of them have been rated for Usability presumably according to the model described above. Of those, 12 achieved 5 stars, 6 have 4 stars and 1 has 3 stars. Amongst the 5 stars winners, one can find such “legacy” products as Epic, Allscripts and NextGen. The 4 and 3 stars awardees are rather obscure. So what can we learn from these results?
The futuristic EHR movement will probably dismiss these rankings as the usual CCHIT bias towards large vendors. Having gone through a full CCHIT certification process a couple of years ago, I can attest that the only large vendor bias I observed was in the functionality criteria, which seemed tailored to large products. Big problem. However, the testing and the jurors seemed very fair and competent. Looking at the CCHIT Usability Testing Guide, I cannot detect any bias towards any type of software. I would encourage folks to read the guide and form their own unbiased opinions. Are we then to assume that the 5 Stars EHRs have high Usability and therefore will provide satisfaction?
I don’t have a clear answer to this question. Obviously these EHRs have all their buttons and labels and text conforming to the Usability industry standards, and obviously a handful of jurors watching a vendor representative go through a bunch of preset tasks on a Webex screen felt comfortable that they understand and could use the system themselves without too much trouble. Many physicians feel the same way during vendor sales demos. However, efficiency and effectiveness can only be measured by repetitive use of the software in real life settings, for long periods of time and by a variety of users. Measuring satisfaction, the third pillar of Usability, is a different story altogether. There isn’t much satisfaction about anything in the physician community nowadays and when one is overwhelmed with patients, contemplating pay cuts every 30 days or so and bracing for unwelcome intrusion of regulators into one’s business, it’s hard to find joy in a piece of software, no matter how well aligned the checkboxes are.
The bottom line for doctors looking for EHRs remains unchanged: caveat emptor. The footnote is that the bigger EHRs are as usable as the Usability standards dictate, just like they are as meaningful as the Meaningful Use standards dictate and when all is said and done it is still up to the individual physician user to pick the best EHR for his/her own Satisfaction.
Margalit Gur-Arie is COO at GenesysMD (Purkinje), an HIT company focusing on web based EHR/PMS and billing services for physicians. Prior to GenesysMD, Margalit was Director of Product Management at Essence/Purkinje and HIT Consultant for SSM Healthcare, a large non-profit hospital organization.
Filed Under: Electronic Health Records, Margalit Gur-Arie
Tagged: CCHIT, Usability
Jun 4, 2010