HIStalk Quotes
Does Cerner Millennium kill children? I don't think so.
An HIStalk reader sent me the full Pediatrics article, "Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System." The discussions it will generate make it worth a careful review.
Children's Hospital of Pittsburgh (CHP) and the University of Pittsburgh studied patient mortality before and after CHP's 2002 Cerner Millennium implementation. They looked only at those patients who were transported into their facility for specialty care (as logged in their transport database,) not the general patient population. Deaths went from 2.80% pre-CPOE to 6.57% after (although the number of deaths actually went down -- it was total number of patients decreasing a lot that raised the percentage.) [UDPDATE: thanks for the correction -- I failed to notice that the study period wasn't evenly divided on either side of the go-live, so the number of deaths went down a little only because the time period went down a lot.] In terms of risk factors, regression shows that Cerner CPOE was the second best predictor of death, behind only shock and just ahead of severe coma. That conclusion is seemingly at odds with another study from the same hospital showing that CPOE reduced adverse drug events, which would presumably have decreased mortality.
First, I commend the authors for looking broadly at the most important outcome: survival. Too many studies focus on phenomena that are observable but not necessarily meaningful: user satisfaction, turnaround time, response to reminders, etc. The ultimate measure is whether you leave the hospital alive. The second most important is whether you leave better than when you came in. The third is how you fare long-term compared to how you would have done without your visit. Everything else is nothing more than a contributing factor to one of those three.
While the study found a positive correlation of deaths pre- and post-CPOE implementation, I would be critical of these facts, all of which the authors readily admit:
- Positive correlation doesn't necessarily mean cause and effect.
- By the nature of looking at pre- and post-CPOE date ranges spanning 18 months, the patient population would have varied by season, by infection trends, by referral patterns, use of newly introduced drugs and technology, a change in house staff, reallocation of staff to the implementation, and other time-dependent variables. It is likely that the first nine months had a lot of differences than the second nine beyond just the Cerner implementation.
- Organizational change naturally accompanies any large-scale clinical system implementation, making it difficult to separate the results of the application itself vs. results of changes needed to use the application, execution of which is the responsibility of the hospital and not necessarily the vendor.
- The hospital used Millennium in the ICU, an area for which it is not necessarily suitable (although I doubt they were discouraged by Cerner during the sales cycle.) Since the most seriously ill patients are ICU patients, that isn't necessarily a fair test of Cerner, although it may be a good warning to other hospitals considering use of any general CPOE application in the ICU.
- The study period was brief given the radical changes that a CPOE implementation introduces. If mortality goes up even short-term during implementation, that's important, but to extrapolate beyond those few taxing months is dangerous. CHP closed the study due to new requirements for resident working hours, which would have invalidated it anyway.
The article does mention some issues with the implementation that they feel may have impacted outcomes. It's interesting and a good lesson for those who follow them (with Cerner or any other vendor,) but it doesn't necessarily support any conclusion about CPOE in general or Cerner in particular. Those observations were:
- Cerner CPOE takes a lot of navigation and clicks. I'm reading between the lines to assume that they believe it's a static, not overly userly friendly GUI, a valid criticism of every major clinical systems vendor that I've seen (usability and human factor engineering is still a largely ignored discipline in healthcare IT vendorland.)
- The hospital says a single order took 1-2 minutes to enter, although they mention that their wireless network was inadequate and often caused computer lockups. They report that it took two physicians to stabilize the patient, one to direct care and the other to enter orders. I don't know that Cerner or any other vendor would claim that CPOE is faster than barking orders to a nurse or scrawling on an order sheet, so this isn't surprising.
- Physician to nurse communication got worse because each group worked alone at their workstations. That would have been the hospital's choice or perhaps an unavoidable decision given available space.
- The pharmacist can't see CPOE medication orders until the nurse activates them. That's a bad idea by someone, either Cerner or CHP.
- The hospital brought Millennium live in ICU without a single ICU order set in place. Someone was really hot to go live to make an implementation error of that magnitude. Not Cerner's problem.
- Millennium didn't allow order entry until the patient is admitted, delaying orders that were previously written while the patient was in route to the hospital. Is this a Cerner limitation or an implementation decision? The article says the policy "later was rectified," so it sounds like something under the hospital's control.
- Nurses could not grab ICU meds until the pharmacist verified the order (which in turn followed the nurse's activation of the order.) This must have been the hospital's choice. I'm not sure that even ardent CPOE supporters would advocate following the traditional CPOE workflow in a near-emergent situation or location. One might also point to JCAHO's requirement for pharmacist order review, which may have influenced the hospital's decision to make pharmacist order review a critical path task not necessarily to the patient's benefit.
- Millennium locks everyone else out of the patient while the pharmacist is processing the order. I'd guess that's a Cerner architecture problem for which there is little excuse in modern systems (which Millennium really isn't, although being 8-10 years old is cutting edge in our industry.)
- Medication turnaround time got a lot worse after CPOE. Not surprising, given the new bottleneck of order review and pharmacy workflow adjustments to accommodate it.
- Doses got off schedule due to CPOE's enforcement of standards, which is both a blessing and a curse of CPOE systems and inattentive physicians. Dose scheduling, particularly when orders are changed, is difficult and often poorly designed.
I'm glad they did the study, but it seems to me more of a "don't do what we did" lesson for hospitals, not an indictment of Millennium. I think their purpose was to raise the awareness of broad outcomes in a major system change and the article does a good job of that.
I'm always quick to jump in a vendor-bashing line, but anyone who sees this article as valid Cerner criticism is wrong, in my opinion. I doubt any other vendor would have done better. I doubt every Cerner implementation has these problems. I'm sure that the hospital would make better implement decisions if they were doing it over again, as would most CPOE early adopters.
The only serious takeaway I got from the article was this: be careful out there, vendors and customers alike. No one wants to kill anyone with software or software implementation, especially children. Changing processes is hard work. Magazines, consultants, and vendors make it all seem easy. It's not.
> Since the most seriously ill patients are ICU patients, that isn't
necessarily a fair test of Cerner
Fantastic, a few comments over on my blog but you said it all.
In layman's terms...seems to me that the bottom line is that computer
systems don't kill people. Mistakes (and poor practice) do. There are no
simple correlations between physicians entering orders on computers and
improved patient safety and quality of care. I have read the article and
its presentation is quite misleading, as it appears to place the
responsibility for increased mortality on CPOE. Indeed, correlations do
exist, but they are dependent on a plethora of factors, not the least of
which are system design, functionality, physician optimization based on the
facility (including best practice-based guidelines and protocols) and, of
course, implementation.
Millennium locks everyone else out of the patient while the pharmacist is
processing the order. I'd guess that's a Cerner architecture problem for
which there is little excuse in modern systems (which Millennium really
isn't, although being 8-10 years old is cutting edge in our industry.)
---
To help clarify, if a Pharmacist is working on a patient other providers
can still access patient result information and non-medication orders. The
reason for the lock-out is to prevent multiple providers from entering
conflicting medication orders that would in-effect bypass all system safety
checks.
Well said. I commend the authors of the study for bringing this to light.
I agree that the study is not necessarily an indictment of Cerner but a
careful lesson.
I've sent a comment to Mr HIS Talk about this, but I'll mention is here:
Usability is just not worth the investment for a Healthcare IT vendor,
sadly. It takes months, if not years, to re-engineer a workflow or UI. This
is because HIT software is very complex and large amounts of people are
involved. Pieces of the software is written by a team in a different
fashion (think Style, not language or platform) than other groups. It's
inevitable. But in order to have consistent workflow and usability, it
often involved re-coding a large bulk of existing code, which Might
actually cause defects if you do this.
While I agree with you that not all can be blamed on the Cerner software,
you have not taken into account that during the implementation process,
Cerner should be acting as a consultant to help a healthcare organization
make decisions that optimize the use of their software. Things like a lack
of a pre-reg procedure, or no pre-defined emergency transport order sets,
should have been something Cerner could guide the customer to have in
place. Aren't they advertizing "clinical transoformation" - this cannot be
done with software alone
I don't understand why you noted this: "(although the number of deaths
actually went down -- it was total number of patients decreasing a lot that
raised the percentage.)"
To ACR's observation...can one vendor be qualified to optimize a CPOE
system across an entire enterprise? Doesn't seem practical, to me.
Clinical transformation is a rather grandiose claim, given the current
levels of specialization. Putting integration issues aside for a moment,
would it not makes sense to implement department- specific solutions that
have been designed to both optimize physician utilization and
promote/ensure best practices in that area, rather than depend on the
software and consultation of a single vendor?
I couldn't disagree with you more on the departmental specialization
comment, Kahuna. I think that the level of speciailization within
healthcare has cause much of problem. Granted that in a hospital, there is
a good balance of the specialized vs. the basic day-to-day care but it is
all based on similar core processes. I think there is plenty of room for
some flexibility in the user interface and flipping configuration rules
"on/off" to streamline flow but systems need to operate on the same set of
rules to ensure that specialists, who often operate as if they only see
their end of the world, and other care providers who don't take the time or
who aren't provided with multidisciplinary views of data, are taking into
account all of the facets of the patient (remember, this environment will
only grow in complexity with the increasing numbers of aging patients with
greater numbers of co-morbidities). I believe that over-specialization
will kill the industry. It will minimize the opportunities for
cross-system alerts, reminders, data mining, pathways, and the like because
each system will have its own data model, its own way of defining rules,
its own controlled vocabulary, and so on. There is too much value in
having the clinical data and the rules in one place, at one time, with one
and only one definition. Great Discussion!
The currently available EMRs are atrocious. They have terrible user
interface and are riddled with counterintuitive menus. There is NO
communication between the vendors of software and the users. The software
should be held to high standards just like medications are. Until the EMRs
can be certified safe in clinical trials, it is better to scan documents
into a file and review them on workstations. Ther is nothing wrong in
getting all the information you need with one or two clicks! In Emergency
rooms all over the coutry, it is now the trend to use a very detailed
histroy generation program which spews out about 9 pages of garbage for
each simple ED visit. It includes a lot of stuff that might not be done as
"default notes" it is hard tomake out the critically important areas of
history by glancing at the chart. Why do they allow this kind of junk to
influence any aspect of patient care?
Our IDN is implementing EMR/CPOE across eight sites. One of the physician
liaisons suggested a renaming of the article:
“Expected increased mortality after prematurely implementing CPOE without
adequate preparation.”
Computers don't kill people, people (or the decisions of people) and nature
kill people. Ok a bit of an off color statement, but there are clearly
issues raised in this study. It seems to me that maybe the data is missing
osme facts. If you actually read the entire article you see some
potentially dramatic holes in the implementation process. For one the
training (3 months earlier and only a few hours), also it's all about "the
click" -- too many steps and mouse clicks and you're clearly going to make
it harder. Steve Krug said it best in his book "Don't Make Me Think."
Usability, training and correct implementation (and implementation support)
are crtical to any system.
This topic is of particular interest in England, where the National
Programme for IT (NPfIT) is planning to implement a range of core IT
systems in the UK NHS.
ACR and Kahuna make interesting points and it should also be noted that
Cerner does have a consulting firm that implements the Millennium system. I
would assume, however, that the customers can take it or leave it. It would
be interesting to know if Cerner's consulting group were involved here and
if not, who implemented the system?
It is indeed interesting that everyone finds reasons to dismiss a fine
study that finds faults with CPOE...but no one questions studies that find
improvements, even if those studies are methodologically weak. Studies
with mixed findings are ignored.
Computers don't kill people, but Cerner may drive its users to! My
experience is that despite the massive hardware it requires, Cerner HNAM is
very sluggish and the middleware constantly crashes (I personally think
their design is overly complex, although they may have had no choice). The
user interface does require a lot of clicks, the tab key doesn't always
work, and different parts of the suite of programs were obviously designed
by different people who don't talk to each other. Did I mention crashes?
The client software crashes all the time ...and not much is done to ensure
transaction integrity when this happens.
DO NOT WASTE YOUR TIME/MONEY ON CERNER.
One thing that everyone here in their comments has failed to address, is
the fact that no matter the vendor, no matter how many consultants, no
matter the best practice advice given, any institution implementing CPOE or
any other application makes their own decisions on how they are going to
design, build and use the system. In my personal experience implementing
various applications at multiple facilities, no matter the vendor, no
matter the application, no matter the advice, suggestions, best practice
direction that is given, it still all comes down on an institutions own
decision to take that advice or proceed with what they feel is their own
best practice designs.
Despite many issues in implementing CPOE from any vendor, physician
sponsorship is the key to success. We have heard many reasons why this will
not work, but no suggestion of alternative solutions to safe, readable,
clear medication orders. Residents have been much more open to change and
adopting CPOE practices.
When your ICU nurse spends all her time typing on a computer screen instesd
of caring for her/his patient people can fare very poorly. With Cerner
excellent experienced nurses were in tears as they couldn't do their jobs.
Is this progress?
I am a critical care RN who has worked extensively at the bedside with many
various clinical applications- at least 5 or 6- some are bad and some are
very good. We need to look closely at what the goals are (or should be) of
a clinical application: Allow clinicians to QUICKLY and EASILY view
pertinent clinical information about their patients, allow clinicians to
document the care given to their patients- meds, orders, assessments,
procedures, and historical data; the application should be quick and easy
to access and reliable, and most importantly, it should not delay the care
and treatment of the patient! Shared data is hallmark to the management and
care of a critically ill patient. Clinical decisions are made on the basis
of this data and the data documented by multiple clinicians about the care
already provided. Unfortunately, CERNER meets none of these. Comparatively,
when I work at another institution which uses a much better application,
the differences are night and day. The system compliments my job and care
provided, a quick and reliable tool with the information I need at my
fingertips quickly, not hinder and delay me from it. When I must document
meds, orders, assessments, vitals signs, etc, it does not delay me from
getting back quicly to the bedside and providing care. In my experience
with CERNER, care is delayed as the clinician must labor through the
user-unfriendly application, must wait for delays related to the slowness
of CERNER, must repeat documentation because of kick outs- every minute of
which takes away and/or delays care, treatment and/or vigilant watch of the
patient. Multiply those minutes by each patient a physician must see or
multiply them by the number of times an ICU RN must touch the system in a
12 hour shift and you can begin to understand how those precious minutes
add up and take away from the care of the patient. Frankly, it's no wonder
mortality had gone up.
