Kimberly Denney^*1

¹ Principal,The Advisory Board Company, USA

*Corresponding Author: Dr. Kimberly Denney, Principal, The Advisory Board Company, USA, Tel: 202-266-5600; Fax: 202-266-5700; E-mail:kimberly.denney@gmail.com

Citation: Kimberly Denney (2016) Improving Physician Acceptance of Alerts with Closed Loop Analytics. J Nursing Palliat serv 1: 101.

 Copyright: © 2016 Kimberly Denney, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Received: October 19, 2016; Accepted: November 24, 2016; Published: November 28, 2016.

Previous research identified the average primary care physician sees more than 60 alerts per day requiring nearly an hour of their clinic time to manage. With 90% of those alerts considered unwarranted, improved clinical decision support (CDS) tools are vital to supporting transitions to value based care and the active engagement of providers. A recent quantitative correlation study focused on physician perceptions of computerized physician order entry (CPOE) alerts found a statistically significant affect between elements of alert design and perceived value in improving adherence to accountable care organization (ACO) quality measures. Application of sociotechnical considerations in the design of provider alerts coupled with near-real-time analytics provides organizations assuming greater financial risk with tools to reduce waste and inappropriate care while supporting improvements in care quality. Advancements in alert analytics supports organizations in closing the loop with their providers to achieve a clinical and financial return on their investments in the electronic health record (EHR).

The abstract along with a presentation of related findings from my doctoral dissertation [1] was shared at a Healthcare Informatics Conference in Las Vegas [2]. While conducting quantitative survey based research as part of my doctoral business studies, I was employed by a start-up healthcare information technology company, privately funded and incubated at Cedars Sinai Health System in Los Angeles, CA. My research aligned with product development efforts underway at Stanson Health, Inc. The firm is advancing the art of CDS tools for providers through the development of sophisticated clinical algorithms deployed as provider notifications at the point of care in the EHR with associated, near real-time alert analytics [3].

As U.S. health care reform shifts reimbursement from a fee-for-service model toward reimbursement based on outcomes, ACOs comprise part of the national strategy for achieving health care’s triple aim of providing better and safer care at lower costs [4,5]. For alternative care models such as ACOs to succeed, numerous researchers highlighted the need for more sophisticated healthcare information technology (HIT) systems and tools.

A heat map published by Leavitt Partners in 2014 depicted the beginning of an unstoppable wave of transformation as approximately 500 ACOs were identified. More recent estimates place the number approaching 800 and spanning as many as 23.5 million lives under Medicare, Medicaid and private insurers [6]. Although still in the learning stages, a number of early ACOs achieved notable success in improving quality while reducing cost. Leavitt Partners estimated 105 million people could be covered by ACOs by 2020.

With an understanding of these important forces, I examined earlier published work related to CDS applications. Sittig and Singh (2010) shared a socio-technical framework applicable for health systems deploying CDS in the EHR. CDS viewed both as a tactical and strategic tool is especially suited to serve organizations preparing for down-side risk based agreements. Effectively designed CDS can aid clinicians in making clinically relevant decisions, and it is one of the few proven ways organizations can actually change clinician behavior.

With billions of dollars invested in EHRs in the US, how do we unlock the value of that investment? I chose to examine physician perceptions for the potential of CDS to support better patient care through support of adherence to ACO quality measures. In the past, many CDS tools were used referentially. That is, they required the clinician to stop and exit the clinical workflow to look something up in a journal, textbook, or online resource. By integrating CDS tools in the clinical workflow, we can assist providers to access the right clinical and patient data in real-time supporting more efficient decision making.

Why the emphasis on integrated CDS with an EHR system? [7] published a landmark study in the BMJ. Across several thousand physicians, the authors examined the ways physicians receive information and which method of interceding with information had the greatest impact on decision making. They found that automating decision support in the clinical workflow succeeded as an intervention 75% of the time whereas no other intervention resulted in supporting providers to change their minds.

To examine physician perceptions that EHR alerts might support efforts to improve adherence to reportable quality measures for their ACO patients, I designed a quantitative correlation study. Borrowing heavily from the published literature on alerts and accessing my professional network of HIT experts, I designed a likert-type scaled instrument administered online (via Survey Monkey) with anonymous participation by subjects. The Association of Medical Directors of Information Systems (AMDIS) served as my doctoral research community partner and supported my efforts to recruit survey participation.

 The final instrument was designed with 3 scales, one for each of the three variables in the study. The regression model contained 2 independent (predictor) variables forming a construct for the content of an alert in the clinical workflow of an ordering physician and the timing of presenting the alert during CPOE. The third scale was aligned to the dependent variable. The third construct enabled assessment of the physician’s perception of alert characteristics (content and timing) with ACO quality measure adherence.

The purpose of applying sociotechnical design to complex organizations such as healthcare systems arises from an understanding that harm may result from both human and technical factors [8]. A sociotechnical framework assists software developers in addressing complex systems through an improved understanding of the communication patterns, workflows, and tools required by users across the system [8,9].

My research premise was that understanding physician’s preferences of alert attributes deployed in their workflow would provide software developers fresh insights for improving CDS tools. Sittig and Singh’s [8] sociotechnical framework aligns well with [10] three- phase depiction of an EHR enabled healthcare system characterized by the use of sophisticated HIT systems and tools that improve the quality and safety of care.

A prospective sampling approach was used recognizing the need to survey a homogeneous population. The subjects recruited for the study were physician executives using an Epic or Cerner EHR in a hospital affiliated with an ACO. At 80% power determined with G*Power software, 68 completed surveys were required.

Walden University’s IRB provided approval to test the self- developed instrument in a small pilot with 5 physician executives. I validated the constructs and the instrument through the pilot. The IRB further required I enlist a community partner for recruitment of my subjects. AMDIS assisted by sharing my request for participation with a hyperlink to the online survey from their member list serv. Over a 3week period in June 2015, I recruited a sufficient number of subjects to meet the required sample size.

126 physicians attempted the survey. The first 4 survey questions determined inclusion. Any “no” response among   the first four questions resulted in termination of the survey. The first question was acknowledgement of the informed consent. The next 3 questions confirmed status as MD or DO, ACO affiliation, and EHR use type.

Compared to the .80 alphas I obtained in the pilot, the cronbach alphas degraded to the low 60’s upon completion of the full survey. As this was a new research instrument, lower alphas are acceptable. It is possible that I obtained skewed results in the pilot since the invited subjects were known to me as experts in the field. Additionally, all 5 pilot physician executives were using an Epic Systems EHR whereas 1/3 were of subjects were Cerner Incorporated EHR users in the final study.

At the conclusion of the study, all statistical analysis was performed using SPSS. The overarching research question sought to determine the extent to which sociotechnical factors addressed in the design of CDS software tools affected physician ordering behaviour. The results of the model were significant with R² = .108, F (2,66) = 3.99, p = .023.

Furthermore, a significant correlation was found between independent variable A (alert content) and the dependent variable (quality measure adherence). Therefore, the null hypothesis was rejected: r = .326, p < .003. However, No significant correlation was identified between the independent variable B (alert timing) and the dependent variable resulting in acceptance of the null hypothesis: r = .037, p > .05.

Certain attributes of alert content such as alignment with evidence and specificity to a patient’s clinical and medical status are perceived as beneficial in an alert. Physician nonadherence to alerts due to issues of alert fatigue and information overload is well documented in the literature [11-13].

Previous research identified that primary care physicians saw more than 60 alerts per day requiring nearly an hour of their clinic time to manage [14]. In a follow up survey, physicians claimed nearly 90% of alerts were unwarranted with 2/3 of the physicians reporting they could not manage the volume of alerts they triggered daily [15].

Interestingly, the content related survey questions resulted in high uniformity of responses with physicians agreeing with sociotechnical aspects of the content to be included in a clinical/order based alert. Only 3 subjects disagreed that alerts should be driven from patient specific data contained within the record, alerts should be prepopulated with evidenced based override reasons, and should contain links to patient education materials when available. Surprisingly, nearly 1/3 of subjects disagreed that alerts should include links to supporting evidence. It may be the case that once a provider has been told the provenance for the recommendation comes from a medical society or provider led entity, they no longer see the necessity to access a link to the underlying body of evidence.

 Other findings included a 90% agreement across subjects that alerts should be placed into their workflow to improve their adherence to ACO quality measures. However, the respondents were nearly evenly split in regards to the specific aspects of ideal workflow timing. There was no consensus as to whether alerts should be allowed to pop up as they are triggered in an ordering session or held until the end. For software developers, the sociotechnical dimension related to understanding clinical workflow timing is frequently cited as a significant and complex challenge in the design and adoption of software tools [8,10,16-18].

Physicians were evenly split (for or against) in their preferences for the use of a passive alert notifying them that the ACO patient met criteria such as diabetic, with heart failure, or other cardiovascular and peripheral disease. The study was not powered for a subset analysis by EHR type although a trend toward a difference in response was noted. A few limitations are especially worthy of noting. First, the Workflow Timing Construct was limited to 3 questions in the survey instrument. Second, the Alert Timing Window was narrowed to just the period of Computerized Physician Order Entry (CPOE). Third, all responses were constrained to the Physician’s workflow. Finally, there were no considerations given for workflow alerts for nurses and other care team members or for the use of asynchronously timed alerts.

Recognizing the EHR consolidates vast amounts of patient and clinical data for purposes of supporting better-informed clinical decisions, software vendors unintentionally developed solutions that contributed to fatigue or burnout increasing the risk for patient harm. The research findings suggested a preference for robust clinical algorithms in the design of alerts focused on content containing patient specific factors and the latest evidence based guidance. By focusing alert design on the characteristics physicians viewed as positive, we might design “fatigue proof” alerts coupling them with additional analytic tools to assist in dialing down the alert activity and noise presently viewed by many providers as problematic and non-value add.

With an eye toward making the decision support useful and actionable in the EHR, Stanson Health chose a clean alert design incorporating a brief statement of the evidence based recommendation with a link to the underlying evidence or supporting documentation. Prepopulated over-ride reasons were included so that the physician received credit for following the recommendation when selecting an evidence based reason why the intervention was not possible for the patient as indicated. The research findings suggested these alert design attributes might be well perceived by providers:

•Translating evidence and best practice guidelines into actionable and understandable point of care recommendations

•Ensuring alerts correctly interrogated relevant patient and clinical data before interrupting the physician’s ordering workflow.

•Exploring the asynchronous timing of alert delivery to potentially lessen concerns of alert fatigue.

•Tailoring CDS by treatment venue and specialty with further application of sociotechnical factors

To close the loop on understanding a provider’s behaviour and interaction with alerts, the use of a cloud-based analytics platform can be coupled with EHR embedded CDS [3]. As the physician interacts with the alert, the analytics engine records this information. Subsequently, with a nightly download of order data from the EHR, these interactions are run through additional algorithms that permit the data to be reflected in user configurable dashboards and reports. Several use cases exist for combining closed loop analytics with real-time workflow alerts. At a high level, the health system can determine how a specific alert is performing over time, the percentage of providers who adhere to an evidence based recommendation, and which providers ignore or override the recommendation and why. To fully close the loop, one needs to be able to assist an individual provider in understanding their behaviours in relation to their peers. In the past, a physician might have told his director he did not know why his outcomes were different than his peers. In the future, with more granular alert data accessible we might be able to lend meaningful insight to that discussion. An open question remains whether or not ignoring or overriding evidence based recommendations at a rate significantly different than one’s peers contributes to different patient outcomes.

The significant relationship identified between alert content and physician perception of improved adherence to ACO quality measures suggests the design of CDS software tools might support improvements in the management of high-risk patients enrolled in ACOs. The findings from this research when applied to improving CDS tools might enable organizations with significant investments in HIT to realize clinical and financial benefits through improved ACO clinical and financial performance. Improving the decision support tools for physicians managing ACO patients might support the vision of a technology enabled health care system that transforms clinical data for active decision making for the benefit of health care providers, patients, and their families. In conclusion, there remains much work to be done to better understand user needs and design tools enabling clinicians working in the EHR to improve the quality, safety, and cost of care. These efforts become increasingly important as we hasten down the path of providing payment for healthcare based on value and the outcomes obtained.

1. Denney KB (2015) Assessing Clinical Software User Needs for Improved Clinical Decision Support Tools (Doctoral dissertation, WALDEN UNIVERSITY).
2. Denney K (2016) Improving Physician Acceptance of Alerts with Closed Loop Analytics. Conference Proceedings J. Health Med Inform 7: 3.
3. Stanson Health, Inc. Los Angeles, CA 90024 .
4. Berwick DM (2011) Launching accountable care organizations—the proposed rule for the Medicare Shared Savings Program. New England Journal of Medicine 364.
5. Burwell SM (2015) Setting value-based payment goals: HHS efforts to improve US health care. New England Journal of Medicine 372: 897–899.
6. Leavitt Partners white paper: The Projected Growth of ACOs (2015).
7. Kawamoto K, Houlihan CA, Balas EA, Lobach DF (2005) Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. British Medical Journal 330: 765.
8. Sittig DF, Singh H (2010) A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Quality and Safety in Health Care 19: 168–174.
9. Path AS (2013) Safer path forward for medical device interoperability needed, per new AAMI Report. Environment of Care News, 16: 4.
10. Meeks DW, Takian A, Sittig DF, Singh H, Barber N, et al. (2014) Exploring the sociotechnical intersection of patient safety and electronic health record implementation. Journal of the American Medical Informatics Association 21: e28–e34.
11. Bowman S (2013) Impact of electronic health record systems on information integrity: Quality and safety implications. Perspectives in Health Information Management, 10(Fall).
12. Menon S, Smith MW, Sittig DF, Petersen NJ, Hysong SJ, et al. (2014) How context affects electronic health record-based test result follow-up: A mixed-methods evaluation. British Medical Journal open 4: e005985.
13. Smith M, Murphy D, Laxmisan A, Sittig D, Reis B, et al. (2013) Developing software to “Track and Catch” missed follow-up of abnormal test results in a complex sociotechnical environment. Applied Clinical Informatics 4: 359.
14. Murphy DR, Reis B, Sittig DF, Singh H (2012) Notifications received by primary care practitioners in electronic health records: A taxonomy and time analysis. The American Journal of Medicine 125: 209
15. Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF, et al. (2013) Information overload and missed test results in electronic health record–based settings. Journal of the American Medical Association Internal Medicine 173: 702–704.
16. Perna G (2012) Clinical alerts that cried wolf: As clinical alerts pose physician workflow problems, healthcare IT leaders look for answers. Healthcare Informatics 29: 18–20.
17. Beeler PE, Bates DW, Hug BL (2014) Clinical decision support systems. Swiss Med Wkly 144: w14073. [crossref] 
18. Barnes AJ, Unruh L, Chukmaitov A, van Ginneken E (2014) Accountable care organizations in the USA: types, developments and challenges. Health Policy 118: 1-7. [crossref]