Dr. Weisman, who joined TidalHealth in 2018 as its Chief Medical Information Officer (CMIO), is also the health system’s VP of Information Technology, Chief Information Officer (CIO). He is an internal medicine specialist with over 25 years in healthcare and extensive experience advancing technology to improve quality of care and cost-effectiveness, and reduce waste and unnecessary workloads. He has had numerous leadership roles in clinical management, strategy and development, analytics, and informatics. He received his medical degree from George Washington University School of Medicine in Washington, DC and his MBA from the Isenberg School of Management at the University of Massachusetts. Dr. Weisman is double board-certified in internal medicine and informatics, is a certified healthcare CIO and chief digital officer, and has two graduate-level certificates from the Harvard University Extension School in cybersecurity and healthcare artificial intelligence.
When sitting with members of the board of directors or senior leaders in the organization, you may hear statements like, “We need to be an AI first organization”. I interpret that to mean we want our people working at their highest level, and we should offload the mundane tasks to the computer. However, the question we all must ask is whether this technology lives up to the excitement. It is very cool that I can ask Generative AI (GenAI) in a natural language to draw me a picture of a CIO pulling a rabbit out of a hat, but does it drive business value? As CIOs we need to balance the challenges between bringing the latest and greatest to our organizations versus the need to defend against vendor hype and financial pressures. We naturally want to push our organization out of the dark ages and away from manual data entry, but we also wonder when the hype of GenAI will live up to its potential. This article explores that balancing act and highlights some of the considerations CIOs must make.
One of the technologies I believe best exemplifies these challenges in healthcare is ambient AI. We have all experienced the doctor’s visit where the clinician has their head in the keyboard pecking away because they never learned to type without looking at the keys. Those days are coming to an end. Now, the doctor enters the exam room, hits record on a cell phone app, and has a normal conversation with their patient. At the end of the encounter, the generative AI creates the physician’s note without the need for the doctor to ever touch the keyboard. Similar tools now exist for many other business verticals. For those of us with enough grey hair to remember documenting on paper, we remember the days of having eye contact with patients and when the electronic health record invaded our exam rooms, both patients and providers felt the impact. Ambient AI technology has been called a game changer in terms of restoring the doctor-patient relationship. However, is there any return on investment? At prices that range from $90-450 per provider per month, the cost to a health system employing hundreds or thousands of doctors is significant. Most of us have found that providers are not willing or able to add more patients into their day to cover the cost of this technology, nor are we seeing significant improvements in patient throughput related to this technology. So where is the business value?
This technology raises interesting questions about the value of a “soft ROI”. Does improving a physician’s sense of wellbeing have a measurable ROI? Does having the latest technology improve physician retention or help with recruiting? Does the quality of care improve if the doctor can focus on the conversation and not the capture of various data points to justify a certain level of billing? These questions remain unanswered, but healthcare CIOs are in the position of having to decide whether to invest in these tools or whether to sit on the sidelines to see how the market shakes out.
Other GenAI tools that have sparked interest include the plethora of chatbots that are now on the market to assist with office productivity. Personally, I have found these to be useful, but how much use they are getting across the organization and whether they drive business value remains unknown. For example, there are Excel formulas that I do not use often and rather than watching a YouTube video on the topic for 10 minutes and then trying to fumble through it, I can ask the AI to take a crack at it. I would say it is about 50/50 in terms of getting it correct. Has it improved my productivity? Maybe. Since I do not track my productivity by the minute, it is difficult to substantiate the investment. At $30 per user per month, the risks of a Copilot license are pretty low, until we start to scale these investments across the organization. Many CIOs are running these small pilots to see if lightning strikes, but I am not hearing about consistent improvements in productivity with the introduction of these tools. The gains appear to be episodic, where the GenAI does something great for a user, but then sits idle for days. Perhaps the gains are there to be realized, but we have not figured out how to put them into our workflow. Or perhaps there is a learning curve we need to scale before we see the business value. Either way, the ROI on these GenAI productivity tools also remains elusive.
I believe the use cases will continue to grow, and as CIOs we need to continue to explore the adoption of GenAI. In most of these use cases, the ROI will not be crystal clear, so consider using three measuring sticks to consider if you should dedicate your limited resources to a new GenAI tool.
- Market disruption: Are there enough “soft” benefits to lead you to believe the technology will significantly change your industry? With ambient AI, the feedback from the clinicians is that they cannot live without it, so healthcare CIOs are fully backing this technology. In situations like this, either lead your industry with a pilot or be a fast follower with a strong commitment to adoption. Being late to recognize an industry-changing technology has career risks for CIOs.
- Pilot program potential: Is the cost reasonable where small pilots have low enough risk that they won’t impact the larger organization? There is a large difference in time and money between a project that takes terabytes of your unstructured data and dumps it into a large language model so you can ask the generative AI natural language questions about it versus putting 25 spreadsheets into a model to better understand how well the tools can help end users understand the data. The former will take dedicated resources away from other projects, whereas the latter is less than a day’s worth of work. Minimize your exposure on these projects until the ROI becomes clear.
- Talent development: Will the adoption of the technology lead to more people in your organization exploring AI use cases, learning about AI models, developing their own models, or providing feedback to senior leaders on the pros and cons of these tools? Elevating the AI literacy of your organization will pay future dividends, even if the current tool does not have an immediate impact on the revenue or expense statements. There is a limit to how much an organization is willing to spend on talent development, but as the technology matures you will want to have basic infrastructure in place to take advantage of the advancements. Internal expertise and superusers are critical to eventual widespread adoption, and some degree of investment is worth the time and money involved.
On one side, there will be pressure from vendors pushing the latest GenAI models, board members who heard about the amazing things AI can do at some seminar, and enthusiastic staff members who are well intentioned, but perhaps not aware of the challenges with enterprise deployment of such tools. On the other side there is the risk of being left behind, caught unaware of market disruption, and with a talent pool poorly equipped to adapt to changes. Being a CIO requires finding that balance between those extremes and understanding the organization’s risk tolerance. It is a moving target in a rapidly changing field, so be prepared to reassess frequently.