241121 CHE NigamShah final

Dr. Nigam Shaw's Background and Expertise

• Dr. Nigam Shaw is a professor of medicine at Stanford University and the chief data scientist for Stanford Healthcare, with research focused on safely and effectively integrating AI into clinical use (14s).
• Dr. Shaw has an extensive background, including being an inventor on patents, authoring over 300 scientific publications, and co-founding three companies (27s).
• He was inducted into the American College of Medical Informatics in 2015 and the American Society for Clinical Investigation in 2016 (37s).
• Dr. Shaw holds an MBBS from Baroda Medical College in India, a PhD from Penn State University, and completed post-doctoral training at Stanford University (46s).

Data Quality and Patient Timelines in AI/ML Models

• The quality of AI and machine learning models is heavily dependent on the quality of the data they are trained on, with data being collected from patient timelines (1m33s).
• Patient timelines can include various data types, such as ECGs, blood pressure, respiratory rate, and medication orders, which are used to build models (2m7s).
• In a typical healthcare setting, not all data is collected at every point in time, and there is often a lack of longitudinal coverage for individual patients (2m46s).
• The manipulation and processing of patient timeline data have a significant impact on the performance of AI and machine learning models (3m22s).

AI/ML in Healthcare Decision-Making: Classification, Prediction, and Recommendation

• With large amounts of patient timeline data, models can be built to support decision-making in healthcare, including whether to treat a patient and how to treat them (4m1s).
• The decision of whether to treat a patient can be broken down into classification or diagnosis tasks, or prediction tasks, from a computational standpoint (4m18s).
• In general language, terms such as prognosis, prediction, and classification are often misused, and it's essential to understand the distinction between them, as classification is not the same as prediction, and many things in medicine that masquerade as predictions are actually classifications (4m29s).
• A classification is when a model identifies something that already exists, such as diagnosing pneumonia or identifying a dog in an image, whereas a prediction would imply that the model is forecasting a future event (4m46s).
• The distinction between classification and prediction is crucial, as it affects how we approach treatment and prevention, and it's essential to understand that many models, such as sepsis predictors, are actually classifiers and not predictors (5m13s).
• Recommendation is the hardest task, given the data's limitations and biases, and it's been a 40-year journey in medicine to figure out reliable recommendation systems (5m48s).
• There are three things that can be done with AI and machine learning in medicine: classification, prediction, and recommendation, and it's essential to consider whether these technical exercises are advancing the science of medicine, the practice of medicine, or the delivery of medical care (6m13s).
• An example of advancing the science of medicine is the discovery of three subtypes of heart failure with preserved ejection fraction, which would be a classification that advances our scientific understanding of the disease (6m42s).
• An example of advancing the practice of medicine would be developing a test that can identify the subtype of heart failure and provide a treatment plan accordingly, which would improve patient outcomes and reduce costs (6m58s).

Advancing Medical Science, Practice, and Delivery with AI

• An example of advancing the delivery of medical care is the "Green Button Project," which aimed to query similar patients' data to inform treatment decisions at the bedside, and it has been successful in improving patient outcomes and reducing costs (8m1s).
• A bedside consultation service was developed to provide written reports with recommendations for patient care by analyzing timeline objects from millions of other patients, which helped make better decisions than would have been made otherwise (8m14s).
• A study found that about 80% of the time, physicians did not have prior published data to inform their decisions, and less than 3% of the decisions had a study specific to the question at hand that the physician knew (9m4s).
• The lack of access to prior published data highlights the need to analyze data on demand, which was achieved through a prior project that was later spun out into a company called Atropos Health (9m36s).
• Atropos Health reduced the time it took to conduct bedside studies from a day or two to under 24 hours, and sometimes even a few hours, and with the advent of generative AI, studies can now be done in a few minutes (10m3s).

AI-Driven Healthcare Advancements and Cost Savings

• The use of different technologies such as machine learning, chatbots, and generative AI enables rapid decision-making and can improve patient care (10m37s).
• A relatively simple AI model was used to predict which patients would become medically costly the next year, and proactive action was taken to enroll them in care programs, resulting in an estimated 10-15% cost savings without sacrificing quality (10m54s).
• The use of AI and machine learning can drive advancements in healthcare delivery, including predicting cost, biology, practice, and delivery outcomes, such as no-shows, patient transport, and image classification (11m42s).
• A consistent theme in AI applications is that the AI provides a risk estimate, and the value comes from taking action based on that estimate, such as early intervention in cost blooms or advanced care planning in mortality prediction (12m34s).
• The model stratifies by risk value, and the actual intervention necessary may vary, such as providing transportation support in the case of predicting no-shows (12m51s).
• A three-star logo is used to remember the interplay between the model, work capacity, and the action taken, with the yellow box representing computer science and statistics, the green box representing the number from the model, and the red box representing the action taken (13m10s).
• The interplay between the model, work capacity, and action taken is studied, with about five or six faculty members working on it and publishing around 25 papers on the topic (13m36s).
• The key insight from this research is that the focus should be on what can be achieved given the work capacity, rather than just building a model (13m42s).

The FUR Model and Responsible AI Development

• A plot is used to illustrate the cumulative benefit of taking action on cases ranked by probability, with the goal of determining how far down the list action can be taken before diminishing returns are seen (13m56s).
• The approach used is called Fair Useful Reliable (FUR) models, which is a multi-step process that includes usefulness simulations, financial projections, ethical considerations, and prospective evaluation (14m59s).
• The FUR approach is used on a routine basis in the healthcare system, but it was developed because the current way of doing things is unsustainable, with 220 atomic pieces of guidance on how to do good AI, but only half of them focusing on how to build a model (16m23s).
• A model was developed over 10 years at a cost of $28 million to determine who should receive immediate attention in the emergency department (ED) and who can wait for registration, highlighting the unsustainable nature of current medical research practices (16m45s).
• The current organization of work in medical research is unsustainable, which is why processes like form assessment are needed to make healthcare activities more responsible and sustainable (17m20s).
• A three-step approach is proposed for responsible and sustainable innovation in healthcare: Discovery (solving for the science), Development (validating the intent), and Dissemination (scaling) (17m43s).
• For standard AI, the Discover stage is too slow and costly, and the development stage needs to focus on achievable benefits and financial sustainability, which may require changing business models (18m12s).

Firm Assessment for Responsible AI Deployment

• The firm assessment is a tool used to evaluate the responsible development and deployment of AI in healthcare, and it includes a link to a blog post and a website (fm.stan.edu) for more information (18m49s).
• The firm assessment involves evaluating the workflow, including the steps involved in using a classifier to identify patients with undiagnosed diseases, and ensuring clarity on responsible action (19m8s).
• The assessment also includes an ethics evaluation, considering factors such as equity, reliability, governance, autonomy, and decision-making processes (20m6s).
• A form assessment process is used to evaluate the impact of algorithms on patients, considering factors such as the number of people affected, sustainability, and potential ethical problems, to ensure that projects are fair, useful, and reliable (20m46s).
• The assessment process involves analyzing six cases, with the goal of identifying projects that can benefit a large number of people without causing harm to certain subgroups (20m57s).
• Capacity planning is necessary to ensure that responsible AI can be implemented in a healthcare system, and operational engineering work is done to determine the number of concurrent assessments needed to achieve a certain goal (21m36s).
• Little's law is used to calculate the required team size, which indicates that a team that can handle two assessments at the same time is needed to complete at least one assessment per month (22m12s).
• Good governance is essential to ensure that everything that needs to be done is actually done, and a life cycle is established to make sure that the form assessment is integrated into the workflow (22m35s).
• The workflow consists of four key components: standard work, IT support, governance, and form assessment, which are necessary to make informed decisions about AI projects (23m8s).
• The form assessment process produces numbers and analyses that help the governance body make decisions about AI projects, considering factors such as patients affected, sustainability, and potential harm to certain subgroups (23m20s).
• The goal of the form assessment process is to ensure that AI projects are fair, useful, reliable, and do not cause harm to certain subgroups, and to make informed decisions about which projects to pursue (23m49s).

Large Language Models (LLMs) and Their Application in Healthcare

• The introduction of Large Language Models (LLMs) in 2022 has changed the landscape and raised new questions about the form assessment process (24m2s).
• A language, in the context of computers, is a sequence of tokens from a finite vocabulary, such as a dictionary, and can include natural languages like English, Spanish, and Gujarati, as well as the "EHR language" used in electronic health records (EHRs) consisting of codes like ICD, CPT, and LOINC codes (24m24s).
• The EHR language is a sequence of tokens representing events and actions in a patient's timeline, such as visits, prescriptions, and diagnoses, which can be used to build language models for forecasting and generative AI in healthcare (24m55s).
• There are two ways to build language models: the classical approach using natural language text and documents, and the use of patient timelines to forecast future events and outcomes (25m17s).
• A study was conducted to evaluate the effectiveness of language models, specifically GPT 3.5 and GPT 4, in answering questions from a bedside service, with results showing an increase in agreement and a decrease in disagreement and uncertainty among 12 physicians (26m19s).
• However, the study also found that around 40-50% of the time, physicians couldn't decide whether the language model's answers were correct or not, limiting its utility (26m53s).
• Another project, MedAlign, aimed to assess the alignment of language model outputs with medical needs, with results showing a 35% error rate in answering medical prompts, even in the best-case scenario (27m57s).
• Research is being conducted to train models that can forecast patient outcomes, with a focus on using positive examples to develop forecasting classifiers (28m19s).
• A performance receiver operator curve was used to evaluate the performance of different models, including logistic regression, random forest, and a timeline-trained language model, with the language model showing the highest accuracy (28m35s).
• The language model achieved an accuracy of around 78%, which is better than the highest accuracy achieved by classical methods, and it trains eight times faster and uses 95% less training data (29m16s).
• The models, called Climber and Motor, have been publicly released and can be found on GitHub (29m53s).
• The focus should be on verifying the benefits of language models and generative AI, rather than just building them, as many tech companies are doing (30m9s).
• Academic sites have a crucial role in asking hard questions about the effectiveness of these models, despite not having the same resources as tech companies (30m25s).
• The development and dissemination of a worldview for generative AI is needed, but it is unclear how this will scale (30m38s).

Collaboration between Data Engineers and Data Scientists

• Building fair, useful, and reliable models requires collaboration between data engineers and data scientists, who should work together as part of the same team (31m56s).
• Data engineering and data science are two functions that need to work together, with data engineers playing a crucial role in extracting, cleaning, and preparing data for analysis (32m4s).
• The arrangement between data engineers and data scientists should be collaborative, with decisions made during data cleanup and extraction affecting the kind of science that can be done (32m28s).
• The time it takes to analyze or model data and come up with something presentable depends on the team's maturity, with the first time taking significantly longer and subsequent replications becoming faster, cheaper, and more reliable (32m44s).
• The first end-to-end project took around five to seven years to complete, while the next project took a year and a half, and the third project took four months, with the goal of reducing the time by 50% with each replication (33m14s).

Data Integrity and Model Development in Healthcare

• Electronic Health Record (EHR) data is noisy and contains errors, so it's essential to look for multiple lines of collaboration to confirm any conclusions, such as verifying a diabetes diagnosis with HbA1c levels and medication data (34m2s).
• Stanford is not training its own Large Language Model (LLM) for medical analysis, instead opting to use an open-source model with no copyright issues and fine-tuning it, as training from scratch is expensive (34m37s).

Applications of AI in Medicine: Clinical vs. Non-Clinical

• The most promising applications of machine learning in medicine include operational applications such as transcription, responding to messages, and billing, which tend to have higher adoption rates in academic centers (36m7s).
• In resource-constrained settings, AI may be used for clinical care, such as retinal scanning algorithms for diabetic patients, as the alternative may be no care at all (36m27s).
• The choice of whether to use AI for clinical or non-clinical applications depends on the structural issues of the healthcare system and the environment in which the work is being done (35m29s).
• The circumstances in which technology is deployed can significantly impact its effectiveness, and there are many examples of academic work being applied to solve real-life problems in the healthcare industry (36m52s).
• One such example is the work done by Stanford Healthcare's HMS and HS Davies on predicting mortality for improving Advanced Care planning, which has improved the care of over 6,000 patients (37m12s).

Generative AI, Patient Involvement, and Ethical Considerations

• Generative AI may be applicable in Behavioral Health, but there are concerns about its potential for errors, such as the incident where Gemini told a high schooler to harm themselves while giving homework advice (37m42s).
• Patient involvement is crucial in implementing the FEMR model, and this can be achieved through a patient family advisory council, which reviews the workflow and ensures that patients are comfortable with algorithm-driven care (38m28s).
• Multiple stakeholders, including clinicians, patients, administrators, and developers, should be involved in the development and implementation of algorithms to ensure that they are effective and unbiased (39m23s).

AI Applications in Medical Imaging and Pathology

• Machine learning has practical applications in laboratories, such as in histology, cytology, and flow cytometry, with examples including the use of deep neural nets to sort and analyze cells (39m35s).
• The Pathology Department at Stanford has developed a system called Nuclei, which uses a deep neural net to help pathologists read slides and identify areas of interest (39m50s).
• Other examples of AI in pathology include cell sorters, which use lasers and calculations to analyze cells, and have become a widely used tool in healthcare systems (40m33s).

Challenges and Opportunities with EHR Systems

• The biggest issue with current EHR systems is that there are too many systems, with a typical hospital running anywhere from 500 to 1,000 systems, making it a myth to believe that all medical data is in the EHR, as it is scattered over hundreds of systems (41m19s).
• The biggest gap or opportunity is to combine all the data in one place, as the current system makes it difficult to access and utilize the data effectively (41m55s).

Addressing Bias in AI Models and Healthcare Workflows

• When it comes to removing bias in predictions made by models, there are two interpretations of bias: a systematic difference in the model's performance for people belonging to different subgroups, and a systematic difference in the actual benefit or reward that people receive as a result of the model's output (42m12s).
• The latter interpretation is the one to worry about, as it is dependent on policies and workflows driven by the model's output, rather than just the model itself (42m51s).
• To address this issue, it is suggested to focus on policies and workflows rather than just removing model-side differences, and to consider the human-centered AI Institute's blog post on "when algorithmic fixes failed" (43m25s).

Traceability, Explainability, and Trust in AI Models

• Traceability and explainability requirements can slow down the progress of AI model advancements, but it depends on the purpose of the explainability, which can be for debugging, mitigating outcomes, or establishing trust (43m55s).
• The need for explainability, interpretability, or transparency can be broken down into different scenarios, each with different requirements, and it is essential to understand the purpose of the request to provide the appropriate explanation (44m53s).
• Establishing trust in AI models for medical diagnosis and treatment requires prospective studies to test their effectiveness, similar to how trust is established in conventional drugs like Tylenol, even when their exact mechanisms of action are not fully understood (45m18s).
• Studies have been conducted to compare error rates in medical diagnosis and treatment with and without AI assistance, such as a study by Jonathan Chen, which found that doctors sometimes make mistakes when using AI, possibly due to suboptimal use of the technology (45m59s).
• The study by Jonathan Chen involved giving case vignettes to physicians with and without access to AI, and the results showed that AI alone performed better than doctors with AI assistance, highlighting the need for more research on how to effectively use AI in medical practice (46m4s).

Data Sources for Training AI in Healthcare

• Electronic Health Records (EHRs) can be considered one of the raw materials to teach AI, but they should not be the only source, as sanitized versions of online sources and textbooks are also necessary (47m7s).
• There are many practical Medical Imaging AI models in use, with the FDA approving around 992 AI-related medical devices, half of which are image-based, primarily in radiology and cardiology (47m44s).