Ep. 200: Human versus artificial intelligence: the contest

00:00:00: Welcome to EANcast, your weekly source for education research and updates from the European Academy of

00:00:20: Neurology.

00:00:31: Our professional intelligence is moving very quickly into clinical medicine, including neurology.

00:00:40: And that immediately arises some important questions especially if we think of its adoption in daily practice.

00:00:47: How reliable it can be?

00:00:49: Can we trust the systems are often difficult to interpret?

00:00:53: how do you use AI and a way that supports clinical care without weakening human responsibility?

00:01:00: To

00:01:00: address the many challenges they used by AI raises EAN has created a dedicated task force.

00:01:07: It is a multidisciplinary taskforce involving neurologists, computer scientists and experts in ethics and regulatory aspects.

00:01:16: This podcast is one of our educational initiatives And it's the two hundred episode of podcast series Today.

00:01:24: we are discussing very provocative topic AI versus human intelligence The contest.

00:01:31: To explore all this, I'm very pleased to welcome Gary Leeming.

00:01:35: Gary thank you for joining us today.

00:01:39: Hi everybody!

00:01:41: Gary is the director of the Civic Health Innovation Labs based at the University of Liverpool.

00:01:48: Formerly he was chief technology officer in the Connected Health Cities program with helping technology and infrastructure for learning health systems as well as investigating distributed ledger technologies for management of health data.

00:02:03: He's an expert in digital innovation and health information exchanges, and is actually working on many

00:02:09: projects.".

00:02:10: So Gary this is clearly the right moment to have these conversations because AI has already influenced research education and increasingly clinical practice as well And in neurology that opens a lot opportunities but also a lot of uncertainty.

00:02:29: When we talk about human versus artificial intelligence in neurology, what are we really comparing?

00:02:36: Are we comparing speed, consistency, accuracy or something broader?

00:02:41: it is preferable probably not to compare human vs.

00:02:44: Artificial.

00:02:45: What do you think about?

00:02:48: I think that artificial intelligence There's been a lot of debate about the name artificial intelligence, right?

00:02:54: I mean it is from technical perspective.

00:02:59: we understand that.

00:02:59: It kind of...it is computer or machine with does a task that appears intelligent as though he was done.

00:03:09: human like intelligence but it is artificial.

00:03:12: there are not actual intelligences so they're sat behind in way that would be understanding.

00:03:17: and Though the term artificial intelligence conjures up images of robots and Star Trek computers, things that mean we impute our consciousness to it.

00:03:32: But if you stick with a technical definition... ...it is set-up tools help us reason or understand something in front Interpreting an image or collecting together information from a mass of clinical records, or diagnoses.

00:03:52: Or test results and then presenting An output that we can use to make a decision A clinical decision about the patient.

00:04:02: Then it can help us across those areas you were talking about.

00:04:05: It could be faster, more accurate within speed as well really also drive forwards our capabilities in terms of being able to be better at delivering care through all those areas.

00:04:26: So it's going to be incredibly important, I think how we can integrate this new sets tools into clinical practice.

00:04:38: But just to be cautious about what it actually means I think in terms of actual practice and how we do come to rely on It is definitely one of the big unsolved questions.

00:04:48: Yeah, you're totally right And that's a very helpful place too.

00:04:53: start because in medicine at the word better can me Can mean many different things?

00:04:58: A system can be faster but does not automatically make here its wiser in clinical practice.

00:05:05: So this is an important distinction to have in mind, and I would like now go a little bit deeper.

00:05:14: One of the most common claims about AI Is that it can detect patterns on scale.

00:05:19: you know human can match?

00:05:21: AI can manage huge amount data.

00:05:24: How important are they?

00:05:25: quantity but also quality of their distribution and harmonization.

00:05:35: I think that quantity of data often depends on the particular type of AI or machine learning you're talking about.

00:05:44: You can still achieve some great results with minimal amounts of information, but quality is absolutely a key factor in how we make AI useful.

00:05:58: if we have unreliable data and if it has gaps within it The AI can't help us to necessarily explain those or understand those because it can only use what is knows, how's.

00:06:10: It cannot kind of generate new knowledge in that way.

00:06:12: and I think thats one other key issue.

00:06:15: so you know the biasing data for example can be manifested in ways that are unexpected.

00:06:24: but also we need to think about What Is The Nature or Type of Data We Have.

00:06:31: An obvious example from the last few years has been systems that analyze images of skin for tumors and like.

00:06:43: Those algorithms were trained on paler skin, did not work with darker skin so they're missing.

00:06:50: diagnoses would have been obvious to a human being who could've translated them And I think it's important to recognize therefore that when we're developing this, what has been the source of those data?

00:07:01: How are we curating those data and how will be investing in making sure that we address such types of challenges.

00:07:08: With large language models for example you know they've being trained on a corpus of languages particularly from internet but how does that translate people with English-language systems People who use poor English idioms in their language, how are we training it for different types of languages and making sure that you have the validation.

00:07:30: So there's a whole set questions around absolutely about How do make sure data is appropriate?

00:07:43: Yes, and this point is extremely important because it raises the problem of The issue of generalizability and reliability of a system.

00:07:52: So how can a neurologist know when an AI system is reliable or not?

00:07:57: That the system is performing well in some areas of healthcare?

00:08:01: but what should be The tips we have to remind to be cautious enough to trust blindly any tool?

00:08:11: Yeah, I mean it's it comes down to as well.

00:08:16: How have we purchased the system?

00:08:17: However these how every tested a system how is it sort of implemented?

00:08:23: How was it?

00:08:24: How has integrated into The clinicians workflow?

00:08:30: basically and there's this whole set of challenges around just things like you know, how do We do that?

00:08:40: And i think It's recognising with some of these tools, we need that learning health system approach.

00:08:46: We need the complete integration and testing in validating our outputs and work constantly to be able really gain that trust into new tools.

00:09:03: but in gaining that trust as well you can recognise there are biases in human behaviour things like automation bias, where especially for younger clinicians who don't have the experience or expertise of people who've been working three years they would maybe take an output from a machine learning algorithm and trust it more.

00:09:27: Or not just themselves in the face what appears to be an expert system?

00:09:32: And there may be some mistakes carried through that.

00:09:36: so we As I said, both develop that sort of learning health system approach.

00:09:43: So to kind of integrate a constant validation testing but also train our clinicians to question and understand how use the outputs from these tools more effectively?

00:09:58: That point is really critical because sometimes models can look very strong and impressive at technical points.

00:10:08: it can perform much less well when we're moving to a clinical practice as the skin imaging you were telling before.

00:10:20: And once we talk about trust, we inevitably come to The Black Box problem.

00:10:25: and major concern is this black box issue because if an AI system reduces our accommodation but the clinician cannot really understand how its got there This creates a contrast attention, can we realistically make this system interpretable enough for clinical use?

00:10:45: Because clinicians want to know why the system makes decisions.

00:10:51: Why it gives us suggestions and what is the level of transparency that needs trust.

00:11:02: I think some AI systems are more transparent than others.

00:11:07: But there's no doubt that, especially with the newer more complex deep learning models and large language models.

00:11:14: That we're seeing systems have become very difficult to explain how they did get to the answer that they got too.

00:11:23: However were also saying system so you can ask them How do come up without?

00:11:26: And then kind of start explaining it.

00:11:28: see if I could look at resources on their is a thing definitely need two really look at how we expect these systems to really break down those decision points that they got too.

00:11:45: Some of the black boxness is intentional I think in terms, you know a company has an IP wrapped up into their algorithm and so there are decisions making.

00:12:02: Others are kind of being much more transparent about how those systems can work.

00:12:07: But it's definitely the case that this is going to become a bigger problem over time with, particularly when you look at things like new agentic pipelines or sort-of frameworks where you have lots of AI working together... It becomes even harder to explain which one made decision on any particular point in time.

00:12:29: and so yeah This is an active area of research, it's something that we need to do recognise and we do need to resolve.

00:12:38: But at the same time.

00:12:40: I think some of this also needs to just recognise if the output is correct... ...and its always correct….

00:12:48: Do We Need To Know?

00:12:49: What what a level of trust can actually put into decision making even as clinicians?

00:12:54: how.. How Can We Feel Confident That It Is Working?

00:12:59: And that kind of comes back again to the sort of idea or validation output, showing it can be trusted because its working.

00:13:08: We don't necessarily understand always how a drug works in terms and mechanism but all we may uncover is side effects are actually positive impacts on another condition.

00:13:22: so there's different Ways of looking at this as a problem that I think we kind of need to investigate and understand what where are those guidelines or that?

00:13:31: We can construct it make the safe-to use.

00:13:35: And trustworthy enough through the demonstration of it without us having to understand exactly how he always gets to those answers.

00:13:43: however, We do have his problem for hallucination right because we know especially with the language models and when things are getting wrong.

00:13:51: would you have some mention problems of automation bias?

00:13:54: who got these sort of questions, are our clinicians being trained to use these algorithms and question them appropriately?

00:14:01: I'm not sure that younger clinicians would necessarily have the same level of skepticism.

00:14:06: To an algorithm.

00:14:07: maybe they oughta have.

00:14:08: so this is also something we do start thinking about how we teach it.

00:14:14: Okay thank you Gary.

00:14:16: actually...I can tell you that a clinician wants to understand clinical reasoning.

00:14:22: The problem isn't only if there's right answer a clinical problem, but to understand the pathway to arrive at solutions for patients.

00:14:35: So we would like to know why an LMS system generates an answer in that way or another and what is the extent of reliability?

00:14:46: because this problem brings another question about responsibility.

00:14:52: If AI contributes to treatment decision, for example who is responsible in the end when something goes wrong?

00:15:01: The clinician and institution, the developer of the AI system... I'm not sure that our current clinical culture shared the responsibility between AI and humans, in practice does a responsibility still remain entirely to the physician or not?

00:15:27: I think i completely agree with you.

00:15:30: In terms of where do... can we even split that accountability?

00:15:37: Can we ever split that?

00:15:39: Ultimately it's always the clinician who has to be able putting it crudely, looking at how this is working.

00:15:50: Things like self-driving cars and the ethicists that are working on some questions around even just something like that.

00:15:57: as a question asked to who is responsible should a self driving car get into an accident?

00:16:05: This has done problem absolutely multiplied.

00:16:09: we can't answer any of those questions right now.

00:16:12: I guess yes, so putting it crudely is kind of who gets sued.

00:16:16: Right?

00:16:16: Who ultimately will be held legally accountable for all this?

00:16:21: and if we're going to expect clinicians to use us i think We do need thinking about what levels are protections that we can put in for clinicians using these systems?

00:16:31: but also again equals down to a clinician always feel more comfortable making the decision.

00:16:38: they understand absolutely.

00:16:42: How do we design the systems so that they are explainable is definitely a focus for where we need to go.

00:16:52: I think there's a large language model, you know?

00:16:54: You can ask it to explain but... Can you trust its explanation?

00:16:58: and then there's this whole level of like what- Is it reasonable?

00:17:04: or say oh i did that um..I don't if have seen an article A while ago, I think it was a writer talking to an AI.

00:17:16: And the AI would say something and then write to us saying but that's not true!

00:17:23: The AI would apologize and say yes you've caught me, I lied... But here is another answer.

00:17:30: That one will also be false!

00:17:34: There are whole set of conversations now.

00:17:36: what has happened since?

00:17:37: the AI's have been improved and that as a problem has been recognized, worked on.

00:17:43: And so they're becoming more... It is becoming easier to ask them to explain their reasoning and to trust them.

00:17:49: but what is the point at which we can actually trust him?

00:17:52: How does he define his?

00:17:53: one of the big

00:17:54: questions?".

00:17:55: As I mentioned with the self-driving cars there are lots work on sort.

00:17:58: think our ethics colleagues really kind start to unpick and tease apart.

00:18:05: The self driving car example was an interesting one because Do you know when the first self-driving car drove from across America, one side of one coast to another?

00:18:14: No I did it.

00:18:19: Nineteen ninety five.

00:18:21: Oh!

00:18:21: Yeah exactly right but we still don't have self driving cars everywhere now some that is.

00:18:27: technology was fairly primitive and basic.

00:18:30: But someone who has those questions off Who's responsible if a robot taxi That taxi is ordered by the passenger to stop somewhere illegally and gets out, then that passenger is injured by another car.

00:18:47: Who's responsible if... The instruction is to protect the passenger?

00:18:51: And the brakes fail and it drives into somebody crossing a

00:18:56: road?"?

00:18:57: Those are the same questions I think we do have to answer But we don't have simple answers to and we have to find a way to turn that into law as well.

00:19:07: Which is another layer on top of that, And We Don't Have Any Of That In Place.

00:19:11: So How Do We... That's A Years Long Thing!

00:19:16: I think in the meantime we've got this push To Say AI Everywhere.

00:19:19: You Know Let'S Generate Millions Of Pounds Worth Or Billions Of Dollars or Euros worth Of Investments In Health AI.

00:19:28: But we haven't solved a problem of what happens when things go wrong, because that doesn't generate nature papers and it doesn't create the import from venture capitalists.

00:19:40: You know there's needed but this is needed if you're going to make change.

00:19:44: so There is tension between the two.

00:19:49: I just don't know If anything other than slow incremental development and integration of these tools.

00:20:01: And to be fair, going all the way back to the invention of the stethoscope you know doctors rejected the sthetoscope at first because they didn't understand it right?

00:20:11: Now this is way more complicated obviously.

00:20:13: uh um and it's understandable.

00:20:15: why-why are you being nervous about it?

00:20:17: but I think we can get to a point where we feel comfortable when have that evidence base.

00:20:21: It is about that building the evidence base, it's about building a learning health system approach to sort of saying we need to capture data.

00:20:32: We need test it and validate it so they recognize how its being used.

00:20:37: And through that That approaches not just trust AI but let us use human expertise.

00:20:46: So what do you think this is doing?

00:20:50: As well as trying to deal with it from a technical perspective of making you better at explaining itself.

00:20:56: So that's yeah, sorry

00:20:59: I totally agree and i think your example in comparison between earth scale and automatic driving is really strong and clear for our listeners today.

00:21:12: And probably this suggestion if u agrees we can.

00:21:18: we have to find a right working distance between AI and humans.

00:21:23: AI can support decision-making, yes but does not replace completely professional accountability.

00:21:31: And I would like you to mention in your recent paper which compares the human's and AI to Schopenhauer's Hedgehog dilemma.

00:21:44: This dilemma describes a group of hydrox on the whole day.

00:21:48: They move closer to share worms, but their spines hurt each other so they go away.

00:21:53: if they stay too far apart in trees or get too close we get wounded and over time learn about workable distance.

00:22:02: Close enough for benefit Far enough to avoid harm.

00:22:05: I think this is good metaphor for relationship with.

00:22:08: humans are not artificial intelligence because we do like speed efficiency and support in our daily work.

00:22:16: But

00:22:16: the closer we get, the more will risk being pricked by hidden bias, the skilling dependency manipulation so... We really need a calibrated closeness to get benefit.

00:22:31: Do you think it's

00:22:34: right distance?

00:22:35: I think that's a great analogy actually.

00:22:36: That sounds pretty good, uh...I'll have to look it up about that paper up and um i-i think yeah in terms of what is the right distance?

00:22:44: I think just before i answer that.. I think there's another problem which is you know from The companies that are developing these algorithms perspective particularly wider society.

00:22:56: They sometimes feels like they are trying to push us together Right because the closer we get In that way sort of investment and money, things that are generated from this.

00:23:08: So there is already a tension in how those hedgehogs try to find the right distance because we're trying to be pushed together.

00:23:17: but also though it says what's at the right distances?

00:23:22: It's really hard one answer to be frank!

00:23:27: Some of these I just don't think The right distance is where the human still feels accountable.

00:23:37: This human still feel like they have made a decision that They understand for themselves, and then make the right decisions in that way.

00:23:46: I think now probably there's a the closest I could get to an answer right now because it Is difficult?

00:23:54: I Think clearly There has to be a use of AI in health care.

00:24:01: We're not, we were well past the stage with kind of saying this just doesn't help.

00:24:06: but how do we define that line?

00:24:08: How did you find out why and it kinda says suddenly would become not accountable?

00:24:12: I don't know an interesting area or space within as well things going to be around.

00:24:18: what about patients themselves...how are they using AIs in their lives to manage there conditions?

00:24:24: now how are they responding to those decisions or suggestions from AI that they're not talking to their doctors or clinicians about.

00:24:34: So I think how have you seen sort of patients using tools?

00:24:40: as a clinician is, In the UK, we've had GPs general practitioners family doctors sort of saying that you know They have patients coming to them.

00:24:56: That are perfectly healthy.

00:24:57: But their Apple watchers kind of how.

00:25:00: I said they may Have something with our heart that they need to go and get checked out then.

00:25:06: Actually there's nothing wrong.

00:25:07: so it's kind of generating anxiety as well in that space.

00:25:12: but i think this sort of The patient side is a probably another area that we need to understand more about.

00:25:20: I think you are touching two very important points.

00:25:24: One is that clinicians still feel accountable when using AI, and this is extremely important because it's the basis of our work and job.

00:25:38: The other problem is that patients coming to clinicians with a diagnosis made up by charge of PT or other LLMs And so it is very difficult to confuse these diagnoses sometimes or explain to a patient that we have in mind another workflow, diagnostic work flow and other management because they are still inside this convention such as it is better than humans then clinicians.

00:26:09: So really there's an important topic on.

00:26:12: this brings me one of the points and tackle with this problem of education.

00:26:21: for example we need to know a i and probably our learning curve across yours is going to be reshaded by in the next year.

00:26:35: do you think that it will change the way we train restants and young neurologist?

00:26:44: so.

00:26:45: I think it's probably already happening because i suspect the students are doing for themselves and this is a conversation we've had in our institute, how do we teach even just basic skills around data and statistics to our clinicians.

00:27:08: Do you have that grounding?

00:27:15: Some of this is, we see it as well in teaching code.

00:27:21: The large language models like Claude and co-pilot can generate computer code that our students are just kind of using and generating and they can generate masses of these.

00:27:33: but the question is do they understand those things?

00:27:37: So I think there's two parts to that question.

00:27:39: one which Are we teaching how to use those tools appropriately and How are we developing those materials?

00:27:48: And the second part is With those tools there, how will be making sure that people have still having enough fundamental understanding To be able to use these tools correctly afterwards as well.

00:28:00: You know otherwise We'll end up with you know clinicians who can apparently make the right decisions about a patient but maybe don't actually themselves fully understand it.

00:28:13: How do we get there?

00:28:14: Now, I don't believe that our current education systems would allow somebody who is effectively at a level to get through because we have very rigorous sort of processes for testing and checking in training.

00:28:28: And all the rest of it.

00:28:30: but um i think It Is going To be an issue Of how Do We Sort of Make sure though That We're Finding that Right Balance In Education.

00:28:40: So Part So some of it is probably going to be recognising where it works and finding out whether that doesn't work from a healthcare perspective.

00:28:51: Do you think at this point, do you think it's important that neurologists have some mathematical expertise or computer science expertise on these

00:29:04: points?

00:29:04: I was talking about yesterday with somebody... A lot people are coming Backgrounds another scientific backgrounds and they're kind of learning to code right that teaching themselves how to use Python are because They find it useful to do so.

00:29:23: Um, but they don't.

00:29:24: But often there's a people you know what's all in maths?

00:29:28: So the complexities over here, but the thing about coding is is that its effectively way of setting out logically something that you want to achieve.

00:29:36: So I think there is probably something around how we do teach some basic coding skills, how we look at the challenges of data.

00:29:43: that should be included in that because it's going to become increasingly important.

00:29:49: If you're expecting somebody as we said to accountable for a decision they've made on an algorithm... They shouldn't understand what an algorithm means.

00:29:57: if they are able feel confident in doing so maybe one thing helps us bring our hedgehogs closer together!

00:30:06: Okay, Gary.

00:30:07: Thank you!

00:30:07: Let me close with one final practical question.

00:30:12: If you were to leave a neurologist listening today With one practical message What would it be?

00:30:18: Do you suggest ask two different or three different LLMs when asking for clinical questions?

00:30:26: How do we use OPIN-A in the way that is open minded but still responsible?

00:30:34: just practical issues.

00:30:37: I think, to be honest the most practical thing i would say is find your sort of colleague you're machine learning colleague who can help you.

00:30:48: that's probably the best thing today.

00:30:49: if you're wanting to understand and move into this space go on talk to the computer scientists.

00:30:54: Go on Talk To The Engineers In Your Area Because We Need a far greater multidisciplinary approach to this going forwards.

00:31:09: And they're the ones that can help you understand those skills, so I think more than anything else...I would suggest actually look at who you have around you that can actually help you to understand it more because rather then sort of turning into a large language model or just certainly not their materials come from salespeople visiting our hospitals.

00:31:34: It's probably reach out to your university and kind of work with them, how do you construct these multidisciplinary teams?

00:31:42: That has been very much the approach that we've taken at CHIL where this design studio approach.

00:31:48: We want bring people together around a problem... ...to solve the problems not think about technology.

00:31:54: Yeah I totally agree to team up with computer scientists in the next years.

00:32:03: It's absolutely the central point because it is a human-human interaction, that way!

00:32:09: Okay?

00:32:10: So I think thank you so much, Gary.

00:32:13: This has been very clear and useful discussion And at the end of future probably... not the human versus artificial, but humans with humans and computer scientists and clinicians together.

00:32:26: And that future is one in which Human Clinicians work with AI decision support tools on this support finding these right distance we were talking about before.

00:32:36: so thank you very much Gary for joining us today!

00:32:39: Thank you to all our listeners who are joining us there from this episode of EAN Cast Weekly Neurology.

00:32:54: This has been EANcast Weekly Neurology.

00:32:57: Thank you for listening!

00:32:58: Be sure to follow us on Apple Podcasts, Spotify or your preferred podcatcher for weekly updates from the European Academy of Neurology.

00:33:07: You can also listen to this and all of our previous episodes on the EAN campus.

00:33:11: To gain points & become an EAN expert in any of her twenty-nine neurological specialties Simply become a individual member to gain access.

00:33:20: For more information visit www.ean.org.

00:33:24: That's E-A-N dot.

00:33:26: O-R-G backslash membership.

00:33:29: Thanks for listening!