Episode Transcript
[00:00:01] Speaker A: Welcome to UVA Data Points. I'm your host, Margaux Jacks. Today we explore the rapidly evolving world of digital brain twins, personalized data driven models of the brain that could revolutionize medicine and neuroscience.
Joining the conversation are two leading experts, Dr. Randy McIntosh, a pioneer in brain network analysis, and Dr. Emiliano Ricciardi, an expert in cognitive neuroscience and neuroimaging. Together with Jack Van Horn, professor with the School of Data Science and Department of Psychology, they'll dive into how these digital replicas of the brain could change the way we understand cognition, disease and treatment.
[00:00:44] Speaker B: Today we're exploring one of the most ambitious frontiers in neuroscience and artificial intelligence. The idea of of digital brain twins. These are computational replicas of human or animal brains which could transform medicine, research and even our understanding of the mind itself. Imagine, if you will, a living, evolving computational model of a brain, personalized, data driven and capable of predicting disease, testing treatments and simulating cognition without real risk to a real live person. Sounds like science fiction, but it's happening very quickly and rapidly, becoming a salient idea and scientific agenda for many around the world. And to unpack this, we're joined by two world leading experts in the neurosciences and the computational sciences. First, we're delighted to welcome Dr. Randy McIntosh, who is a professor at Simon Fraser University in British Columbia, Canada. He is a pioneer in brain network analysis and neuroinformatics, and known for foundational work on large scale brain dynamics and multivariate methods for understanding cognition and disease. Welcome Randy.
[00:01:51] Speaker C: Pleasure to be here.
[00:01:52] Speaker B: Also Joining us is Dr. Emiliano Ricciardi, who is a professor at the IMT Scuola Alti Study in Lucca, Italy. He is a cognitive neuroscientist and studies sensory processing and advanced neuroimaging, particularly on how the brain organization supports perception, language and cross modal plasticity. Welcome, Emiliano. Thank you, Jack. Let's just start at the beginning, Randy, maybe you can kind of start us off here. When people hear the idea or think about digital brain twins, they might imagine anything from some detailed MRI reconstruction to a kind of a conscious or sentient AI avatar. From your perspective, what precisely constitutes a digital brain twin?
[00:02:38] Speaker C: Oh, good question. I mean, I think with, with the technology as it currently sits, I think it's safest to say that a digital brain twin would be taking different sorts of data that one may acquire from your brain. So for example, typically it'd be something like an mri. So get structural data, you might get functional data from there and integrate them into some sort of combined simulation that would be A simulation of part of what your brain is doing. You can add more information, of course, and get down to details of neural firing, for instance, and things of that sort. The idea is to take data that we would collect from your brain and then merge them back into making a digital replica of what your brain ostensibly is actually doing. In this case, it's not quite as far along as things like for example, consciousness. I don't think we're ever going to get there. But certainly that's one of the sidebars that people can talk about, is that we'll just take your brain and do the replicas that can be there for eternity after you're long gone. It'll still be a conscious version of Jack Van Horn somewhere in cyberspace.
[00:03:41] Speaker B: That's scary.
But Emiliano, does a brain like this, does it have to be individualized to a specific person or an organism? Or can it be just sort of like a generic brain with a capital B? Or does it can be specific to some individual or patient, for example?
[00:03:58] Speaker D: Well, you know, I think that probably the leading aspects of the digital brain twin is really having a sort of like a virtual model of individuals brain. So the idea is really to really act at different level. I think Randy was properly highlighting how you really to recreate some sort of like similar architecture that then you mirror into similar function. And then you know, from this model you try to, you know, primarily even predictor of trying to simulate. Okay, that's the study of digital twin.
[00:04:30] Speaker B: So Emiliano, what kind of scientific or clinical problems could a digital twin uniquely solve that current methodologies just don't do?
[00:04:40] Speaker D: Well, let me first of all highlight two aspects that I really like of the current approach to the digital twin. One is really the idea of multimodality as a necessary step. The idea of describe the brain through different measures. And this is really something that digital twin are really pushing forward. The other idea is really that as compared to other computational method, this is really theory driven. So most of the time you really do need to have a sort of scientific question theory that you would like to study to investigate. Currently I'm approaching the field specifically tackling the issue of the sensory deprived brain. You know, that obviously one of the big questions in neuroscience is how much of our brain functional architecture is innately present at birth and how much is actually modeled by sensory experience. So studying model of samples, for instance, you know, congenitally blind individuals or congenitally deaf people is really tricky aspect because we are still struggling to understand what is actually occur in their brain, why they are so able to efficiently interact and represent the surrounding world. But at the same time, what's going to happen to those brain areas that typically do not receive, are not receiving anymore, the preferred sensory input?
After more than 30 years from the first description of what has been called as cross model plasticity, we're still trying to understand, to break the code of what is really happening in these deprived brain areas.
[00:06:26] Speaker B: Yeah, Randy, you kind of touched on this and Emiliano picked up on it as well. This notion of multimodality, it kind of gives us a level of resolution, if you will. It gives us a level of sort of specificity, but also integration. Right. How do these things all come together to help us to simulate a real living brain? Can you comment a little bit about what those modalities would be and what that level of granularity or resolution might need to be to be convincing as a digital twin?
[00:06:57] Speaker C: So that will depend on who you ask. So let's say, for example, you're building a digital twin for someone who has epilepsy, which one of my colleagues, Victor Yersa and Marseille, his colleagues are doing that. So there the data typically would be mri, which you'd have collected on individuals usually structural and functional. So that gives you sort of a high level view of brain organization and dynamics.
You may also get diffusion tensor imaging, which allows you to infer structural connections in that person. But for epilepsy, you must want a bit more information about the focal dynamics of trying to get down to the level of subpopulations, if you will, of neurons, which is where intracranial EEG comes in there.
That's what the surgeons are going to want to see when they think, is this digital twin to be useful for me? Well, if I can actually identify a seizure focus to the level that I can actually intervene with either some stimulation or excision, that gets down to like, you know, several millimeters. But there, there are others which will want to go in more detail and actually have some sort of pharmacological representations. So neurotransmitter systems, it's a bit more challenging in humans because we don't really have the capacity to measure individual synapses to the level of knowing what their relative balance of excitation inhibition is.
So some people will be a bit more dismissive and say, well, you're not getting down to the level, which is really important, but that's getting back to looping back to the theory part is what do you want to really understand about the representation? And do you actually need to go down to the sort of cellular molecular level to really get some insights that are helpful for either fundamental understanding or in the case of epilepsy or even sensory deprivation doesn't inform you in terms of some potential clinical applications that can be directly translated. And I would say that you can actually make very useful digital twins that are at a scale that's going to make a big difference for that person's life.
[00:08:54] Speaker B: Emiliano, do you have any thoughts on that?
[00:08:57] Speaker D: I think that Franny already highlighted really fundamental aspects of it potentially. It was clearly mentioning pharmacological modulation, but also thinking sites of neurostimulation. That's another field we are actually trying to develop digital twin model and the idea of figuring out which is the best part of the brain that you want to stimulate. Model and this get back to what we were saying before. You know, the idea of having an individualized approach in order to improve this kind of treatment or even in your study.
Clearly what is also attracting of this perspective, as Randy is saying, is not that you can really model all aspects, but I see that also as a fundamental aspects in human to bridge what we know about microscale, even from physiology, from animal study to what we generally observe in terms of macro scale response at the brain level. So as a whole community, in this idea of going meso, meso scale, try to get this more specific description. I think really also the digital twin could be a very interesting tool towards that direction.
[00:10:14] Speaker B: Well, I think the big idea would be to try to come up with a human digital twin. And we're like, we can kind of do that now. At the same time we can also create like toy models which are just like little computational play things, perhaps animals models are somewhere in the middle.
Where do you think we are now on a spectrum of being able to do something between these, say a toy model all the way up to some clinically actionable digital twin.
[00:10:41] Speaker C: So I think there's two parts to your, to your question, Jack. I think the first part is building models, or you call them toy models that are, that are, that are useful for understanding fundamental processes.
And that's the getting back to the theory that you know, what is, what is your question is a question simply about some circuit dynamics and how circuit dynamics can reorganize themselves in the service of learning, for example. And you can do that now, I think quite nicely, may not be a digital twin in the sense of personalized. It could be some sort of derived representation. But it's still useful because it does give you some, some fundamental insights into how Network dynamics can reorganize, you can push that even more and start thinking about well, a lot of the models that we build are sort of brains in a vat or brains in a bottle. And to really sort of contextualize brain function, we need to put the brain and the body in the environment and so on. So you almost call it like almost to the end of being robotics in some respects.
And that's really getting at a different level of a digital twin where you start thinking about that the brain is not really, it's in the head obviously and it's semi autonomous system, but it does interact with the rest of the body. And I think that's an important aspect that digital twinning really hasn't addressed very well.
And so the next leap going from sort of that rudimentary ideas of brain networks to brain networks in the body and so on is going to be a huge leap. It is being looked at, but I don't think it's been done in any systematic way other than saying it, we should do it.
Now. On the other side, I mentioned the work from Victor Yers's group and there's a clinical trial they just finishing called EPI Nav and that's a clinical trial for using what's called the virtual brain, which is a brain simulation platform that we developed about 20 years ago.
The virtual brain makes that sort of digital twin of an epilepsy patient's brain and can be used to help guide surgical intervention. So I think the fact that it's been used in a clinical trial now suggests that there is a way of making digital twinning useful.
And we'll find out the outcome of the digital sort of the clinical trial probably later on this year they did the one year follow up, but I think that's pretty good in the sense that you do get useful models that can be used for clinical decision support. As I said, that can be used in the service of making patients lives better.
[00:13:12] Speaker B: I'm curious about, we were talking about data types before and the different levels of granularity. But it also brings to mind like what are the computational requirements for doing something that's like structurally similar to a real brain? Hopefully a digital as high fidelity twin in terms of its brain shape and structure, but also its temporal dynamics. Could one effectively look at and emulate say EEG recordings out of a synthetic brain? And what sort of computational requirements would be needed for that? I'm thinking in particular about something that was done a number of years ago called the Blue Brain project. They were just Trying to simulate what's called a cortical column. So a little collection of neurons actually spans a little tiny area of the brain. And that was a huge computational challenge. Trying to do this at scale is very, very challenging. And I know Randy's, you've been very interested in this at kind of these large scale neural models. What sort of requirements are needed computationally?
[00:14:20] Speaker C: Just to give people an idea of what that the challenge really is.
The brain works across space and time. So in the context of what happens at current time is actually dependent on what happens in the previous, say whatever time step, say previous 10 milliseconds. So in the computational domain, you have to keep track of all that stuff that happened before.
The other challenge, of course, is that brain areas aren't all connected at the same distance, which means that obviously areas that are close together are going to have a much faster time scale if they work at than areas that are far apart, like frontal and parietal cortices, for instance.
And that time separation is quite important probably for both healthy brain function, but also when things start going astray. So all that information has to be stored in real time as you're computing the simulation going forward, you can imagine as you start expanding the complexity of the simulation, either adding more areas or more granularity and having different synapses, but all of a sudden the computational requirements go through the ceiling. That's, that's kind of why blue brain kind of is stalled, because it really has reached the computational sort of barrier. Now you can do hybrid modeling, which is something that's been done a little bit within the Ebrains consortium in the eu, where they put together a sort of neuromorphic computing system with sort of large scale simulations. So neuromorphic means you actually create hardware that it's. If each piece of the hardware represents a neuron, for example, and that increases the computational efficiency quite a bit. But you can imagine if we had to build a billion of these things because you want to do a billion neurons, you need a billion hardware that represent neurons. That's going to be an enormous machine and probably generating a lot of overhead for resources and things like that. We're not going to go that direction. But anyway, it's a computational challenge I don't think we've actually addressed well yet.
And are there ways to do it? Perhaps, you know, quantum computing? People will ask. Davis will suggest that might be a, a source. But the other thing is thinking about that the, you know, all the simulations are digital and is there some gain to thinking about the computations are actually analog in the brain. So it's an analog system. And that's something like people like, for example, Earl Miller are talking about the fact that it's an analog system. So therefore we should think about how you translate what we measure electrically with EEG and FMRI and so on and so forth and go back to the fact this is an analog system. If that helps to reduce the computational load, I have no idea. But it's a formidable challenge, to say the least.
[00:16:53] Speaker B: Emiliano, what is the view from Europe on the kind of computational requirements for digital twins? Are there any efforts going on now that are not part of Blue Brain, but other things where the capabilities might lend a hand?
[00:17:08] Speaker D: Not really a lot to add to what Fred already said, because I think that he properly allied the idea of that really the computational requirements of the digital twin depends on its level of complexity somehow. So you can play from simple, let's say predictive model to, as you were saying, a whole brain dynamic simulation updated even near real time. So obviously that's actually relied a lot. That goes a little bit of what we are seeing probably before. The idea of having these large scale initiatives also from a computational perspective could be actually fundamental.
[00:17:49] Speaker B: Are there opportunities for say, North American as well as European partnerships on this to bring computational power from both sides of the Atlantic to kind of help create an initiative? I know that Randy mentioned the E Brains consortium in Europe.
There's various things. Big initiative in Canada goes from Montreal all the way over to British Columbia in terms of being able to share data and computational resources and several different initiatives here in the US which may just need to be hooked together. If all that could be done, would that be a useful thing to do?
[00:18:26] Speaker C: Yes, it would be.
The problem is we have humans that are overseeing a lot of this stuff. So there's a bit of a territoriality issue that adds a bit of a barrier to things. There's not a lack of willingness to share data, I think, and that's something that is a.
A different positive things for these movement toward digital twins. I think the challenger comes in. We're going back to the comment that Emiliano made beginning about theory and how what do you want to do comes in so it's more than just crunching numbers.
And I think that's something ends up being a challenge is that yeah, we can get tons and tons and tons of data through tons and tons and tons of classifiers and so on and so forth, but I don't know that gives us any more insight really. And I Think there's. It's not so much linking the infrastructure per se, but also thinking about how to. Different approaches to understanding the brain need to be integrated. And that's where it's kind of missing right now, is that we're moving fast and building the architecture to do the simulations, but we're not moving fast in terms of thinking about what the simulations are actually telling us.
[00:19:35] Speaker B: Certainly classifiers are something that you see in data science applications all the time.
Many people will leverage neural networks that are really only neural by name, as far as I'm concerned. They, like, do one thing. They have a little sort of neuronal, like, behavior, but to stretch them out to being, oh, this simulates the brain is not really, in my view, a fair comparison. I think they're very powerful, obviously. But to say that they're doing what the brain does, I think that that's a misnomer.
And I'm curious about, Emiliano, what your thoughts on this. I mean, we're talking about theory. Do we. We probably, in order to make these twins as high fidelity as possible, we need to bring in classical neuroscience. We need to bring in, you know, levels of partial differential equations and other things in order to simulate these systems at a level that we'd be comfortable calling them neural networks, truly neural networks, because the current form of them just really is a, you know, it was a convenience to add these, you know, rectilinear response functions just to force the computer to do something. How do you feel about that? And do we need to know that kind of level of theory before we can start building on new theories?
[00:20:58] Speaker D: Well, that's obviously sort of like, tricky question because, you know, if you think that we have been actually and still doing in acquiring brain data from signal, physiological signal or metabolic signal that we hardly understand, you know, fully understand.
Nonetheless, one thing that I think you. You've been highlighting two aspects. One is really you need to model sort of like very basic neuronal level physiological response. Okay. And that obviously requires deep knowledge from that perspective. And the other thing that I found fascinating is getting back probably our field towards a tight correspondence between structure and function. Okay. Something that probably we were actually a little bit missing in the last year, pushing much more towards changes or representation. We were actually identified in different approaches at a functional level, but not often highly coupled with structural changes. So this is one of the outfits that I, I'd like to highlight, really, how we are getting back to this sort of, like, neuroscience axiom of having a sort of like, parallel between structure and function. Per se.
[00:22:21] Speaker B: Randy, what are your thoughts on that?
[00:22:23] Speaker C: I mean, you made a good comment in the beginning of the question, Jack, about the use of the word deep neural nets and whether they are really neural nets in the way we think about neural nets. We being the sort of neurophysiology community, I guess I'm kind of part of that. So I can call myself a neurophysiologist for the purposes of our talk today. The terms are the terms. And as long as you can understand what exactly is being done in a deep neural net, I think that's fine. The complementarity, I think is an important thing that we tend to sometimes miss in trying to be too pure in terms of how we approach things.
So I think extracting data features like mission predictive modeling, Jackson I think that's where the framework for machine learning, deep neural nets, and then the transition to AI of different flavors is really important for understanding what data features are really important for predicting whatever you want to predict, whether it be memory function, it could be motor behaviors, it could be cognition or some combination thereof to come up with the optimal combination that does that prediction for you very well. What you don't know is why is that important? Why explain why those predictions are actually working? And that's where that sort of the generative modeling part comes in. That's sort of at the heart of virtual brain, for example, as well as a lot of the work that Carl Friston's group has been pushing for the last couple decades as well, is that you really have to think about what is this, what is generating that signal. And that's where the theory comes in. But I think like the deep neural nets approach to predictive modeling, I can see that as being an important complement and that's, that's, I think, where we need to put a bit more effort into thinking about how to combine those things in formal way so that you can define the statistical space within which all these data lie using something like predictive modeling. And then use the generative approach to understand the uncertainty in that predictive space and say, well, why is it that we do we predict well in this part of the distributions could be like, you know, 40 year olds. We really crap out when we're trying to do predictions in the 70 year olds or predictions in the kits.
And then I think that's a direction that is worth going. I know there are people who are doing this, but I think that's where we can be less sort of dismissive of saying, well, we don't want to deal with predictive modeling, because that's just going to tell us about predictions. We can merge them and that's the architectures that we have available. And that kind of gets back to the question you had Jack earlier about this effort's going across the globe.
That's I think where people can really find the bang for the buck is not so much integrating data as much as integrating approaches.
[00:25:01] Speaker B: And Emiliano, Randy brings up something kind of interesting. It's almost like what's the validation of these. Right. We'll, we'll bring in math, we'll bring in theory, we'll bring in all this data.
How will we know that we've been successful? Is this a statistical thing? Is this a, you know, we're, we're generating signals with the right characteristics and that will be our tell that we're or get close to being a good twin. Any ideas on what will. I guess also what happens if you have a pathological sample, like somebody for example, with congenital blindness? Can they be a test of the system and provide a validation framework for us?
[00:25:40] Speaker D: Well, I think that again the wide gamut of application that we were actually talking about, obviously there might be several idea of validation.
You know, could be does the twin accurately reproduce individuals anatomy and connectivity? You know, that's, that could be a sort of like very basic question. Or can the twin reproduce observed neurodynamics? That's something for instance that I'm actually trying to respond to with my model of sensory deprivation. You know, we see phenomena in the brain of congenitally deprived people that we are still, you know, trying to understand. Why just to give an example, Jack, if you record resting state activity EEG in blind people, congenitally blind people, that resembles much more study individuals with eyes open, not eyes closed. And in last 30 years we have been actually comparing blind people with blindfolded sighted with auditory stimuli. But you know, their activities is much more similar to when we are eyes open and we are, you know, things stimuli. So it could be just do we have, you know, am I able to model my parameters in order to reproduce the observed observed dynamics?
As if we go in pathology, you know, we're talking about pharmacology, neurostimulation, but also pathological states, you know, can my twin predict unseen future states of disease? So obviously that could be a sort of like correlation with for instance, clinical outcome.
And obviously this could be some example of how we are actually trying to validate a sort of robustness of my model.
[00:27:35] Speaker B: Yeah, I mean it's a really interesting question about kind of taking a current version of a twin and asking a question about the future state of that twin and whether or not the person who it is a twin of would follow that same trajectory. Which sort of begs the question of, you know, real brains change all the time from through development, through kind of, you know, middle age into old age. And there's learning and there's disease processes going on. So is a digital twin like a snapshot of a person? Is that good enough? Or do we need to build in sort of a lifelong trajectory for digital twins as well in order to be able to make high quality predictions? Randy, do you have any thoughts on that?
[00:28:23] Speaker C: I mean, it's almost a rhetorical question. I think the answer to the course is yes, but we have to make sure we build in sort of a longitudinal component to our models. The challenge, of course, is we can't do that. Really true to the word of the longitudinal, we can do longer snapshots, if you will, so taking like a decade. But I mean, it's incredibly expensive to do longitudinal studies, as anybody who's done that knows, like UK Biobank, you know, abcd, Human Connectome Project, all these things have enormous challenges maintaining that data acquisition aspect. It's certainly, I think, the case if you're looking at a progressive change.
Like, for example, you think about degenerative disorders like multiple sclerosis, dementias, those things. The nature of time is going to be quite important. Time. Now we're talking on the scale of years and being able to predict if we have a snapshot of someone's brain when they're 25, can we predict whether they are going to show dementia when they're 50, for example, assuming they have, for example, some genetic predispositions for that.
And those things are going to be critical for us and being able to have a longitudinal aspect.
You can see examples where longitudinal and cross sectional data diverge. And I think that's an important thing that people tend to forget is that they're not, can't. You can't substitute cross sectional for longitudinal.
You have to make some serious assumptions. But then that's coming back to like, can we use a digital twin then to start testing whether or not our longitudinal assumptions really work? If we land on that, that could be quite helpful.
We actually, we just got a grant recently from the Canadian government to actually do that to try and see if we can find ways to use the, simulate the digital twinning platform essentially to build distributions of trajectories that take somebody like at 25 and using some assumptions about how someone may progress. Can we build that trajectory and then validate it? And that's the other part, is that we can build these things. They look really cool on paper, but if we actually then measure that person who's 25 when they're 50 and say, well, we were going to predict they had dimension, but they're doing well, they're kicking ass. So why, why did we miss there? Or vice versa. Right. And that's, that's the kind of challenge that we face.
But I think the challenge we can address if again we kind of merge these different techniques to really come up with more robust ways of predicting, but also more robust ways of explaining and using other approaches to do that. I think there's not a lack of approaches, it's more a lack of time and mental power to actually do this as a collective.
[00:30:57] Speaker B: There's a lot of research going on in the estimation of brain aging. You know, if you are a, you know, 59 year old guy in Charlottesville, Virginia and you look like you have the brain, your brain looks like somebody who's five years younger. I hope that's a good thing. Right. If it looked five years older, you know, or it was looked significantly older, that would be something, would be clinically useful because you could take some action on it.
And so there's a lot of research going on in quantifying that and giving somebody a relative brain age. And I'm curious if digital twins might play a role in estimating brain age, but also estimating the, you know, the, basically the trajectory of what's going to happen to your brain as you age out from underneath it. Right. And I'm curious, Emiliano, if you have any thoughts on that. And then Randy, I bet you have some thoughts too.
[00:31:54] Speaker D: Well, I think that Randy correctly pointed out that currently we may be quite good in having a sort of like defect of acute modulation. So it could be treatment, could be task modulation, but really the big challenge is actually being able to understand this specific trajectory of development.
Really have no specific idea is quite challenging and fascinating.
Make also business out of that. I'm just afraid of how many variables really have to take into account in order to guess the prediction in this case.
[00:32:37] Speaker C: It's interesting you mentioned the brain age issue because I'm teaching a class right now at SFU on neuroimaging and network neuroscience and we just had a lecture on big data.
Actually we had one of your papers there for the assigned readings. Jack for you might be like, thank you.
But one paper that comes to mind is Actually one that really showed the challenge of the brain aging. And that's from Videl Pinero Group that was published in eLife 2021. Individual variations in brain age relate to early life factors.
More than two longitudinal brain change. And that's, you know, that kind of says where you start predicts where you're going to go and less so than being able to do some sort of cross sectional thing. And they showed a very sobering graph where they showed the correlation between the actual brain age predicted from the brain aging model and then brain age predicted longitudinally and found that there was zero correlation in that prediction. Which is kind of sobering to say like, well, maybe this predictive model is not doing as well as we think because it's based on cross sectional.
And that's the kind of thing that, you know, it's, it doesn't get assessed because it's hard to do. But I think at the same time it also also kind of gives us a moment to pause and say, do you really need to understand like why you're getting this prediction that gets back again, explanation versus prediction are not the same thing.
And that's if that paper really kind of, you know, opened a lot of people's eyes, oh my God. This is actually something we need to be careful about.
[00:34:07] Speaker B: It's a very important point. Thank you. I want to shift slightly and talk about some of the ethical and societal implications of digital twins and the pursuit of thereof because they raise a couple several actually profound ethical concerns.
For one, who owns the digital twin, who owns the data that went into the creation of that digital twin? It's a digital twin of someone. Does that someone own their digital twin own quote, unquote?
Does the researcher who gathered and built the model or does it belong to perhaps some company? You know, if Google Brain decided to do this, would, you know, it belong to them?
I'm curious about that also. Can you, could digital twins be used to predict behavior or mental health effects? Things which, you know, maybe are not something somebody wants people to know about. Maybe they don't have mental health issue now, but they may be concerned about that if it were made public or something. And also things like, you know, are there personal cognitive traits that would be reflected in a digital twin? Which again might be something that is a little more than just, you know, a clinical assessment, for example, something that actually would emulate your clinical quirks and whether or not there's an ethical issue there. And just what are some of the moral issues which are associated with as a twin becomes More highly realistic.
Are there some moralistic things we need to be worried about? And I'm curious, maybe Emiliano, you can can take this one.
What are some of those things we need to be a little concerned about ethically as well as kind of, you know, should one even do it to begin with?
[00:35:50] Speaker D: Well, I think you're really raising fundamental aspects of you know, through your examples. Obviously from one side it is clear that implication go well beyond is sort of like standard medical data ethics as we are used to talk about. And as you were saying because you know, the digital twin does not just model biology, you know, it's also as you were saying, cognition, behavior and potential aspects of identity or you know, somehow even your future life. So obviously if we already start, you probably remember then you know where you know, we're all around the same institute around When GMAX in 2001 publishes Pioneer Paper on on science and starting this idea of mind reading. And starting from that, you know, we have been questioning about sort of like what my friends are calling like neuro rights for people and participants. Okay. So as you were saying, there are several aspects that might relate specifically privacy and protection, but also elements of how deterministic are these data.
And you know, probably this is much more sort of like North American issue about, you know, thinking about insurance or employers or whatever, you know, how really to protect from this predicting profiling. Okay. And then furthermore there is also aspects relative to, let me say probably self food. The idea of, you know, your emotional responses process fails, the idea of your decision, your cognitive styles. These are elements that we hardly thought so far. You know, we are getting really sneaking into some, you know, private details that probably we cannot really foresee. But it's good if from this philosophical moral aspects we are also start discussing about something that is goes beyond, you know, clinical responsibility. Let me like two other aspects that we often think about. One is clearly the idea of sort of like equality access. So we often be discussing about these equality elements when dealing with large databases in neuroscience. And so task for and this is sort of like important elements and then what Brandy, you and Randy were mentioned before, you know, ethical aspects relative to computational elements. So you know, this is another thing that we may avoid.
[00:38:33] Speaker B: Randy, what are your thoughts on that?
[00:38:35] Speaker C: You know, that's another podcast in of itself almost really where to start. The ethical issues around this I wouldn't say are unique to neuroscience. And certainly they grappled with this during the Human Genome project era. Trying to figure out when you start characterizing as a person's gene, does that tell you everything you need to know about their life trajectory? The answer is no. But there was a concern, of course, that when that genetic information was captured and then shared, does that have implications in terms of the person's ability to kind of have control of their destiny? So if that information got out to a company that was trying to hire them or was used in terms of sort of screening, does that contravene ethics? And of course it does. I mean, and this is an issue not so Emiliano, it's not so much a North American issue, it's actually more of a U.S. issue, you know, and those kind of expands now in the sense that it's not a new thing in terms of dealing with personal and private information, personal health information, and that's some central digital twin really is in some context. So there is precedent for having those discussions and precedent for figuring out how to deal with that. And we can rely on those things. I think the scope, what we're trying to do with the digital twin for the brain is obviously larger because we're talking about things beyond just sort of cardiac fitness or whether your lungs are going to be okay. It's actually more of a combination of that and also influence one another. So you have the brain body thing coming in there as well.
But these are issues that I think we're being deliberate to address them. And that's certainly most initiatives that I know of have ethics embedded in their aspirations directly to make sure that we're addressing these things not just in the sense of individuals, but also the diversity of how we're looking at these data and understanding that, you know, we have to be more considerate of other populations that typically don't contribute to these kinds of initiatives. Like, we don't typically have measures from people in remote communities and impoverished nations and so on. And then their. Their brains are.
See different challenges than we might in.
In other societies. So that. That's another aspect is making sure that it really represents the diversity. And again, I can talk about papers that have tried to see, well, if I have this sort of characteristic brain that shows changes across an age group that predicts education, if I apply that to a different ethnicity, for example, do I get the same prediction? The answer is usually no.
And that's because you're missing something. And that's. So the assumptions around there are quite important.
And then last part, I think Jack, you mentioned, which is something that's still, I don't know, has been solved as to who owns the data.
And we grapple with that Big time at my university because we do a lot of work, you know, with, with Indigenous communities and data sovereignty. There is a big issue, you know, there's a whole host of historical transgressions, if you will, that really compromise the relationship between Indigenous communities and other Western cultures. And that's something that we, you know, data sovereignty is a big part of that. Like we don't just take the data and walk away. The data actually belongs to the person who provided the data. And so we have to be mindful that it's actually we're taking something that belongs to that person and trying to use it hopefully for the better. But sometimes, you know, you can some more nefarious sort of applications and trying to build some sort of better AI that, that a different company can use to mine people's information in some potentially unethical way, although we don't know that that line's going to be crossed. But really that the conversations around that, conversations around data sovereignty, the conversations around who own the data, really has to be done with the people who provide the data as well.
So that notion of engaging the communities in that conversation, I think is one thing we don't do very well as a, as a collective across. You know, there are various groups that are doing these kinds of big data efforts and trying to build these digital platforms that we really need to, to talk to the person who's providing the data and say, what do you think about this? Do you want to really share the data that broadly? And you mentioned Jack, you know, the, you start taking this information and making it available through these open neuroscience platforms. That means it's open, that means anybody can get it.
And that has implications in terms of use for good and use for evil. So, and I think that's something we're faced with now. We had that recent thing with the ABCD data set that was, you know, publications that came out that were talking about racialization of the data set and really making some very unfortunate statements about the data.
And that was because the data were ostensibly open. And those are the challenges that we're going to face in this game that has to be really thought about and put in the appropriate checks so that we don't fall down that hole again.
[00:43:24] Speaker B: Thank you for that. These issues of trust are very important between our research community and the people who we're going to gather data from.
If their twin is being created because of a clinical need, there's obviously patient privacy related things. On the other hand, if they're contributing data to a study and it shows up in a database and then somebody goes and does nefarious things with them. That is something that, you know, is we hope will be avoided. But frankly, as we've just kind of recently seen, some people have an agenda and that may be difficult to sniff out in advance.
Let's play this forward a little bit and let's say that we've, we've, you know, we seem to have the ability to start on this now and if this becomes a thing over the next, say, 10, 20 years or so, could a sufficiently detailed digital twin brain ever become conscious? Now, I say this because Emiliano brought up the research was done on brain reading, which was where people were looking at particular stimuli and then using the brain's response to those stimuli to ask the question, what was the person looking at when their brain responded this way? And it's a way to sort of reverse engineer the movie of what they were looking at. And it's really compelling. Some of these things and people have posted them on YouTube and on other social media channels that can be found. But you could imagine being able to do something like that at very high resolution, at very high, say MRI magnetic field strength, so you get the properties thereof. But also with, with incorporating information about the burgeoning large language model activities which have now kind of taken over our lives.
Could you see all those things coming together to create these virtual avatars which could, reason, could detect things and make decisions that could basically become sentient?
Is it not too far off in the future that something like that could happen?
[00:45:32] Speaker C: Masif answers I don't know.
There, there are things certainly I you mentioned, Jack, I think are possible.
So certainly they can build models, whether it be LLMs or something of that built, that can make reasonable predictions for a given data. And those predictions can pretty helpful. And they, they already are in some respects. Some people say that, you know, LLMs are, are doing well for some things and they are. I think that's actually a good, a good, a good thing.
Whether it's sentient, I think is a harder question because that comes back to how do we define what sentient is? I will even go further and say how do we define consciousness? And even if you have an answer to that question, it'll vary dramatically depending on whether you talk to a neuroscientist, a philosopher, a psychologist, a medical doctor. All those things factor into how one envisions consciousness.
I think we're going to have certainly models that show behaviors that are close to what the human body does, whether it has the same nature of experience is a tougher question because that's. That model's not going to have the history that we have. And all those things, you know, if you talk about complex adaptive systems and nonlinear dynamical systems, the same thing that history really has a huge impact on how the system actually functions.
And those sentient systems are not going to have that background. So the initial conditions to get more technical are different for a model than they are for us. So our initial conditions, we're basically at birth. And all those things that transpired after that are really going to affect how architecture forms. So we're not going to make a perfect replica of our brain because they don't have that history. Unless you're going to be measuring someone's brain from the time they came out of the womb, which would be hard to do, to say the least. But I think there's. There will be behaviors that look a lot like the system is sentient. But then we get to sort of ethical, you know, philosophical thing is how do we define what that means? And then if it is in fact conscious, we've had this argument, not we, but we, the global. We have had this argument of what if AI becomes conscious, then does it have actually some kind of rights that humans have? And it goes, okay, I don't want to have that conversation, but that's kind of going in that direction.
[00:47:53] Speaker B: Emiliano, I don't know if it must have made its way to Italy, but there was just recently an AI company called Sea Dance. They're actually from China, actually, so it probably has made it to Italy that just produced a video the other day showing of two very prominent Hollywood actors in a fight scene that was basically created using their AI software.
And it's very convincing. It's like got all the jump cuts, it's got all the kind of fight action scene that you would imagine. And it has caused a lot of a stir in Hollywood circles about, oh, we won't need actors anymore, we won't have to pay them. But then getting to some of the ethical questions kind of related to what we were talking about, it's like, well, the actor owns their likeness and their, you know, their, their. Their being. Right. And yet you can use an AI to simulate them.
Is something like that possible with AI systems now to create a digital brain twin? Or might it be in the future? Do you have thought?
[00:49:01] Speaker D: Well, I think that Randy already pointed out that currently, when we're talking about digital twin, we are talking currently about mathematical models. Okay. Or that are trying to sort of like based on structural and functional data try to simulate aspects of brain functioning. Okay, so not sure obviously we can imagine what's going to happen in the future, you know, but there are elements that relates to the definition of consciousness and aspects of what Randy was calling sort of like being able to being aware of the interaction with external world.
So we can imagine the certain point our digital teams to become sort of like autonomous in getting their decision or getting their behavior. There's kind of fear that we also have for AI system.
Probably you ready too? But you know, it's quite interesting that if you have a system playing on, on a social media for, for AI system, they're like becoming like humans, you know, like gossiping or writing down the sense. So this is not really encouraging from that perspective. So you know, we can discuss about what is consciousness. It's not really my field, but really I think that for the moment we are just mathematical models.
[00:50:29] Speaker B: So what would, what does success look like with a digital twin? Is it, is it a clinical comment or is it something statistical? Emiliano?
[00:50:40] Speaker D: Probably both. Probably both.
I think that you can look at the same thing in different ways. Okay, let me just parallel to be also with machine learning tools we can get to a specific outcome that is what you desire for your AI based system. Machine learning, even without mirroring really what the brain is actually working or doing here is a little bit different really. The idea is not just getting a proper clinical outcome or prediction, but really try to nicely and properly reproduce brain function in order to understand really its way of functioning and working.
[00:51:30] Speaker B: Before we wrap up here, I want to ask you guys one final question.
If digital twin brains do succeed beyond our expectations, what sort of positive transformations would you expect in science or in society?
And what would you hope to see? Randy, you want to take a crack at that?
[00:51:52] Speaker C: We are awash in data, ourselves, individuals, not just scientists.
I think having ways to collate information that tells us about ourselves in some useful way is going to be a continued trend. So we've got wearables, you've got your Apple watch or Garmin, whatever that's measuring your heart rate and your stress levels and so on and so forth.
You could envision a similar kind of thing evolving now with the digital brain twin architecture. I'm not sure how you do that with glasses. We've had to have little things on the side, but you know, something to provide you with a way of making informed decisions about your own health.
And I don't think that's necessarily bad. I think that's actually allows us to be able to have more agency in terms of our own health trajectories. And this is something that varies across, you know, different geographical systems. We talked about, you know, the US healthcare system. I mentioned Canada. There's different kinds of things in Europe as well. And, you know, being able to have a direct interaction, more facile interaction with healthcare would be beneficial, I think, in a lot of sense. So I think that's one area where digital twins will be, will be quite helpful. The other one I think is a bit more difficult to answer in terms of the sort of the ethical issues around having a completely successful digital twins. And it kind of gets back to like, what do we do with that information when we have it?
Do we actually gather fundamental knowledge about how we work? If we do, and this is going to be sort of pie in the sky kind of thing, it may lead us to understand that we really aren't that different from one another.
That there are some similar things that, that affect us. That to try and put in divisions in our society based on things like ethnicity and race and so on may actually not be accurate in that if we unders. If we understand that really at a fundamental level, by looking at each person, individual twin, that might lead to something, some realization that we can be more interactive, that there is our benefits of really encompassing our experience. Understanding that experience is emerged or reflected in the dynamics that are within our brains. And we can actually see that our brains do that very similarly so that we're not that different.
And understanding that closeness, I think is something that we actually would benefit from right now in our current political environment.
[00:54:23] Speaker B: Emiliano.
[00:54:25] Speaker D: Well, thank you Randy, for sharing your thoughts. Really could sign what you said. Well, let's say that if we are going to be able, let's imagine digital brain tweener becoming scientifically robust, clinical validated.
That means that really we got a quantum leap in the knowledge of neuroscience and medicine.
That means that probably we will push, push even more something that we have been already living in, neuroimaging neuroscience, moving from a little bit more descriptive neuroscience to a little bit more predictive neuroscience and then thinking that obviously we can imagine a sort of like precision neurology, precision psychiatry.
Do never forget that we hardly understand the Etsy pathogenesis of all these disorders. Okay, so we're still struggling to understand why you get a specific neurological psychiatric disease. And if we are lucky, we have some kind of treatment. But, you know, if not really have not even any drug or treatment to help.
But I also like what Randy was properly highlighting. Beside the ethical elements, what we're seeing in that transformation in research methodology, research legislation, low aspects. I think really the hope is really to go towards those topics that we were, you know, it was mentioning that the sort of like the concept of the uniqueness of individual, the personal identity or even, you know, the topics that we are just slightly touching through neurobiology, like free will or aspects of decision making, that's really hope is going to be probably the, you know, sort of like dream of.
[00:56:30] Speaker B: So, gentlemen, this has been just such a fascinating discussion. I've really enjoyed kind of exploring these things and just, you know, getting your impressions about what this looks like, what the road ahead for digital brain twins look like. You know, we're clearly moving from a place of we're collecting data and analyzing it and producing statistical results to measuring parameters and turning them into actionable models. And I think it's really exciting. I mean, this really brings to bear data, data science, AI, high performance computing, but also neuroscientific theory, and hopefully our ability to make predictions. And I just, you know, I think some of these, the ethical issues are really profound and fascinating, but also to some of the points you both made, how individual differences make us unique and uniquely human is something that a digital brain twin really needs to capture. And I hope we will be able to do that in however successful these become.
So let me wrap up by saying today we've explored a future where the brain, our most complicated and complex organ, may one day have its own living digital counterpart. Whether it's for curing disease, whether it's for understanding cognition or probing the nature of the mind, digital brain twins sit at the intersection of neuroscience, of AI and data science, of ethics and philosophy. And this conversation has really been enlightening. It's been enlightening for me and hopefully it's been enlightening for you.
[00:58:03] Speaker A: Thanks for listening to UVA Data Points. More information can be found at datascience virginia.edu. if you're enjoying UVA Data Points, be sure to give us a rating and review wherever you listen to podcasts. We'll be back soon with another conversation about the world of data science.
