Episode Transcript
[00:00:02] Monica: Welcome to UVA Data Points. I'm your host, Monica Manney. In this episode, we explore how data science is helping researchers simulate and understand some of the most extreme physical events on Earth, from floods in Texas to hypersonic flight. Our guests are Stephen Baek, a leading expert in geometric deep learning and an associate professor of data science at the University of Virginia, and Jack Beerman, a PhD student whose work is already shaping real world applications.
Together they discussed discuss how AI is transforming fields like weather forecasting, materials design, sports performance and military innovation, and why graduate researchers like Jack are essential to moving this work forward.
[00:00:45] Stephen: So we've been spending a lot of time recently in applying AI for solving physics problems, especially solving physics problems with some extreme characteristics. So in your view, Jack, what defines an extreme physics and why is it important to use AI for solving those problems?
[00:01:07] Jack: Yes, so my view for extreme physics at a high level, it is scenarios where you have a rapid event occurring very fast with a big effect, at a possibly very large scale with that. And so think of things such as explosions or rockets or aspects of flash flooding, so very fast physical phenomena. And so there's various reasons why we may want to actually utilize a AI to understand these. One that we do specifically in our lab is, hey, how can we actually predict these events? So given what the actual scenario is at a given time? So say for an example of flooding, hey, here's the conditions of the river. We want to know, based on these conditions and the rainfall that's occurring, what's going to be the impact on the terrain? And so for using artificial intelligence, we're able to essentially make accurate rapid predictions. So speed up the predictions on these types of scenarios that we're looking at.
[00:02:09] Stephen: This is a fascinating problem as a data scientist. I mean, those physics phenomena like extreme weather events or detonation of an explosive or burning of fuels and propellants and those kind of things, from physics perspective, those are all very exciting and very dynamic problems. But from data science perspective, those are really unique challenges because a lot of machine learning models, a lot of AI models, as a matter of fact, are designed to follow some sort of an average trend in the population.
If you have epidemiology models, they tend to follow the average trend of how disease propagate and things like that. A lot of social scientific models, they try to capture the average trend of the population and so on and so forth. So that's how a lot of machine learning models are designed for.
In a sense, these extreme physics events are complete outliers. Tornadoes and hurricanes. I mean, those are Outlier events, the formation of high energy during the initiation of propellants, for example.
Again, it's an outlier event.
Very small area in volume may develop really high temperature, but then the rest of the area is just like room temperature and nothing really interesting happens. And if you just apply the usual statistical model, what they will do is focus on predicting those usual temperature in the ambient space as opposed to pinpointing those extreme events and pinpointing their locations.
So that's where I think the challenge arises.
We got to have some specialized model to be able to learn and predict those kind of extreme outliers.
And how do we do that, especially given that those extreme events are very rare and scarce. That's where I think the real kind of scientific challenge is. Jack, why don't you talk a little bit about your project because you have some specific examples.
[00:04:23] Jack: So one project that I was able to work with our team on that we believe to have a very large impact was the river forecast flooding.
And many of the listeners may be aware of the catastrophic flooding that occurred in Texas in July with the Guadalupe river, where the river rose over 25ft in less than 45 minutes. And so we obviously saw catastrophic damage and death from that. And there was little to no time of actually warning the people of what was occurring. And so when we talk about using artificial intelligence in order to predict these events, we saw a large gap of hey, again, how can we predict this outlying physical phenomena, but provided in a rapid manner. And so we used a lot of our fundamentals in, okay, using artificial intelligence, but constraining it with actual physical law and theory from numerical simulations and what we see from our physicists in their particular disciplines and integrating that into how do we actually predict these river simulations. And so with the project that we were working on, we were able to actually speed up the current computations of simulations by over 100 times and deliver a three day prediction in less than a second of what a river forecast would look like. And so again, when we're talking about life or death in these scenarios, being able to predict these rapid events and provide that information in less than a second can be very beneficial to the lives of others.
[00:05:55] Stephen: So what's the input and output to your model?
[00:05:58] Jack: That's great. So what we're particularly inputting is terrain data that was covered from LiDAR and satellite data of the actual terrain topography of these LOC physical locations that we're looking at. And then on top of that, what are the current river conditions? So we're looking at the inflow we're looking at the rainfall, we're looking at the speed of the water. And so we're taking what we call our static information, which is the topology that does not change, that geographical location is pretty constant there on how the slope and terrain is positioned. And then our physical evolving fields. So that, again, is the actual river flow, the height of the water, how fast it's moving in terms of velocity. And so we take that information and we input it into our model that then performs those calculations with artificial intelligence to predict the outcomes.
[00:06:48] Stephen: You can sort of conduct your own sort of a weather experiment given a train. So, like, you know, let's say we do a lidar scan of Charlottesville area, and then, you know, all these rivers, you know, that is flowing through Charlottesville. What you could do is essentially change a lot of conditions, like how much of a rainfall in the area and so on and so forth, and then your model will basically tell what's going to happen. Is that a fair statement?
[00:07:15] Jack: Exactly. So if we wanted to look at Charlottesville specifically, we could look at the Rivanna River. We could take that terrain, we could say, hey, what's the actual inflow right now? The rainfall. And if we increase the rainfall, we increase the flow by a large amount. What's actually going to happen to the terrain and the infrastructure around us? Are we going to experience catastrophic flooding?
[00:07:35] Stephen: So how do you make sure that output of your machine learning model says, you know, something that complies with, you know, physics principles and rules and the prediction is not completely off, you know, based off of physics?
[00:07:52] Jack: That's a great question.
At a very high level, we are simply comparing the prediction of the output of our model to the actual ground truth. And so we call that based on past events or simulations that we've run that have very high accuracy. On a physicist side or numerical simulation side, how does our output actually compare to their output that we know is accurate? And so we look at that, you could dive deeper into talking about the actual loss functions that we build onto artificial intelligence and neural networks. But at a very high level, we are looking at what is our output that the models predicted versus what is the actual ground truth.
[00:08:35] Stephen: Is there some sort of a mechanism to embed the physics equation into your neural network model?
[00:08:42] Jack: Great question. So another high level, what we focus on are the concepts of differentiation and integration.
So with differentiation, on a physics standpoint, it's looking at the rate of change of a variable.
Typically, that's with time. So we have in our neural network these blocks in the artificial Intelligence that focus on differentiation. Given those physical evolving fields that we talked about, such as water flow or water height, the velocity, how can we see how this is going to change with a small step in time? And then we have our integration blocks. So again, building on, you know, physical theory and law and numerical simulations, hey, how do we take what was our rate of change? And then we just simply integrate this over time to predict the next time step. And our artificial intelligence does this in what we call a recurrent process. So given the initial time step, we predict we what one time step forward, and then we do this for the entire simulation. We keep repeating this process over and over, given the next consecutive time step.
[00:09:43] Stephen: So the application of this, I mean the whole mathematical framework of what you described, which is to predict the rate of change of some physical variables, and then you make sure that those rates of changes follow certain physics governing principles. And that sounds like something that can be generalized to a lot of different applications. So should we talk a little bit about other applications at a high level?
[00:10:10] Jack: That's wonderful and very accurate in the aspects of generalizability. And so we hear all the time on headlines of artificial intelligence, of, hey, can we actually generalize this to multiple different tasks? Or is it specific to a certain niche that cannot be used elsewhere? And a big focus of our lab because, you know, these physical theory and fundamental laws can be applied to a large range of scenarios. How do we actually use what we were doing on the river, forecasting and flooding to other examples? And so with using differentiation and integration, we've been able to apply this type of model to other applications and actually focus on more specific governing equations. So looking at conservation of momentum, we actually work with what are called the advection diffusion reaction equations, which are found in physics. And so we specify, based on a family of equations in physics, how are there different applications that we can use for our physics model? And so we've done other aspects in hey, using differentiation integration and those fundamental equations, how do we look at stuff such as supersonic flow, or how do we look at applications such as weapons development in computational fluid dynamics?
[00:11:31] Stephen: So far, the problems that we've been dealing with are somewhat simplified to a limited scale terms of both space and time. So a small area of interest, we were able to simulate them using AI and then also small timescale.
So something that is limited to small region in short time period.
I think we developed some confidence that we'd be able to model them accurately and predict them accurately. But we never actually took on the challenge of Modeling something bigger, instead of simulating things in a millimeter scale or micrometer scale, what's going to happen at a much bigger application scale. The examples of hypersonic flow, for example, we've been kind of playing with physics simulation data that is around a small simplified setup. But can we then generalize this to much larger scale? Maybe the scale of an actual hypersonic airplane? Something like that, I think is going to be a challenge. And although compared to the traditional computational simulation, AI is much faster and more efficient.
If we go up to that scale, I think we're gonna face with the practical limitations in terms of the GPU processors, the memory space, and all this kind of stuff. So I think it'll be interesting to kind of explore in a direction of scaling things up, make those small pieces of knowledge generalizable to much bigger scale, where so much complicated things happen. And how are we gonna be able to predict those kind of situation? I think is one future direction, and then the other thing is generalization. I mean, right now I think a lot of physics models are kind of focused on one specific thing. So we train an algorithm and let the model predict certain things on a very specific physics problem.
But that model will only be useful for that specific type of situation.
In terms of a different physics problem, we're going to need to develop a new physics model eventually. What we want is something like ChatGPT. I mean, ChatGPT knows everything, right? It is not limited to only like, you know, coding questions or generating poetry. It can do like all kind of different things. Maybe down the road, what we would like to do is to have this foundational model that knows everything about all kind of different physics. And then we learn from lots and lots of different physics simulation data. And that's actually something that we started working on already and we had a lot of fun developing that model when our friend Florian from Germany visited us. So I think that also is a great challenge and interesting scientific problem. From your perspective, Jack, what are the immediate challenges?
[00:14:43] Jack: Many data scientists and machine learners will always run into the obstacle of, hey, do we actually have data to train our models on? Our lab tries to work on how can we use as little data as possible and have great results with that.
However, that being said is, hey, can we actually get experimental data of these large scale events? Can we actually get simulations of these large scale events? And so working with more collaborators that have this data and that present it and so that we're able to take it into our pipeline and explore with. And I think one of the unique Challenges that we've yet to really touch on is the human motion side of our lab.
And one of our students, Jason Wang, has done a great job in actually extracting pose information. So looking at the human body of tennis players simply from YouTube videos, because he did not have enough data, he went to YouTube and he began building this pipeline to extract this data and actually analyze the pose and predict the speed of these tennis players. And so right now we're talking about, okay, now we have all this data and we have these physics aware model, can we integrate them together?
And so taking our physics side of the lab and then integrating that with our sports side and building out our model again to not only work on these extreme events over here when we're looking at flooding or weapons development, but using this model that's generalizable on the human motion and predicting sports performance and how athletes can fundamentally change their fundamentals and processes by which they operate with looking at and analyzing it through our physics models.
[00:16:25] Stephen: I'm actually glad that you brought up the sports application, because when we say extreme physics, I think in a lot of people's mind, they may think of rockets or tornadoes, but sports is something that everybody loves, right? The listeners may wonder, what's the connection between extreme physics and sports? So do you want to talk a little bit about that?
[00:16:47] Jack: Of course, that's a great question and really not that easy to discern at first when you're looking at, again, a rocket versus a heat.
But again, when we talk about governing laws of physics, we have the same fundamental theory that applies to both types of objects. And we could think of it of a very simple sense of, you know, f equals ma that can be applied across all different types of applications. But when we're looking at it for us specifically, we're looking at, hey, how can we actually take the biomechanics of these players? How can we take the current motion of them, the velocity of these joints, the movement, the acceleration of these players, and again, use these conservation laws that we used over here on the extreme physics side over into the aspect of these players where we view extreme physics, such as Ohtani hitting a home run and that motion of cracking the ball with the bat and sending it over the fence. We view that as a very fast, rapid change of physics motion. And so we want to continue to explore and expand on ideas like that.
[00:17:53] Stephen: Elite sports players movements can be characterized as explosive and a lot of abrupt changes, a huge amount of torque generated, their muscle activation and all that. I think at a mathematical level, I think they all kind of fall under this category of extreme dynamics, which is very nonlinear and hard to predict.
Sort of going back to my earlier comment about the data science challenge there machine learning models that tries to predict human movement and human motion. They are trained on average people like me. The motion that people like Ohtani exhibit is completely different.
If you have a statistical model that just follows the average trend, then the chance is you're going to miss a lot of important characteristics. So I think that's where the really exciting intersection happens between this idea of extreme physics and AI with sports application and human performance in general.
[00:19:02] Jack: I think for where the field as a whole is going in the next 10 years, or even the next five years, we can see not only at the university level, but also in private industry in these corporations, there's a lot of just money and pressure to continue to develop AI and advanced AI across a broad spectrum of applications.
And so one, we already see that in our current day to day research of hey, let's hop on Google Scholar and see the newest things published to archive or publish online so we can, you know, adapt to the most cutting edge changes. And I foresee that being something that we have to keep pace with on the next five, 10 years because it's not just our team specifically that's developing AI. There's a large range of approaches and methodologies across universities, universities and corporations. And so I think continuing to essentially just learn, I think learning at the end of the day is one of the most fundamental aspects that you remind our lab of. You have to always continue to learn and see what's out there, read the research and find ways to continue to improve. Because if our team gets complacent, we're going to fall behind because there is so much work and so much research going into these aspects and these type of applications.
Steve, why don't you kind of give some background as a associate professor of data science as well as the founder of the Visual Intelligence Laboratory on what your laboratory specifically focuses on.
[00:20:34] Stephen: So we are interested in a lot of different things. Just to throw out a keyword, we're interested in AI computer vision. We call our research geometric data analysis, Applying machine learning and artificial intelligence to analyze the things that has some geometric attributes and something visual, something that involves shapes, appearance and those kind of things in terms of application. In particular, we're interested in two things. One is physics of a lot of different engineering systems like airplanes or engines, propulsion systems and those kind of things. Because you know, when you do physics simulation, there are a lot of interesting Geometry that's popping up. Our mission is to develop machine learning algorithms and computer vision algorithms to recognize those patterns coming from some physical phenomena and then use those patterns to predict what's going to happen or design a better mechanical system. And then the other thing is about humans. We all have a very interesting geometry from the inside of our body. We have organs and a bunch of different cells that exhibit certain, you know, geometric characteristics.
And depending on your health condition, we're interested in kind of finding the correlation between the geometry of, you know, a bunch of different shapes with, you know, certain type of disease and so on and so forth.
But also if you kind of, you know, zoom out and, you know, look at the entire body, then as we move and, you know, do certain things, our body shape changes. And what's the dynamics governing the change of our body? Especially when you engage in activities like sports or manufacturing workers, they kind of repeat same kind of motion over and over again.
And how can we teach computer vision algorithms so that they can be more skilled and better at recognizing those different body movements and the changes in the body shape so that we can kind of derive more meaningful conclusion that can help people mitigating injury or improving their performance and things like that? So that's kind of really high level overview of what we do.
[00:23:02] Jack: That's great. And I think it's pretty broad still of what our lab encompasses. And so I'm kind of curious as the founder again, how you built out your students and how the lab kind of functions on the two different aspects of the extreme physics versus the human sports motion and just your kind of vision for the lab and the students that you work with.
[00:23:22] Stephen: We have a big team, as you know, and I look for students who have passion and enthusiasm on a certain topic. And if that topic aligns, we just let them do what they want to do. And I have a privilege of mentoring students who has so much, you know, passion and enthusiasm towards research. They're doing their job very well. So, you know, we have kind of two branches within our group.
One focusing more on the physics side of the things, and then the other focusing more on human modeling. And then they know their clear goal and they're dedicated to make that happen.
From my perspective, as a person who is kind of mentoring those students and advising them, it's actually fun to watch them kind of absorb knowledge, experience expertise from different people from different background, and they kind of help each other. We have computer scientists who are excellent at implementing computer codes and making those computer code more robust and reliable. We also have people from physics background who knows their thing about different physics phenomena, how mathematical equations govern those physics phenomena, and so on and so forth. We also have mathematicians, we have engineers. There's all kind of people from all kind of different background. And then they are dedicated in contributing to the team in their own way, and then they're helping each other. And the strength of our group is that they can talk to each other. I mean, if you're from different background, sometimes people speak different language, depending on their educational background, and so on and so forth. But, you know, our guys are very good at communicating with each other and, you know, helping each other. And then it is just a joy to, you know, see them kind of absorb each other's expertise and experience and then, you know, grow to become a better researcher. What do you feel about this multidisciplinary setting from a student's perspective?
[00:25:36] Jack: I definitely want to hit on that communication aspect that you just mentioned there.
That has been one of the most vital, just kind of instruments that I've learned in the lab, because like you said, we're working with mathematicians and people that are physicists and computer science background. And like you said, at times it's almost as if we're speaking different languages when we're discussing these technical fields. And so learning to actually navigate and communicate with our different collaborators has been just something very impactful for our lab, as we've seen building in different students and our collaborators with that. And so that has been amazing. In just working with the broad aspect of different backgrounds. And then coming in myself, too, as a computer scientist, I was almost a little apprehensive of, oh, we're doing all this physics work. You know, I took Physics 1, 2 in my undergrad, but, you know, what are we actually going to do with all these complex physics. But it has been really awesome being able to just build a team that leverages each other's expertise in order to achieve our goals of kind of the research projects that we're working on.
[00:26:36] Stephen: You are a great example of a person who demonstrated huge growth.
You've been with us for two years now. I still remember when you first came in and your level of academic maturity and, you know, things like that.
And then, you know, comparing that with where you are. I mean, I'm proud that you, you know, were able to achieve that huge improvement in almost everything. As an advisor, I'm kind of curious to hear what were the challenges or experience of going through that process.
[00:27:08] Jack: That's a great question. I think one of the personal aspects that Motivates me and spurs me on is working with just high performers. And so joining in a lab that had a lot of people performing at a high level and developing products and different research that was, you know, very impactful to the field, just motivated me of like, okay, I really need to kind of just sharpen my skills and ensure that I'm contributing to our vision and mission. And it was quite a learning curve at the beginning. I was like, okay, wow, this is a lot of work to do, managing classes and then hopping in on research right away. But just how we have delegated out how we do our lab meetings, I think has been very beneficial in a variety of aspects of one, working with fellow students that have already been in the program and working on the research, allowing us to just be mentored by them. And then when we have our presentations or reports that we're doing and we get our interaction with you as a lab, getting that direct feedback and the mentorship that you've provided to us, being able to effectively just build upon things that we already do well as an individual, but also highlighting, hey, like Jack, you might have been weak in this area. Let's kind of focus and target on that because I see a lot of potential there. And so receiving that feedback from you and from others in our lab, where we really are just kind of like, hey, we're here to grow and we're here to get better with the research that we're doing. But that also means, hey, there's some areas that you need to work on. And I think that you've done a good job of really showing those to me as well as others in our lab to build us into higher perform students.
[00:28:42] Stephen: That's good to hear. All of this is possible, I think, because of sponsorships from the government agencies and private partners and things like that, because we've been very lucky with funding and supporting our students. So that kind of gives a huge degree of freedom in terms of us kind of working together and collaborate with each other. If we had a financial pressure or things like that, then, you know, we would have not been so successful in terms of, you know, working with each other and, you know, things like that.
[00:29:16] Jack: I think a major aspect of that is how practical some of the applications that we're working on, where we're looking at flooding forecasts, or we're looking at actual, you know, development of weapon systems, or we're looking at the human motion pose on sports, how we can actually improve the performance of athletes. I think that a lot of our collaborators and Our funding, they can see, hey, this is practical relevance here in the real world.
[00:29:40] Stephen: Yeah, that's an excellent point. Let's not do research just for the sake of doing research, right? We gotta solve problems that, you know, matters to people. We gotta solve problems that are actually relevant to, you know, everyone's daily life. I think part of the reason why we were quite successful with funding was probably because, you know, we have this mission oriented mindset of delivering, you know, solutions, delivering knowledge to those partners and, you know, government sponsors of what they want. I also want to mention that now is the time I think the science has somewhat mature. We gotta start thinking about translating this into something real like a professionally developed software tool that people can actually benefit from. Because a lot of our research codes and the knowledge exist in a form that is not so user friendly. I guess only highly skilled people can understand the code and actually execute them.
And in order to yield more practical benefit to a broader part of a society, I think we got to turn this into some actual software program that people can download and use and benefit from it.
[00:30:54] Jack: That always gets me excited on just the opportunity there of how we're developing that. And again, I think it broadens the accessibility to so many people to use our tools. So I find that to be a potential area for large, large impact.
[00:31:07] Stephen: Yes, absolutely.
[00:31:09] Monica: Thanks for listening to this episode of Data Points. More information can be found @datascience.virginia.edu. and if you're enjoying UVA data points, be sure to give us a rating and review wherever you listen to podcasts. We'll be back soon with another conversation about the world of data science.
