Data Meets Art

[00:00:01] Monica: Welcome to UVA Data Points. I'm your host, Monica Manney. In this episode, we explore the intersection of data art and storytelling. Our guest, Nathalie Miebach, is an internationally recognized data artist in the School of Data Sciences inaugural Artist in Residence. Using materials like reed and paper, she transforms complex data sets into woven sculptures and musical scores, inviting us to view and even hear data in new ways. Joining her is Alex Gates, assistant professor of Data Science at the University of Virginia. His research examines how patterns of connection shape creativity, innovation, and discovery. Together, they discuss what happens when data meets art. [00:00:46] Alex: So to kick us off, I thought it would be really fun to introduce the audience to one of your works. For those of you who have seen the posters for this event or visited our exhibition in the School of Data Science, you've probably already encountered of Nathalie's most striking works, Ian and Nicole. It's a vivid sculptural collage that seems to swirl with motions, ribbons of color and weaving like the storms that they represent. Beneath that movement lies data, temperature, wind speeds, barometric pressure, ocean currents. So now let's start there. For those listening who haven't seen it in person, could you describe the work and what it means to you and what the story is that you're trying to tell with it? [00:01:24] Nathalie: Sure. Well, you did a great job describing it visually. Ian and Nicole is a piece that I made in 2023, and it's about Hurricane Ian and Hurricane Nicole. Both of those storms caused a lot of devastation in a lot of areas of the Southeast, but I'm particularly focusing on Daytona beach, where my parents have a small condo, and their condo was compromised as, but particularly the neighboring areas as well. And so the piece is really about the actual storms that caused that devastation, both Ian and Nicole, and as well as a look at the recovery efforts and the rebuilding efforts a year and two years later. So the piece itself is a very colorful collage, and it's perhaps a little bit surprising to see all that color thinking of this as a fairly devastating event that caused a lot of damage and destruction. But one of the things that surprised me by going down there every few months and visiting my parents and helping with the recovery efforts and helping them negotiate and rebuild in some ways was the optimism that I kept seeing in people. This idea of, we're going to rebuild, we're going to rebuild faster and stronger and we're gonna make it kept coming up. And so the colors, the colorful objects and umbrellas are really trying to talk about that kind of optimism that I kept seeing, even if it feels absurd. Sometimes as well. There are two central visual features in that piece. And these are two rectangles and they're the size of beach towels. And that's deliberate. The one on the left translates data from hurricane, and the one on the right translates Hurricane Nicole data. And they're both translated in a way that they can be read as a musical score. And then surrounding those two rectangles are umbrellas and flowers and little visual elements that speak about the sort of idealism of beach life and paradise. But then when the further you go to the right, you begin to notice that there are these life. Lifeguard towers that actually have these strange looking buildings underneath it. And so you begin to see that these lifeguard towers actually show you how many houses are gonna be underwater depending on how high the sea surge is. And then the further right you go, you see these strange tree houses that are made up of images that I took while on the beach of the destruction of houses that have been demolished, of structures that washed onshore. And these images of the destruction sort of rebuild themselves into these tree houses that then have these beach umbrellas and these beach chairs. And this idea of rebuilding paradise at all costs sort of comes out. So there's this. There is this sort of an attempt at trying to understand what that optimism is, and if it really is optimism. I keep sort of struggling with that word. But that's basically what the piece is about. [00:04:46] Alex: Yeah. And it's so striking how you've interwoven both the physicality of the pieces with the abstractness of the data that they're representing. Right. And the stories that it's trying to tell. That it's not just the numbers on the page, but it's the objects that those numbers are representing. [00:05:00] Nathalie: Yeah. And actually I'm glad that you pointed that out because it is unusual in some ways that this piece is a combination of a lot of abstraction, but also a lot of images. When I was trying to gather the research for this particular piece and looking at the data, what I kept going back to were the images that I took on my phone while I was on the beach watching this community reclaim the beach and the economic rebuilding that was going on all around us. And it was actually the images that kept being really important in giving these numbers a context. So that's why they're in there, because I feel like they were very important lenses through which to understand, or maybe not even understand. I think that's trying to explore the rebuilding efforts that I was watching and trying to make sense of it. And I think the images helped for that purpose. [00:05:56] Alex: I think it really also represents a bit about what we're trying to accomplish with the data and Artists Residency program. You've said it before. The data only becomes meaningful when it's translated into human experiences. And I think that a piece like Ian and Nicole really illustrates and exemplifies how human experience can be expressed through these numbers and visualizations. It'd be wonderful to hear how the storylines evolve for these types of pieces. Where does the inspiration come from? And how, for example, do you find the data first and then you make the piece, or do you have the emotions and the piece and then you look for what data is capturing? Some of that. [00:06:36] Nathalie: Yeah. Okay. Yes. So all of my work begins with numbers. It always has. And that hasn't changed in the 20 years that I've been doing this work. And so every piece begins with research, gathering data related to an event, such as an extreme weather event, such as a flood or hurricane or a forest fire, or sometimes it's data related to looking at a particular ecosystem and seeing that change over time. So it begins with numbers. And then I do the research for several months. And I'm not just looking at numerical information. I'm also gathering anecdotal data. I'm doing a lot of reading. How are newspapers and media outlets covering these extreme weather events, how our community's rebuilding two months after the event, three months, six months. So there's a lot of gathering processes. I become this sort of hoarder of stuff, of numbers and stories related to the event that I'm looking at. And what I'm looking for is some sort of traction, something that I want to tell the story about. And that generally, sometimes it takes. Sometimes it comes up immediately, and sometimes it takes a long time for me to find it. And oftentimes it comes in the form of some sort of metaphor, something that gives these numbers a kind of context. And to kind of give you an example, when I was doing a series of works on Hurricane Harvey that hit Houston in 2017, I was particularly focused on the Houston area. I did a lot of research for that one. And I was like, six months into it, and I came across this data visualization in the New York Times that showed the Twitter messages that were sent during the storm when the 911 system broke down. And it was such a beautiful visualization. Because you had the highway system of Houston, you had little red dots that would show you where the Twitter messages were sent. And because the storm, the way that the storm behaved, it was very Interesting. Hurricane Harvey sort of went. Came off the waters, moved over Houston, sort of parked itself over Houston for a week, and then retreated back over the waters, back to the Gulf of Mexico, and then came back onshore further east. And you can see these dots following and literally drawing the path of that storm as it comes over Houston and then retreats back to the ocean and then goes back into the eastern part near the Beaumont area. But what was really so poignant about this visualization is you had this very analytical way of showing you where those Twitter messages were sent, as well as the Twitter message content. So, you know, stuck on a roof, no more baby formula. I haven't talked to my mom in two days. So these really, really emotional messages. And that was sort of the breakthrough for me. And then I used these Twitter messages to start building bodies of work. And so sometimes it's something like that, sometimes it's a metaphor that comes up earlier. So it sort of depends, but that's kind of how it starts. [00:09:59] Alex: And one thing that many of our listeners might feel is when you work with data, there's a lot of constraints around it. Right? You can't fabricate pieces of it that don't actually exist within the confines of the data. And the modeling assumptions that you put on the data really constrain the lens that you can take to look at what's the story being told there. So I'm wondering, as you're translating the data into more physical objects and you're dealing with the constraints that a data set might impose, how do you navigate that? And do you feel like you're dancing with the data, or do you feel like you're wrestling in it, trying to shape it into a form that you see. Right. [00:10:38] Nathalie: I feel like actually I'm more of a mediator between the data and the sculpture. Because once I start translating some of the data that I'm looking at into a sculptural form, the sculpture starts having a say in this too. And so it starts to bring up suggestions of what I might want to include in the translation process or. And that brings me back to more research. And so it's this back and forth between the data, the sculpture, and sometimes the musical score comes in at that point too. So it's this really sort of two way, sometimes three way conversation that I'm. That I'm facilitating between these three, three players. And each of them bring possibilities, and each of them bring limitations. And I think the fact that it's a process that is a little bit more free form, I have to kind of figure out when I have to move back to the sculpture, when I have to move back to the data, and when the music comes in really also means that I don't have a formula in how I work with data. And sometimes that's a little bit frustrating because people want me to tell them, well, how did you do this? And there's this sort of expectation that I have this ready out formula, but I don't because every story requires its own approach. Every story about a weather event is different. And so I have to figure out a way to kind of renegotiate that three way relationship anew every time. [00:12:06] Alex: Yeah. So really it's interesting that you represent yourself outside of the conversation that's happening as more of a mediator or catalyst for that conversation rather than an active participant. [00:12:17] Nathalie: Oh yeah. I've learned long ago that if I don't listen to the sculpture, I'm a fool. And the same thing with the data and the music. So it's. They're really the ones driving the process. [00:12:27] Alex: And many of your pieces have at their root basket weaving. So I'd love to hear a little bit about why basket weaving and how did that weave its way into your artistic storyline. [00:12:38] Nathalie: Yeah, so basket weaving came into my process somewhat serendipitously, but also at the very beginning of when I started to use sculptural methods of exploring to investigate science. So in many ways, I became a sculptor because of my interest in science. And right around 2000, I was finishing a master's degree in art education and I was writing this curriculum on the study of time and wanting to use the visual arts to study different concepts, different ideas about time that different disciplines have. And I found myself having to take some science classes. So I took science classes at Harvard Extension School, which is their night school division, and ended up at the same time taking this basket weaving class. And as a very tactile learner, somebody who's constantly learning by taking things apart or building things, basket weaving sort of happened to be the medium that I was engaged with or that my hands were engaged with at the time that I was looking at astronomy. And so it was sort of natural that when I started to search for a way of investigating questions I had about astronomy, that I would be reaching to basket weaving, because that's what I brought with me to the lecture hall. And then the very first piece that I made were models of diagrams that I saw in the astronomy textbook. And these are beautiful, beautiful pieces. There's the Hertzsprung Russell diagram In particularly that I was really gravitating towards. It shows the evolutionary stage of every star based on luminosity and surface temperature. And the very first thing I did with basket weaving was take that two dimensional graph and turn it into a 3D donut, essentially. And it was huge. It was like three feet in diameter. And I handed in to my professor and he accepted it. And I said, yeah, if this is how you learn best, let's do it. But what it did, it opened up this idea that maybe I could rethink the basket and not just create these 3D structures with it, but what if I think of it as a matrix, as this grid that actually can be used as a translation method to give numbers the ability to dictate the form. And so that's when I started to explore using the basket weaving matrix as a way of translating information related to astronomy. So using the basket essentially as a clock with vertical elements representing the hours of each day, and then the horizontal elements being the data that I'm translating. And so over time, because I'm using this natural material called reed, which is a material that has a lot of tension and the kind of material that if you enforce too much strength on it, it'll break. And what that means is that it's really more the numbers that are distorting the form than me putting any kind of pressure on it. So breed and basket weaving became this method of revealing a kind of dimensionality that numbers were showing in 3D that weren't coming out through the graphs that I was, or the spreadsheets that I was, that I was taking the data from. And so the very first pieces with basket weaving were these really almost like organic looking, strange, morphed structures that were really distorted forms made by data. But then after a while, I started getting a little frustrated with this, because working with data can be very loud. When you're working with data, at least for me in an art process, data has a very strong presence and there's a lot of expectations placed on data. And so when people were looking at my sculptural pieces, they kept asking, well, what does the sculptural piece tell us about the data? What am I learning about the data? So the sculpture kept being this sort of vessel or the scaffolding on which the data sat, but it didn't really have a voice in how it might be interpreted, how the data might be seen or read. And so that's when I started to rethink the role of sculpture in data and wanted to understand if There was a way that I can bring in a more nuanced lens through which the data, especially in its relationship to the human experience, could be interpreted. And that's when metaphor came in. That's when musical scores started coming in, and that's when the sculpture really started to sort of breathe a little bit. That's when installations came in and the sculpture started to have much more of a voice in how the kind of lenses I was creating through which the data could be read. [00:17:34] Alex: Could you give us a little insight into, like, the algorithm that's in your head behind how you translate some of these data functions into the actual weaving of the basket? [00:17:45] Nathalie: So all of my pieces have some sort of time structure, whether it is literally a woven timeline, whether it is the basket matrix on which the data is woven, and because the data changes over time, it distorts and twists this matrix, or it is a metaphor that I build on which the data sits. So it really sort of depends on the different approaches, but there's always some sort of time element in there on which it sits. [00:18:22] Alex: And one idea we have is that of forecasting, where you're trying to predict what the shape of the data is going to be in the future. Do you try to do that as you're evolving the basket form, or does it kind of take you for the ride and give you some surprises along the way? [00:18:37] Nathalie: Yeah, people often ask me whether or not I use any kind of computer modeling program to figure out what the form's going to look like, and I don't. I'm in some ways, very low tech. I have the numbers. I use the computer and the Internet to gather data that I don't collect myself. But once that's done, I like to stay in my sculptural world and just weave the form. And part of it is also, you know, if you know what it's going to look like, what's the point in building it? It's a part of that. Yeah. It's the discovery. It's this element of surprise where you don't know where the sculpture is going. And then also having to deal with gravity. I mean, some of the forms that I'm building with data cannot stand up straight. They're going to fall over, they're going to fall apart. And so I have to problem solve that. I have to problem solve and how to make this object sit in 3D space. And that means I have to sort of rethink how the data is going to sit on the structure. And that wouldn't come about If I already had this prediction on the computer telling me it's going to fall over. [00:19:45] Alex: Yeah. The physical integrity of the piece is crucial. [00:19:48] Nathalie: Yes. Yeah. The other thing I love about baskets is that basket weaving is a really. Basket weaving is really this matrix and there's different techniques that create different matrixes within the basket definition. But your eyes are pretty much useless when you're learning how to weave a basket. It's your hands that have to learn how to listen. So it makes me very aware of how our senses give us entry points into certain types of information. And I like the fact that I'm working with a medium and a process that forces me to really listen to my hands and not just to my eyes. [00:20:33] Alex: Maybe we should talk a little bit too about the themes that you touch upon in a lot of your work. A lot of it is about weather, data, storms. Can you tell us a little bit about why you choose those as the central focus? [00:20:48] Nathalie: So I started focusing on weather right after grad school, partly because I wanted to find some sort of subject that I could very easily collect data myself. Prior to that, I had always been using other people's data. And the sculptures that I was creating out of this were having, you know, were taking this particular form. And I wanted to see, well, what happens if I'm the one that actually collects it? How does that change the way I think about the sculpture? And so I found myself on Cape Cod for two years off season, living in this small town right at the tip of Cape Cod called Provincetown. And it is surrounded by beautiful beaches. And so for 18 months, I would go out to specific places on the beach and collect data on a daily basis. I'd go to the local hardware store, find any kind of low tech data collecting device I could find. So we're talking about, you know, thermometers from the kitchen aisle, a rain gauge from the garden aisle. I would build my own data collecting devices and I would schlep all this stuff out to the beach every day and collect numbers as well as writing down what I was actually observing in my journal. So what kind of birds were out? What was the color of the water? What sort of plant life was washing up on shore? Because one of the things I learned is weather is this amalgam of systems that interacts with an environment. And to really understand weather, you have to understand the environment in which you are gathering the data from. So it's not just about understanding weather, it's also about understanding the ocean. Since I was right by the ocean and that interaction between land, ocean and air. And that interaction reveals itself very slowly. And even after 18 months, I felt like I barely understood weather. And that was like, wow, what a gift to have a topic that the more you learn about it, the more you realize it's an endless, endless, you know, an endless pot of questions and mysteries. And over time, I started to not just want to understand weather as a scientific system, but also focus more and more on how we as a species respond to weather, especially in the context of climate change. Weather instruments are metronomes. They can measure the weather every minute, every five seconds. They're very good at that, but they're not necessarily very good at taking a more holistic understanding of weather as a human being. That's not how we measure weather. We're constantly responding to whether weather is this witness that's with us from the day we're born to the day we die. There are moments in our lives where we really remember the weather, other times where it falls into the background. Weather is always with us. And I was getting more curious about how we are responding to weather as a species. And this became particularly when I started to look at hurricanes and how communities are responding after extreme weather events like that, how they're rebuilding their community, and the kind of sometimes contradictory, complicated responses that can ensue out of these events. And so weather became much more of a lens to look at us as a species of how we're responding to climate change. And again, when that switch happened, that's when the sculpture had to change, and that's when the translation process had to change. So that's when music came in as a way, or as an attempt to try to bring in nuances into the translation process without actually changing the numbers. That's one thing that hasn't changed throughout this process. That even though in some ways you could say, well, you know, you're dealing with human experience, that is going to. That's. That's very messy. It's going to. It's going to really change the way that the data is. Is being translated. It's making it way too. Too subjective. And so one of the things that I've always. That has always been very important to me is that. That the numbers never get changed for any reason. Even if aesthetically it might look a lot better if the wind was doing something else at this point. The numbers always wins. So if aesthetically it doesn't work, then I have to find some sort of other way of visually changing it, because the integrity of the number is important. If I start fudging the data, it's no longer data, and then it sort of defeats the whole purpose of trying to make work that can, if not always, comfortably sit between these worlds of both visual arts, craft and science. [00:25:23] Alex: And I love how you, like, embrace the idea of a socio technical system. Right. You have the people, you have the, the environment in which they're embedded, and you're using the two to kind of shape the story about how we're interacting with the weather and vice versa, how human behavior reflects the weather patterns that are affecting it. Like in the story of your Twitter inspiration, right? [00:25:46] Nathalie: Yeah. Tell me a little bit about what you're hoping an artist in residence can bring to the data science community in your school. [00:25:56] Alex: So I really see analysis and artistry as two sides of the same coin. Right. Every act of modeling and engaging with data involves a ton of choices. It involves choices on what to include, involves choices on how you're cleaning it, what you're going to remove or not remove. It involves choices about what the storyline is that you're trying to tell and what is the consequence of the data that you're viewing. And so in that sense, I think that every data scientist is already an artist themselves. They're shaping though, that story out of abstract forms of mathematics and graphs, but they're not necessarily engaging with it in the way that a material artist would think about how that data takes form in front of them. And so what we were really hoping with the data and artist residency program was to bring that creativity to, to the school and show the community that the work that they're doing, exploring the data, actually can bring life to the data in brand new ways. [00:27:02] Nathalie: One of the things that I love about this residency program is that you are inviting different ways of thinking into the school and letting students discover a lot of different people working with data for different reasons and from different perspectives. And one of the things that we all have in common is there's so many people who use data for all sorts of reasons, making work towards all sorts of audiences for all sorts of purposes. But at the end, we're all looking at that same base material of what is data. And I love the fact that you are inviting artists to come and question that in the context of your school and what the students are thinking about and questioning. I think one of the things I've always looked for in artists residencies is an opportunity to break habits. So one of the problems that I find when I'm in my studio and I'm making work is that I Very quickly build habits around the way that I'm looking at data, how I'm translating data, how I'm building things with data, and. And that can feel very comfortable and at times starts to become predictable, but it's also very dangerous. And so artist residency programs are great opportunities to come into a new environment outside of your own references, outside of your own materials and habits, and really begin to talk to people who are also working with data and seeing how they are approaching it as a way of finding new ways or even just becoming aware of what your habits are around data. I think the hardest part is actually becoming aware of what are the habits that we bring with us to the process and whether or not these habits are healthy. So I'm looking forward in this residency to break habits, to see how other scientists and creative thinkers are thinking about data, what they're hoping to do with data, and how their questions can help me break some habits. [00:29:16] Alex: I love that. And I think that that's so true that as data scientists, we often get stuck in the habit of, let's take our favorite method, crunch the numbers, turn out some report, and move on to the next story without thinking about the conversation that we should be having with all parties around that data. Right. Both from where the data comes from, the fact that you go and you measure your own data in order to have pure knowledge of where the origins of that data are, the decisions we make around how we process it and mold it and shape it, and then how we release it into the world. And the fact that that data can be interpreted in many different ways to many different people. And it's important for us to be involved in those conversations. Just as you go and let your art speak for itself, but also engage in conversation around that art. [00:30:05] Nathalie: Yeah. And another question that I have that I'm very curious about asking people while here, while I'm here at the School of Data Science is what does craftsmanship mean to you and what does it mean to some of the students? What is craftsmanship in the context of data, and what does that look like? I have a very concrete idea what it looks like in the world of craft where I'm coming from. So I work with Reed, which is this natural material that has a certain amount of tension. So to really understand Reed, you have to. To really understand Reed and know what it can do for you in terms of building structures. With basket weaving, you have to break, twist it, burn it. You have to fail with it 100, a thousand times. So I know what failure looks like in the Context of craft. But what does failure look like in the context of data? And how, how do we create the sort of circumstances in which that type of failure, that sort of productive failure, the failure that actually leads to new learning, can be encouraged and fostered and not immediately kind of squashed down as becoming bad data? [00:31:16] Alex: I love that analogy of data scientists as a craftsman or person. It's so true that. So one of the primary goals of data science is to try to extract the story or the signal from the noise that when you're given a raw data set, it can sound almost like white noise, like there's no pattern, there's no clarity to what's going on. And the tools that we try to teach here are methodologies to extract that pattern and refine it to understand where the key signals are coming from and then how to amplify those signals. And so one way I view craftsmanship in the context of data science is how well have you understood the signal that's actually present within the particular data set? What, or how well did you get lost in all of the noise that's present there? There's another form, though, that I think we also find some tension with almost, and that's what we call rigor, or in particular mathematical rigor, or this idea that the laws and axioms of math and statistics require us to think about things in certain ways. But almost inevitably, data breaks every one of the assumptions that goes into a rigorous mathematical derivation of a particular formula or a signal. And so then it's a, it's a, I don't want to call it a battle. It's probably more of a conversation around how do you circumvent the assumptions that you need in order to make something work, to still come up with something that you can trust? Because once you start breaking the assumptions, then it can tell you anything. The data can tell you any story that you want it to tell you. And so it's a constant struggle to try to say, well, I know that I can trust that my data is telling me a true story and that I'm not over exposing it to certain requirements or assumptions, but I'm being rigorous enough that given the confines of the data, that I can have that trust. And so I think that those are the two primary analytical craftsmanships that we think about. But we also think here in the school of data science that data analysis is much more than just these analytical components, right? We fully embrace the design and storytelling component of it in that producing data driven argumentation is not enough. You have to actually craft the story and the context in which that data was taken abstracted, and the consequences of that data in order to get a full holistic picture of how that data engages with society. We also think very heavily about the implications of that. So what happens when you release one of these new AI models on society and how does that transform them? How people are going to be engaging with the data that underlies it? What are the ethical implications for having scraped lots of creative artists data in order to train your model and allow it to produce things that are almost indistinguishable from the original ones? And how does that affect the way that we should be approaching solving these problems on our end? And so I think that's another place where the true craftsmanship of data science comes through, which is thinking through the implications of the. The work and not just conducting the work like a monkey on a computer. So I'd love to hear a little bit about how you see public understanding of data evolving and the role that art might have in that. So in particular, we're living in a time when data shapes nearly every public debate, yet it can feel extremely abstract or inaccessible. How do you see art and visualization helping people to reconnect with the meaning of data in everyday life? [00:35:11] Nathalie: So one of the things that I hope is that art can reach a much broader audience in terms of educating people and making people aware of how much data shapes our own lives, but also I think, hoping that it can make people a little bit more aware of how biases can infiltrate these data visualizations or these discussions about data. One of the things that I hope to do a little bit more is getting us to think more critically about why we rely so much on the visual to understand data. There's such an enormous focus on translating information visually, and yet I wonder what are we losing out when, if we rely so much on the visual and not consider, well, what would it be like if we could hear data or taste data or feel data or what, touch data? Like, how can the ways that. How can our own senses create these filters? And I feel like by relying so much on the visual, we're sort of missing out on a lot of other potentials of how data might exist and the kind of stories that it can tell. [00:36:25] Alex: So one thing I see happening is that the sonification and alternative forms of visualization are data interpretation allow you to feel the data in different, different ways. Right. We experience the world through our senses differently, and we interpolate between the points and the signal very differently. And so these alternative forms of engaging with the data allow us to feel that data structure and pattern in different ways. And so I think that one it's for creativity. It's a lot of fun because you get to have a whole new insight into what's happening in the data set. But moving forward, I think it allows you to break out of that box of just showing your data through almost stylistic graph representations and really think about what's the shape of the data in other forms. [00:37:21] Nathalie: I love that. Yes. Thank you. [00:37:24] Monica: Thanks for listening to this episode of Data Points. More information can be found at datascience virginia.edu. and if you're enjoying UVA Data Points, be sure to give us a rating and review wherever you listen to podcasts. We'll be back soon with another conversation about the world of data science.