We Get Privacy — Episode 22: Rethinking Cybersecurity in the Age of Unleashed AI

Transcript

Joe Lazzarotti
Principal, Tampa

Welcome to the We Get Privacy Podcast. I'm Joe Lazzarotti. I'm joined by my co-host, Damon Silver. Damon and I co-lead the Privacy, Data and Cybersecurity group at Jackson Lewis. In that role, we receive a variety of questions every day from our clients, all of which boil down to the core question: How do we handle our data safely?

In other words, how do we leverage all the great things data can do for our organization without running headfirst into a wall of legal risk? How can we manage that risk without unnecessarily hindering our business operations?

Damon Silver
Principal, New York City

On each episode of the podcast, Joe and I are going to talk through a common question that we're getting from our clients, and we're going to talk it through in the same way that we would with our clients, meaning with a focus on the practical. What are the legal risks? What options are available to manage those risks? What should we be mindful of from an execution perspective?

Joe, our question for today is an interesting one. There's been a lot of news recently around some of the major artificial intelligence (AI) labs, like OpenAI, Anthropic and Meta, having different types of scenarios where they effectively had to acknowledge that they lost control of some of their AI models and AI projects. This has raised concern among companies across lots of different industries around what this means in terms of their data security and cybersecurity.

Just to level set, Joe, do you want to share a little bit about what happened over the summer, some of these incidents, and then we can get into a little bit about what it means for managing a data privacy and protection program and, of course, what to do about it all?

Lazzarotti

This summer has been a little bit scary for a lot of folks using AI, and even those not using AI, after reports came out that several large language model providers announced that, despite efforts to contain AI agents in their environments, circumstances permitted those agents to go outside of the environments in which they were intended to be contained.

In an effort to achieve the goals that they were set out to perform, perhaps in ways that may not have been intended, those agents got out and began to engage in activity that the models did not really anticipate. Since then, there's been quite a bit of commentary around how these tools are being structured. How is it that companies can contain them if the developers are having difficulty doing so?

It's raised a lot of questions around the types of tools that are out there, both in the U.S. and around the globe, and whether and to what extent companies that are using, or may not be using, AI are vulnerable when these agents get out and engage in activities that could result in compromises to those systems.

Damon, as a next step, feel free to expand on that. The sense I got is that there's some significant concern around companies and organizations that are worried that the power of AI can be unleashed to make it much easier to compromise their systems. They may not be equipped to fight back at this point. What's your thought there?

Silver

That was definitely something that jumped out at me in listening to and reading coverage of some of these incidents. It seems like, while there is a lot of potential for AI to be helpful from a cybersecurity and data protection perspective, at this point in time, the uses of AI for malicious purposes are ahead of the uses for defensive purposes.

In particular, it seems like AI can be used very effectively to find and exploit vulnerabilities in current systems. That is happening more quickly than the systems can be updated to address those vulnerabilities. To my understanding, a lot of the technical safeguards that are currently in place, understandably, were created based on the speed at which and the methods by which human bad actors would try to detect and exploit vulnerabilities, but AI moves much, much faster.

It can find a vulnerability and, apparently, basically on the fly, improvise the tools it needs to exploit that vulnerability. In the past, if human bad actors found a vulnerability, they first had to start writing code, develop tools and figure out ways that they could use the tools for one incident and then adapt them for another.

That's all happening basically in real time, or at least on a much more accelerated timeframe. This raises a question for a lot of our clients who have invested very heavily over the last number of years in things like multifactor authentication, endpoint detection and response, and encryption. They have put a lot into these technologies to try to address the threat as they knew it.

Now, all of a sudden, it looks like those safeguards, while not worthless, are probably going to be overcome with more regularity than we've been accustomed to. We've been accustomed to a pretty high volume of intrusions as it is.

Joe, in terms of mindset, if we operate under the assumption, at least for some period of time, that AI is going to make it so bad actors are more prolific and more successful, and that some of the safeguards we currently have in place are just not going to be enough, what should the strategic priorities be for someone who's in a chief information security officer (CISO) role, someone who's on a board, or anyone who's trying to figure out, big picture, where they should be focusing their money and attention from a data protection and cybersecurity perspective?

Lazzarotti

That's going to mean some tough decisions for organizations, especially as cultures have tried to evolve to improve cybersecurity, thinking that if only they could strengthen and harden their environments, that might be what they need. I know we've talked to a lot of organizations about that.

At some point over the last several years, given the number of data breaches, the idea became that it's all about preparedness. Everybody says it's not a matter of if, but when, right? In some ways, we're already getting to that point, but AI has accelerated it dramatically.

When we hear things like "reasonable safeguards," what does that mean now? I don't think people really knew what it meant before AI. The whole idea of what's reasonable is not always as clear as we want it to be.

It's really about focusing on being prepared and knowing that, hey, look, this could happen. We may or may not be able to prevent it, and we think we have a good story to tell around the reasonable safeguards, but we really have to be prepared to respond and to do it as best we can to minimize the interruption of the business, satisfy our regulatory obligations and have that plan practiced. 

Those are a lot of the same things we've talked about over the years in terms of an incident response plan. That's the sense because, if you listen to some of the commentators, it can be pretty scary, especially in the short run, the next two or three years, where the two sides kind of battle it out over who's able to have a more successful use of AI to carry out their goals.

That's the sense I got from what I'm hearing and what we're experiencing, and the way we've approached things with clients is a good way to help the organization as best we can.

I'm curious about your thoughts on that. Also, besides just the information technology (IT) security controls and what we're calling reasonable safeguards in the traditional sense, are there other things that, more proactively, we might be able to do to minimize the risk of these situations and lessen the obligations or challenges if there were some type of compromise going forward?

Silver

All of this, Joe, is good cause for a lot of organizations to take a really big step back and focus even more intensely on an issue that I know you and I have discussed a lot offline and that we've discussed with many of our clients. We did a podcast episode on it: data minimization.

AI is really interesting because, on the one hand, it makes data minimization a lot harder in the sense that it makes it much easier to do stuff with your data, which creates this incentive to have more and more data because, all of a sudden, you can unlock all this potential in it.

But if we operate under this assumption that we are more likely to have intrusions than we were previously, one of the best things we can do to protect ourselves, both from the standpoint of data breaches and the ensuing class action litigation and regulatory investigations, in terms of protection of our intellectual property (IP) and other sensitive company information, and even in terms of disruption to our business, is to really be thoughtful about what data we need to collect in the first place. What we're going to be doing with that data, and how long we need to keep the data, at least in our active environment as opposed to a more secure archive.

The bigger a footprint we create within our systems, the more the bad actor who gets in via AI is going to be able to do. These attacks are going to become more and more sophisticated in the sense of moving laterally and being able to leverage. This is another thing that AI raises as well. There are all these great integrations between different applications.

There are ways for those applications to talk to one another and, through a single agent, be able to pull data from all these different places. That's all great in some respects. But if you assume that, again, a bad actor is going to get in, now they may be able to pull from lots of different places much more easily without having to have access to accounts affiliated with each of those siloed applications. Everything's going to be more centralized.

At the end of the day, a lot of these things come down to a business decision around the value of the data and the value of what AI can do with the data. On the flip side of that, if you are not putting guardrails early on around what you're collecting and what you're doing with it, the magnitude of these incidents will grow and grow.

We had already been seeing that before. If we look at some of the reports related to the cost of a data breach, the trend was definitely up and to the right already, but we could see a real acceleration of that. That was one point that definitely came to mind. There are others as well. Are there things that you've been thinking about on this front, Joe?

Lazzarotti

I just want to dig into something you mentioned about data minimization. Some of what I'm hearing is, well, we minimize data in a few ways. One is we don't collect as much, or we try to have a good, practical retention and destruction program. But we also have a lot of de-identified data, and we feel like we're pretty comfortable with that.

What do you think about that, Damon, in terms of the de-identifiability of data and how successful that's going to be as a strategy going forward?

Silver

I'm going to try not to say "de-identifiability" too many times because that will get me all tied up. It's an interesting point you raised because, again, even pre-AI, there was this overreliance on de-identification and aggregation of data.

What I mean by that is that a lot of times, for example, in a vendor agreement, the vendor said they’re going to use your data for their own marketing purposes and to develop their own products, but only after they’ve aggregated it or de-identified it. A lot of times, the standard for doing that wasn't defined.

As a practical matter, in many instances, it's not that challenging to take something where someone's name was removed and then, based on all the other pieces of data, cobble back together who that person is. With AI, obviously, that task becomes all the easier to accomplish.

Even data that may have been de-identified under previous technical capabilities may now be even easier to re-identify. While aggregation and de-identification are definitely still tools in the toolkit to leverage, along with archiving and purging data, organizations are going to need to be mindful that they have to really do it right.

If they're not going all the way and doing it in a way that really de-identifies the data, then they are putting themselves back in the same place. They've achieved some marginal mitigation, but it's not going to be anything that saves the day for them if they have an incident.

Lazzarotti

I was talking to a healthcare organization, and they mentioned the Health Insurance Portability and Accountability Act (HIPAA) de-identification rule, which was issued in 2000, about 25 years ago, without any real understanding of how large language models would work today and all that good stuff. It really makes you wonder whether that's an appropriate standard going forward, but we'll see.

As far as some other things, these are all good steps to take generally, but perhaps even more so when trying to anticipate threat vectors from AI tools and whatnot. Really, continuing to manage access to systems and ensuring that administrative access is locked down can be important.

In some cases, segmenting networks and having the ability to thwart the lateral movement that you mentioned, Damon, across different networks can make the really important, sensitive data harder to get to. If one of your systems is compromised, it makes that data a little harder to get at.

The only other thing I would mention, and you can speak to this as well, is getting back to some basics around incident response. We talked about this a little bit earlier, but it's about really figuring out how to plan around that and being ready to respond, which is something that organizations do in a lot of different ways, but it might be a place to rethink.

How do we really do that as an organization, and are we really prepared to address these types of incidents that, from what we're hearing, are going to increase pretty dramatically over the next couple of years?

Silver

It's part of incident response, but it touches on lots of other areas as well. If you're not doing regular data mapping to really understand all of the different tools, applications and systems, what data they store, and how they interact in terms of data being accessible or disclosed from one to another, things are going to feel even more chaotic than they otherwise would if you have an incident.

Both of us have worked with some clients who do have very sophisticated data mapping. They don't always know exactly what was stored in a compromised system, and they might have to do a little bit of research at the front end to try to figure it all out. But they have a pretty good idea that this system was used for processing this type of data for these types of activities, and the same for various other systems they've used.

At the other end of the spectrum, we've had clients who really had no clue. Any kind of data could have been in scope. They really didn't have an understanding of where things were stored, how systems were integrated with one another, what was stored locally, what was in the cloud, whether they had backups, or the status of those backups. It really is a different level of chaos, for lack of a better way of putting it, when you don't have the data maps to work off of.

If we're talking about, say, a ransomware attack, you might be locked out of your systems entirely, and a lot of your data may be permanently compromised. You have backups; you don't. It can take a very long time to restore from the backups. During that whole period, you're in the dark, and you're uncertain where exactly to go in your incident response plan because you don't know what you're responding to.

As organizations are experimenting with lots of different tools, doing lots of different pilots and having some tools that are going to integrate with a lot of their existing applications that process data, it's really important as part of that process to take a step back and ensure that there is data mapping going on or updating of existing data maps so that you don't lose control of things, you have visibility into what's going on, and you're able to do some planning around that.

Joe, you can speak to a related issue, because this is a place where data mapping can fall short, which is vendor management. As many of our clients are experimenting with all these different tools and different use cases for the tools, it does seem like vendor management, which obviously was important before, has evolved in some respects and certainly taken on additional importance. You could speak a little bit to that before we wrap up.

Lazzarotti

In thinking about this topic and also thinking about conversations that I know we've both had with organizations about particular vendors, they say, "This is a big vendor. They're in business, they have locations all over, they have tons of IT employees, surely they must know what they're doing, and we think we're probably in good shape."

To hear over the last summer that several of the largest AI companies that invest billions of dollars into this technology had their own incidents kind of humbles all of us in a way that says, look, we have to continue to be vigilant, we have to trust but verify, and really be more adept at asking the right questions.

It's not going to give you all the answers, but it's really about, one, doing what you need to do to feel reasonably comfortable; knowing that the grass may not always be greener on the other side; knowing where the gaps are as best you can; and having some good contract protections in the event of an incident, where you have to assume that it'll happen.

Think about, to your point, Damon, data minimization. What do we need our vendor to know? What do we want our vendor to do with that data? Try to get that as certain as you can. Know who your vendors are. These are some basic things that we've covered on prior podcasts. It really is critical to do the best you can to understand who you're working with.

One last thing is to think about downstream. Who are your vendors using as their vendors, and how do you manage that? Practically, how far downstream can you go? Or is it enough to just ask the vendor what their process is for assessing service providers.

I do think vendor management is really critical. It's a challenge for organizations, but, again, putting together a good process and documenting what you're doing on a consistent basis really can help manage liability going forward.

I don't know if there's anything there, Damon, you want to jump in on.

Silver

That's all spot on. I guess the only thing I would add, it's something that we've talked about in other contexts, is that there is sometimes this propensity to think that data security and cybersecurity are an IT function. If we just get, to your point, the right investment in hardening our systems, that's going to take care of most of our problems.

We have policies, or we don't. We have some templates, and we go through the motions of doing some type of risk assessment or data mapping, really just to check the box. But at the end of the day, we're really putting all our chips on the technical safeguards.

That was always a flawed strategy, but now it's become one that is causing you to assume very, very significant risk. If we're going back to thinking about standing in front of a judge or dealing with a government agency investigation, we do not want to be in a place where that's the only story we can tell.

Certainly, that's a piece of it, but if all we can say is that we had these technical safeguards in place, but we weren't doing data minimization, we weren't doing data mapping, and we can't show that we vetted our vendors and got good contractual protections in place, we're going to be playing a really weak hand.

These types of situations are going to arise. It's just a matter of time. The specifics of what they look like may vary, but in some form, these things are going to happen to a lot of organizations across the spectrum. They really need to start thinking more broadly about what the different levers are that they can pull to put themselves in the best position when these types of incidents do occur.

Lazzarotti

We’re going to have to wait and see, right, from what the experts tell us. But this has been a great discussion. Thanks, Damon.

Thanks for listening. If you have any suggestions for topics for future episodes, please reach out to us. We'd love to hear those. You can email us at Privacy@JacksonLewis.com. Damon, thanks as always.