We Get Privacy — Episode 20: AI Procurement and Vendor Risk: Contract Terms Employers Should Watch

Transcript

Damon Silver
Principal, New York City

Welcome to the We Get Privacy Podcast. I'm Damon Silver, and I'm joined by my co-host, Joe Lazzarotti. Joe and I co-lead the Privacy, AI and Cybersecurity group at Jackson Lewis. In that role, we receive a variety of questions from our clients every day, all of which boil down to the core question of how we handle our data safely. In other words, how do we leverage all the great things data can do for our organizations without running headfirst into a wall of legal risk? How can we manage that risk without unnecessarily hindering our business operations?

Joe Lazzarotti
Principal, Tampa

On each episode of the podcast, Damon and I talk through a common question that we're getting from our clients. We talk it through in the same way we would with our clients, meaning with a focus on the practical. What are the legal risks? What options are available to manage those risks, and what should we be mindful of from an execution perspective?

The question for today surrounds companies; more and more companies are developing, deploying, or dabbling with various types of AI technologies, oftentimes what they obtain from vendors. Or they may be getting services from vendors, and should be suspecting that they're using some type of AI technology. In that process of contracting or procurement with a vendor, and knowing that the likelihood is that the vendor is using AI at a variety of levels, indirect to the services or direct to the services that the company's getting. What does that process look like from a contracting perspective?

Damon, help me frame the issue a little bit better and outline some of the high-level considerations that businesses ought to be thinking about as they go down wherever they are on the path of use of AI in their organizations.

Silver

You touched on this a little bit already, Joe, but as a threshold matter, it's important to understand and make sure that your counterparty is disclosing to you that it's going to be using AI in some capacity and then understanding what capacity that will be. The specific uses of AI are going to heavily influence the risk that it poses to your organization and also what is going to be important from a contractual perspective. For instance, if the vendor is only going to be using AI for its own internal administrative purposes, it's just going to be using it to organize information and do stuff that's purely internal; that's one thing. If it's going to be using it to perform analysis that ultimately later factors into the services to you, that's another thing. If it's going to be using AI to actually prepare the deliverables that you receive, that's its own thing. You do want to understand what role AI will play in the provision of services to you.

There are some other more targeted questions it might be helpful to drill down on in the front end if it's not obvious from the statement of work and your conversations with the vendor. Things like: is this party going to use an AI meeting assistant to record and take notes at all of your video conferences with them? Are they going to be processing any of your personal or company confidential information? Is that something their AI is going to be doing? Are they going to be using AI to support any decision-making they make on behalf of your company – for example, if they're going to be helping you with recruiting, are they using it to screen applicants? Say you're in the real estate context, are they going to be using AI to screen tenants? If you're originating loans, are they going to be using AI to screen potential recipients of those loans? Those are all things that are helpful to get some handle on at the front end, because that's really going to influence a lot of what's important to you from a contractual perspective. 

Joe, let's say you do some of that front-end analysis and you're now at the point of reviewing an MSA or a DPA and other related documents. What are some of the key provisions you're going to want to look at? One way I've been thinking about it is: what are some of the novel may be the wrong word, but really AI-specific terms that you want to look out for or, as a default, make sure you're adding into your agreements? What are some terms that exist very likely in your MSAs and your DPAs, but that you want to take a look at from an AI perspective because you're going to need to tweak them in some way to account for the use of AI by the counterparty.

Lazzarotti

You mentioned internal uses, and most MSAs have the term services. Services are a defined term. You really want to have the services defined; otherwise, you don't know what the vendor's doing. What are those services? Oftentimes those services are not going to include some of those internal things that you talked about. Things that look like they were just general office processes to manage the project, all of a sudden now with AI may become, although indirect, more important. For example, it's not a service, but it may not just be an office administration function like recording a meeting or summarizing notes from a meeting and then relying on those notes, and whether those notes have to be maintained confidentially. The notion of services and what exactly the AI is doing may bring to the surface more things than you may have thought about with how AI is used than the services themselves. That's one thing I'd be thinking about: expanding the scope beyond the services just to consider what AI is doing and how important it may be. That hasn't really been a concept for a lot of businesses. That's a term that has to now be added to and considered.

The other thing I would say, and there are other things, but record-keeping. We have record retention periods, and that's something new and specific to AI because, as we're seeing in some of these state laws, there are specific record-keeping obligations with respect to certain types of data pertaining to how AI models are generated and inputs and outputs that you have to really think about from that standpoint in terms of record-keeping. 

To that point, Damon, for some of those things or others, what are you seeing in terms of the competing interests? What's the response you can get back from the vendor? How do you counter that? We were all feeling our way through it a while back. It has gotten a little bit clearer, but when you talked about record keeping, people are like, well, which records? The statutes aren't really clear. Maybe you can talk a little bit about how the parties approach those kinds of and other similar challenging provisions.

Silver

A couple of places that I've seen somewhat heated negotiation as it relates to some of the areas you already raised are, one, vendors want to be able to use the data in some form to train their models because they want to improve those models and be able to provide services to other customers. A lot of times, part of the service they're providing to our client, as well as others, is benchmarking, data analytics, and that benchmarking and those analytics are only valuable if there's a pool of data across customers. A lot of times there is negotiation over whether there's some form of our data, whether it's aggregated or de-identified, that can be used for improving the service and for providing services to others. An important point to be mindful of there is that there are specific standards for de-identification and for aggregation and anonymization under various different frameworks, whether it's HIPAA or the CCPA or the GDPR. You do want to make clear in the contract that it's not enough to just say, we're going to only use anonymized or aggregated data. You want to make sure that the aggregation or de-identification or anonymization is being done in accordance with applicable standards. Those standards, if you look at HIPAA, for example, are extremely onerous. It's very hard to meet those standards. Is the vendor actually doing it? Are they willing to indemnify you if they do it improperly?

Lazzarotti

I was just going to say, as a side comment, that there was a recent case of AI being used to identify someone that the judicial system was investigating, a judge. I don't know if you heard about this case, but a bunch of people plugged in some information and were able to identify the judge pretty easily. You talk about these standards, and this is a tangent, but it goes to the point of: are those standards really adequate given AI? Are the de-identification standards really going to de-identify because of how powerful AI is to re-identify that data? It's something to think about. In the contract, are we even doing enough to really achieve de-identification? 

Silver

That's a great point. Another issue that has come up is that vendors do push back; I have seen, around having disclosure obligations and having to ask for permission to use AI, particularly when there is sometimes a blurry line between internal administrative use versus use that is materially impacting the services. That's been another place where I've seen the parties have to navigate arriving at a place they're both comfortable with. Because you absolutely do want visibility into your vendor's use of AI, but the vendor at the same time can't make a whole series of different commitments to its various different customers when it's using AI across a spectrum of activities, some of which relate to providing the services, some of which are more internally oriented or may relate to services being provided to others. It's definitely a place where it's not simple. The definitions matter a huge deal.

You were talking earlier about the definition of services. If you look at a lot of DPAs now and a lot of addenda related to AI, the definitions go on for pages. There's an important reason for that because the definitions drive a lot of the impact of the contract. A lot of the terms themselves may look similar across agreements, but the way that the definitions are crafted creates very different meanings potentially for the exact same provision. 

Another one, Joe, you could talk about a little bit is when you want to impose on a vendor an obligation to have human review and to take responsibility for deliverables, and what should that look like in the contract, and what are some of the flash points for negotiations there.

Lazzarotti

Having a human in the loop has become a popular term. From a regulatory perspective, a lot of the consideration there is what are the use cases that trigger additional obligations and how do we avoid that? Usually, where there's an AI that's making or substantially making a consequential decision, such as whether to give a loan, whether to treat someone, whether to hire someone, those types of decisions have been under various laws identified as higher risk. The way to eliminate the impact of the AI is to have the human in the loop so they're making the decision. If there's a way to focus the contract and if that's what the contract's about, then that's a place where you might want to have stronger language around ensuring that there's going to be some human involvement to argue later on that those laws didn't apply because of the way that the decision was being made. 

I'd also add, you mentioned this before, which made a lot of sense. It's all part of the same thing: in certain cases, depending on where your company sits in the supply chain, you may have a customer that put some terms in place. You negotiated a contract, and that customer wanted you to do certain things and have certain protections for the data you handle and the services you provide. Now you're turning around and subcontracting some of that, or you have a vendor that's helping you deliver those services upstream. You may be constrained by the terms of the agreement that you have with your customer. That could include human-in-the-loop requirements or other things where they are a little more skittish about AI and want to make sure the work they want you to perform is done in a certain way. That's going to flow downstream as well. For that issue and others, you might have to really push back on the vendor because of obligations that you have by contract, even if it's not mandated by law. That's one area where I'm seeing that.

From that perspective, Damon, moving the ball a little bit, how are you seeing a big area that we're seeing in these contracts is just allocation of liability, right? Talk a little bit about that and what are the types of provisions that you're seeing focused on. They're the same, the usual suspects, but how is AI changing that analysis?

Silver

A big area of negotiation I've seen in this context is that, by default, particularly if it's not just a vendor using AI, but a vendor that has developed an AI product. They probably pulled this from looking at some of the big AI players in the space, and a lot of times their agreements are non-negotiable, and they will say we'll indemnify you for IP infringement, and that is it. A lot of times that is the default position. Obviously, that leaves huge areas where there is no indemnification, one big one being data breaches, any types of claims related to the AI having inaccuracies or biases. There are a lot of different claims that could arise that are outside of the IP infringement context. You definitely want to, if you have the leverage, negotiate much broader indemnification than for just the IP infringement claim.

I'm also seeing a lot of language basically saying that our AI product has defects and you're assuming it as is. It might make mistakes. It's not a replacement for getting professional advice in all these various different areas. Basically, you, the customer, the user of the product, are assuming responsibility for those defects. Again, that is a place where if you have the negotiating leverage, you want to push back. That might circle back to the issue of human in the loop, where you're going to use your AI product to provide some aspect of the service. Are you going to agree to some oversight of the tool, some auditing of the tool? Maybe we'll get into audit rights later, but what kind of visibility are you going to give us? If we're ultimately responsible for the tool not functioning properly, what tools do we have available to make sure that we're able to vet what's going on and monitor what's going on? Do you see anything else from a risk allocation perspective?

Lazzarotti

On that, it's the Colorado statute, if I'm not mistaken. They actually address that, at least in part, by saying there are two things that are important that I can remember about that in connection with AI. One is that the parties are really going to share liability based upon their fault. The statute sets that up as the way to allocate liability in contracts involving certain types of AI. I'm being very general here, because there are several provisions in the law. 

The second thing is, and this is a little lobbying by the developers, saying we're not going to be liable if you don't use the tool in the manner in which you contracted for it, or it was advertised. That's really important. We're talking about contracts, but I know several other episodes we talked about really understanding the tool and thinking about if you're going to deploy this and you feel like your legal team, patting themselves on the back, negotiated this great contract, you turn it over to the people who are going to actually use it, and all of a sudden they go off the reservation and start using these tools for all kinds of purposes that are adjacent to what the intent was and all of a sudden you don't have the protections. At least in Colorado, you have protections if you're using it for something other than what the purpose was, what was advertised and so on. It's really important to not just negotiate these terms, but to operationalize them in terms of how you use them, because you may think you have protection by the negotiated terms, but those terms may not be enforceable in certain jurisdictions. I would want to make sure that the team understands what the limitations are and you build that into your governance structure around that so you avoid that issue.

Silver

On that point, Joe, I've seen several occasions where the vendors proposed terms that say that we only anticipate that you're going to use this very limited set of data to prompt our tool. Any additional data you use to prompt the tool, particularly any sensitive information, is beyond what we would ever imagine you would do with our tool. If They sometimes they explicitly say that you know you're assuming the risk, but even if they don't, that that would be the argument they would make is that you know we had no reason to believe you would ever prompt this tool with you know information about somebody's you know religious beliefs or with their social security number that was a shock to us and we are not responsible for the data breach that it ensued. That's another place to definitely focus attention: how the tool is going to be used, but also what data is going to be used to prompt the tool. I've pushed back successfully on efforts by the vendor to defer to us and say whatever data you plan to put in, that's what you plan to put in, but we have no visibility or control over that. Instead, in the DPA, I've tried to spell out what data is anticipated after conferring with the client about anticipated use cases, so that from a contractual perspective, there is more alignment around the data processing and so that the counterparty doesn't have that argument that they were surprised by the data that their tool ended up processing.

Lazzarotti

You mentioned audit rights, and that's an interesting one, because it may get to some of this in terms of how many businesses actually trigger the audit rights that they have. They spend a lot of time negotiating these rights, but are people really using them? Some are. I know that there are some required provisions, or at least maybe not specific audit right provisions, but the CCPA and service provider agreements include a right that the business may have to evaluate what the service provider is doing. 

Talk a little bit about that, Damon, because that's a great point about the data. Then how do you go back after the contract and set it up so that you can, and what does that process look like? What are some of the things you think about when you want to talk about an audit rights provision?

Silver

It's a good point. I have seen more exercising of audit rights in the context of data privacy and security and now recently with AI than I had in the past. There have been some situations where there was a vendor breach, and one of the things that the owner of the data got dinged for is the fact that they never exercised their audit rights. The CCPA does say that if you don't exercise your audit rights, you lose the inference that your vendor was complying with their contractual obligations. There is more incentive to exercise these rights.

What the exact audit looks like is going to vary depending on the risks posed by the vendor and how much you can understand based on policies and procedures and reports by third parties. There was a scandal recently that Joe and I wrote about where it turned out that one of the providers of SOC 2 audit reports is basically just sending the same report to everyone, and no one was noticing it. I certainly wouldn't recommend just doing that, just getting the third-party report. I have seen a lot of instances where that is the default negotiation position of the vendor: sure, we'll give you our then-current SOC 2. That can be part of what you have under your audit rights, but you also want the ability to ask for materials that show that the vendor has an AI governance program in place. You want to be able to see what they are doing in terms of their subcontractor agreements. You want to, at least in some instances, have the ability to do analysis and testing of the tool. Again, the circumstances are going to dictate both what you are able to negotiate in terms of the rights you have, and then also what you choose to do in terms of exercising those rights.  It's important to have this be part of your regular program. Most of our clients at this point have a program for onboarding and vetting the vendor; they have a program for contracting with the vendor. They also need a program for exercising audit rights. At the very least, if you are on a regular basis, say once a year, looking at the SOC 2 report and really looking at it critically, asking for some other supplemental materials, and documenting that analysis that's going to put you ahead of most other organizations and in the event there is some type of incident happening on your vendor side, you're going to be in a much better position than you otherwise would have been to show that you had reasonable safeguards in place and this happened anyway.

Joe Lazzarotti

There's certainly a lot to think about now with AI and procurement and the actual negotiation and then what comes after that. 

Thank you for listening in. If you have any questions or suggestions for future topics, please reach out to us. We'd love to hear it at privacy@jacksonlewis.com.