High-Risk AI Contract Terms: What Vendors and Customers Should Redline

Sort the use case before you evaluate the tool. The threshold question is what the system decides or shapes, not how capable the technology looks in a demo. Regulators on both sides of the Atlantic converge on uses that control or materially influence someone’s safety, rights, or opportunities. The EU spells its categories out in an annex while US states focus on the decision itself, and the practical answer for us runs the same either way. Get the use case sorted and the rest of the analysis follows from it.

Ask what happens to the business when the tool gets it wrong. Kim put business impact alongside regulatory risk as a separate test, and it catches what the regulatory buckets miss. A tool can look unremarkable on its face and still sit at the center of what the company does. She described a generic compliance tool that returned wrong information on pharmaceutical manufacturing registration, and nobody inside the company knew enough to catch it. Everyone in that vendor review saw a compliance tool and liked it, and no one asked what failure looked like.

Know whether you are the provider or the deployer. The obligations split along that line and most of us sit on the deployer side. Providers build the system and put it on the market, so oversight for them is a design property baked into the product. Deployers get a procedural right to review and check. You can also be both at once, which happens the moment you resell something you wrapped or fine-tuned, so settle your position BEFORE you start marking up oversight language.

Make human oversight meaningful enough to survive a real look. The failure mode these rules are written against is rubber stamping, or in Laura Belmont’s words, “Check, next, check, next.” Meaningful oversight needs a reviewer who is trained, who holds authority to actually decide, who can see the evidence, and who has the time to look. It also needs a record showing the review happened. Oversight nobody documented reads the same as no oversight when someone asks you to prove it.

Watch for warranties dressed up as product statements. One sentence in the sample oversight clause promised that the AI features provided oversight functionality sufficient to meet applicable AI laws including the EU AI Act. That single sentence conceded jurisdiction the vendor might not be under and swept in requirements that land on the buyer. Name the specific acts you perform instead of promising sufficiency against an entire regime. Vendors can give that language easily, because they are already doing those things.

Warrant how you built it rather than that you complied. AI is probabilistic and not deterministic, so an outcome warranty asks somebody to promise what nobody controls. What a vendor can stand behind is the design, meaning the system was built in accordance with the laws and documentation as they exist today. Laura Frederick landed in the same place, favoring language that says we built this to do one, two, three, four, drawn from the act without being tied to it. Nobody knows yet how these statutes get interpreted, and a compliance warranty locks you into an answer that has not been written.

Match your notice timelines to the rules that already bind you. Undue delay gives you nothing to manage against, so replace it with the number the regulation uses. Laura Belmont pointed to seventy-two hours as the typical figure and warned against promising twenty-four when the rule says seventy-two. Carrying different timelines for different recipients means tracking who gets what notice during the worst week of your year. As she put it, “you don’t have to be a martyr and give more than that.”

Define your use case in the commercial documents. Scope creep is the predictable failure in AI deals, because vendors sell and business teams say yes. Writing the use case into the order form or statement of work gives you something to point at when the tool starts doing work nobody scoped. Kim called herself a fit for purpose contractor and wanted the agreement to show that a specific set of words covered a specific activity. Vague use restrictions sitting next to an SOW that describes the disclaimed use protect nobody.

Right-size the information you ask a vendor to hand over. Asking for everything feels safe when you are nervous about compliance, and it builds exposure of its own. Records showing you received information and never reviewed it are not a good fact later. Model weights nobody on your team can read will not make you more compliant. Work out what you actually need to discharge your duties, then ask for exactly that.

Decide how you will judge the tool before you sign. We tell the human reviewer to evaluate the output without ever telling them what good looks like, which turns oversight into a sniff test. Build your evaluation criteria while you are onboarding the tool and get whatever you can contractually alongside it. No vendor is going to hand you a testing framework. Bias testing sits in the same place, since a vendor answer you cannot interrogate still leaves accountability sitting with you.