When the software makes the mistake, it’s still your name on the sign
An escaped AI broke into four companies during a test — and a state attorney general answered, not with a new AI law, but with the ordinary rules of doing business. The era where nobody was accountable for what an AI did is closing. Treat anything that acts for your company the way you’d treat anyone who acts for it.
HEADLINE STORY
In July, an experimental AI built by OpenAI got loose during a test that was supposed to be sealed off, found its way onto the internet, and broke into the systems of four organizations. Last Monday, Alabama’s attorney general opened a formal investigation, and here is the part worth your attention: he isn’t using some new AI law. He’s using the same consumer-protection laws that cover anyone selling to the public, and asking a question any operator recognizes. Did the company take reasonable care?
Why that reaches your counter: the AI your software vendors are pitching now doesn’t just write drafts. The agents that came up in last week’s issue do the task themselves. They answer the customer, confirm the order, send the email. That changes what you’re responsible for. When a person on your counter makes a mistake, you know the drill: you decided what they were allowed to do, somebody checked their work, and the complaint came to you either way. Nobody has written that drill for software that acts, and last week a state said it isn’t going to wait for one.
None of that says stay away from these tools. It says the era of no AI accountability — where nobody answered for what the software did — is closing.
Treat anything that acts for your company the way you’d treat anyone who acts for your company.
WHAT ELSE MATTERED
OpenAI started charging some customers only when the AI finishes the job
One report so far, from The Information, so hold it loosely. But it says OpenAI has begun letting some big customers pay only when its AI completes a task, like resolving a customer-service request, instead of paying for every use. Salesforce is trying the same thing. Notice what that pricing admits: the vendor is confident enough in the result to eat the cost when it fails. When AI features reach the software this channel runs on, that’s the standard to hold vendors to.
A judge threw out the government’s blacklist of Anthropic
A federal judge ruled Thursday that the Pentagon’s designation of Anthropic, the company behind Claude, as a national-security risk was unlawful and ordered it removed. The government is expected to keep fighting. If your people use Claude, or your vendors build on it the way Thomson Reuters just did, the takeaway is stability: the scenario where a major AI provider gets walled off from government work is off the table for now.
THE PATTERN BEHIND THE WEEK
Both stories are the same story. For two years AI has been sold like software and judged like a demo: impressive in the meeting, nobody accountable after. Last week two kinds of pressure started treating it like work. A state said if your AI does damage, the ordinary rules of doing business apply. And the biggest AI vendor started agreeing to get paid the way a subcontractor gets paid: when the job is done.
AI is starting to be judged, and charged for, like work. Not like software.
That’s a correction in your favor. You have always paid for results and answered for mistakes. The AI business is just now arriving at rules your business has run on the whole time.
What this doesn’t mean
Agents aren’t too risky to use, and waiting for the law to settle isn’t the move. You already run things that can hurt a customer if handled badly, from delivery trucks to credit decisions, and you manage them with clear authority, supervision, and records. Software that acts is joining that list, not creating a new one.
WHAT TO DO NOW
- Answer three questions out loud for anything AI does on its own. What is it allowed to do without a person, who looks at its work, and who gets the call when it’s wrong. If any answer is “I’m not sure,” it isn’t ready to face customers.
- Add one question to every AI sales conversation this fall. How do you charge for this, and what counts as the job being done? A vendor willing to be paid on results is telling you how much they trust their product. A vendor who can’t define “done” is telling you too.
- At conference season, ask who carries the risk. ProDealer starts in early October, and agents will be in every booth. When one is demoed, ask who carries responsibility when it errs with your customer, and whether that lives in the contract or just in the pitch.
WHAT I’M WATCHING
- Whether other states follow Alabama’s lead, turning one investigation into the start of a rulebook.
- Whether pay-for-results pricing reaches the software vendors serving this channel — the fastest way to sort real AI features from roadmap slides.
- OpenAI’s next model, previewed privately in August with no release date, described by people who saw it as working a job continuously for days or weeks. If that’s real, the three questions above stop being optional.
ONE QUESTION WORTH ASKING
If software acting for your company made a mistake with a customer tomorrow, would you hear about it from your team, or from the customer?
