Firms are asking the wrong questions about AI. Here are the right ones.

Warren knowles

Every firm has bought AI tools. When you ask what those tools are doing, you get blank answers. No “Adoption is strong here,” or “This changed how we handle X.” Just a vague sense that they had to have it, for reasons of FOMO, leadership mandates, or vendor pressure. But what they’ve largely got is shelfware. The spend is real, but the impact isn’t visible.

SUSAN COBB

And the cost is starting to catch up. Firms are moving from “More AI!” to harder questions: “Which of these tools is actually worth it? What did it change? Who’s accountable for that answer?” That’s a healthy shift. But a lot of firms don’t yet have the framework to answer it.

Warren knowles

We borrow a question from how professional firms think about taking on new business: “Can we do this, and should we do this?” For firms, the “can” is often a compliance question — conflicts, independence, client commitments. For technology decisions, the answer to “Can we?” is almost always “Yes,” given enough time and budget. So the real discipline is “should.” Should this AI be doing this work? What does success look like, and how will we measure it?

SUSAN COBB

The firms that are moving past the shelfware phase are the ones asking “should” before they buy. They’re not evaluating features — they’re evaluating outcomes. What process changes? What gets faster? What gets better? If you can’t answer those questions before you deploy, you’re not going to be able to answer them after.

Warren knowles

At the individual level. A lot of AI spend right now is driven by individuals — someone builds a little agent, someone subscribes to a tool, someone has a workflow they’ve automated. That’s fine at the margins. The problem is when you aggregate it. The firm is now spending significant money on AI but struggling to understand the value of that investment. That’s not an AI strategy. That’s FOMO scaled up.

SUSAN COBB

There’s also a gap between what AI does for the individual and what it does for the firm. An associate who’s twice as fast at document review is great. But if the conflicts bottleneck is still there, client intelligence is still siloed, and leadership still can’t see the full picture — the firm isn’t operating more effectively. Individual productivity and firm productivity are not the same thing.

Warren knowles

The same way they think about any other operating resource. If you hired a business services team member, you’d have expectations: “Here’s what this role produces, here’s how we measure it, here’s what success looks like.” AI agents deserve the same scrutiny. “What did you do? What did it cost? What changed for the better because of you?” Firms that start holding their AI accountable like that are going to get much better at knowing where to invest and where to stop.

SUSAN COBB

And frankly, the AI tools that survive that scrutiny are going to be the ones built for firm-level outcomes — not individual convenience. The question isn’t just “Does this make my day easier?” It’s “Does this make the firm better?” Those are very different bars, and right now most AI tools are only being held to the first one.

Warren knowles

They’ve named what they’re trying to do. Not “We’re adopting AI” — something more specific. We’re using AI to compress the time from business development to conflict clearance. We’re using AI to make sure every relationship insight lives in the firm, not just in one partner’s head. We’re using AI to run our intake process at the quality of our best people, not our average ones. When you can say that, you can measure it. And when you can measure it, the “should” question becomes answerable.

SUSAN COBB

The firms I’ve seen nodding hardest at this conversation are the ones who already felt it — they just didn’t have language for it. Once you distinguish between AI for the practice and AI for the business of the firm, it’s hard to unsee. That distinction is where the real strategic question lives.