#100 - Fraud teams are the worst AI builders

"80% of the game is to have that platform and structure in place."

When I sat down to talk with Lalitha Rao, Chief Risk Officer at Imprint and a Stripe, Square and Capital One alum, we mainly talked about AI agents.

Lalitha and her team are really pushing the frontier when it comes to running agentic AI in production, at scale, within a risk team.

They already hit one of the most impressive milestones I could think about - suggesting features and model retrains. That’s literally the end of the AI transformation journey as I laid it out myself.

But the most interesting part for me was when we talked about what came before them.

Because right now, every risk team I speak with wants to build its own agents.

But for most teams without engineering backing, that's a trap. 

Let's talk about why.

Why everyone jumps in

It all looks so easy.

You open a chat window, describe what you want, and the first demo works in an afternoon. It pulls the data, drafts the summary, and even formats the report the way your manager likes it.

Best case, you were curious and wanted to play around with AI. It’s fun, and easy, and you suddenly feel like god. Or more accurately, like an engineer. And that’s addictive.

Worst case, you were told to do it by your manager. Whether it’s about spending your token quota or just showing the team is “AI-first”, there’s a lot of downward pressure to just do something with AI.

Anything.

But regardless of the reason, the result is the same: teams dive in head first, before they plan what they actually want to achieve.

No goal, no owner, no definition of done - just a lot of enthusiasm and API keys. And the result is a bunch of personal pet projects.

An analyst builds an agent for their own queue, another builds one for weekly reporting, a third builds something nobody else ever sees.

I’ve seen plenty of automation projects that took a full week of work over a month, just to automate an hour of daily work.

And before you get your time back it’s broken or abandoned.

No joke - from what I see across the teams I work with, about 90% of these die within a month.

Nobody decided to kill them. The analyst got busy, the data source changed, or a downstream process changed and broke it.

And the 10% that survive?

Those are the ones that should worry you, because now something nobody planned for is running in production.

Agents are code

Building agents is software engineering, whether you call it that or not. The moment an agent touches production data or feeds a decision, it inherits every responsibility any other system in your stack carries.

Prompts, tools, data connections, credentials - it all adds up to a piece of software running inside your risk stack and making or shaping decisions.

Engineering teams have spent decades building the DNA to manage that kind of thing. Release trains, branch management, code reviews, version control, on-call rotations.

But fraud fighters think differently. We think in cases, rules, and patterns, which is exactly what makes us good at fighting fraud. It just isn’t how you write software.

And this never explodes in the demo.

It creeps up on you.

It shows up three months later, when the agent is part of a workflow and someone asks who is responsible for it.

Why isn’t it covering this edge case, or that tool, or - God forbid - why are there clear PANs in its folder?

Here's the thing:

Owning software comes with three obligations most risk teams don't see coming:

Maintenance: Data drifts, prompts drift, and the tools underneath change, so an agent without an owner degrades a little every week until someone notices the damage.

Security: Every agent holds credentials and data access, and someone has to govern who gave it what, and why.

Serviceability: When it breaks during the holidays and the owner is on a 2-week vacation, who you gonna call?

I wrote about the AI hangover a while back. Unowned agents are one of the fastest ways to get there.

Before you build, check

So before you let your team loose on agents, pause.

What's likely missing is a mindset shift - from a risk mindset to an engineering mindset.

A risk mindset asks whether the agent catches fraud. An engineering mindset asks who owns it, who approved it, and who fixes it when it breaks.

The good news - someone in your company already has that mindset. Your engineering team, and to a degree your data science teams as well, have run this playbook for years.

So borrow theirs.

Sit down with them and run through four questions:

Goal: What outcome is this agent for, and how will you measure it? If the answer is "we'll see," you don't have a goal yet.

Scaffolding: Does someone give your analysts tested building blocks - shared tools, standard connections, known gotchas - so your toolkit is supported and working?

Data: Does every table and metric have a canonical definition and an owner? An agent that can't tell what "declined" means will give you a confident wrong answer.

Review: Does someone other than the author approve before anything hits production?

And that last one is the easiest to skip.

At Imprint, Lalitha told me, risk team members build the agents themselves, on a harness from their central AI team.

But every change still gets a code review from an independent data scientist before it reaches production.

That single step is the engineering mindset in practice. It's also the first thing to disappear when a deadline gets close.

Missing any of the four?

Then you have two honest options. Buy, and let a vendor carry the engineering burden, or bring your engineers in to fix the platform first.

Anything else is how you end up stuck.

The bottom line

Building an agent is easy. Owning one is an entirely different commitment.

Draw clear goals, check for the scaffolding first, and you build only what's worth maintaining.

Lalitha put it plainly when she described her own team's platform: without it, the whole effort "would have become so much more ad hoc project based and not as impactful."

If you're about to greenlight an AI agent program, listen to the full episode before you do. So many gold nuggets like this one.

Are you running agents at scale and in production? Hit reply and tell me - I'd love to hear how other teams have approached it.

In the meantime, that’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Free Discovery Call - Unsure where to start or have a specific need? Schedule a 15-min call with me to assess if and how I can be of value.
​Schedule a Discovery Call Now »​

Consultation Call - Need expert advice on fraud? Meet with me for a 1-hour consultation call to gain the clarity you need. Guaranteed.
​Book a Consultation Call Now »​

Fraud Strategy Action Plan - Is your Fintech struggling with balancing fraud prevention and growth? Are you thinking about adding new fraud vendors or even offering your own fraud product? Sign up for this 2-week program to get your tailored, high-ROI fraud strategy action plan so that you know exactly what to do next. ​
Sign-up Now »

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!

<