#97 - Did foundation models just end fraud experts?

Last week we shared at Sardine our new foundation model for issuing fraud.

The results, even if not game-changing, are impressive. As I already hinted at, while LLMs still seem to be lagging behind, foundation models trained on fraud do show a clear advantage.

But I don’t want to rehash that, you can go ahead and read the blog or the white paper we published about it, you’ll find all the juicy details there.

What I do want to address this week is a thought that came to me during this research work:

Are foundation models spelling the end of fraud domain experts?

What do foundation models do?

Here’s a pattern I had to break in my own mind, and one that took some time - foundation models are not scoring events for probability of fraud.

It sounds straightforward when I write it like that, but it took me quite some time to separate the words “model” and “scoring”.

In hindsight, I’m not sure why. I already saw and built models that produced features: assessing user nationality, rating emails for bogusness, and more.

And yet, I had to see it to understand it.

Sardine’s model, for instance, created one embedding per transaction, but that embedding had 192 dimensions. Or in other words - a single model created 192 new features.

And these 192 features were fed to our “regular” XGBoost scoring model alongside the 77 existing features for issuing fraud.

And that was how we managed to increase recall (fraud caught) by 24% to 35%. Better and richer input - better results.

What does that have to do with domain experts?

What do foundation models actually do?

These 192 embedding dimensions weren’t random. We trained the model to look specifically at historical spend behavior, and so the features captured exactly that.

How likely is it that this new transaction is fraudulent, judged by looking at it in the context of past behavior, trained across customers.

But you have to ask yourself - can this not be represented by “hand-crafted” features? Why do we need a model to calculate this?

But here’s the thing: it’s not that simple.

I mean, yeah, you can build features like “average spend last 30d” or “average payment amount” without any issue. In fact, these were already included in our baseline issuing model.

But these features don’t really do a lot in terms of predictive power. You need much more nuance than that. 

Let me give you an example:

If my customer just paid for a gift card, that can be well within their normal spending averages. And it also might be that they bought gift cards in the past.

But this late at night? From a merchant they never bought from before? And then trying to buy another gift card again?

Sure, you can build hand-crafted features to answer all these questions. But it takes time and effort. Meaning, it needs a budget.

And more than budget - it requires a particular skill set.

The cutting edge

So far, it was pretty easy for me to guess which fintechs or fraud vendors are going far:

To be a successful, you need to be excellent in risk management.

To be excellent in risk management, you need to have excellent AI/ML.

To have excellent AI/ML, you need excellent data (note: not a lot of data - excellent data).

To have excellent data, you need excellent data features, built on top of the raw data you collect.

And to have excellent features, you need fraud domain experts.

But not any kind of domain experts. The kind that encompasses two traits: deep domain knowledge and technical chops.

And it’s hard to find these two coexisting at the same time.

Side note: To be clear, I’m not saying that good fraud feature engineering will make any fintech successful. It is the foresight to invest in that that tells me the founders know what they’re doing.

If I saw a team with this kind of talent, I knew with high certainty that they’d beat the competition. Because most organizations simply don’t have it.

So here’s the question that’s on my mind: are foundation models about to change that?

Because suddenly you don’t need this expertise anymore. You simply train a foundation model on your data, and you have all the features you want.

Did AI just kill the Fraud Data Scientist role?

The likely answer is “no”.

First, because not all teams have enough data to train foundation models on. Or at least, not for the problems they want to solve.

Second, because hand-crafted features are still extremely important to have - whether for explainability or to be used in rules.

Side note: Yeah, we still write rules today. Are you not? You’re leaving money on the table.

Third, because foundation models proved effective in describing sequential data. But we haven’t seen them doing the same on fraud problems that don’t involve accounts.

And fourth, and probably the most important one, because I don’t buy that BS about AI taking jobs anymore. It changes jobs.

But I do think that the field is going to open up and allow smaller teams, with less budget and access to talent, to punch above their weight.

I’m very curious to see how that would go.

Are you looking into foundation models for fraud prevention? I’m looking to connect with more teams and compare notes.

In the meantime, that’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Free Discovery Call - Unsure where to start or have a specific need? Schedule a 15-min call with me to assess if and how I can be of value.
​Schedule a Discovery Call Now »​

Consultation Call - Need expert advice on fraud? Meet with me for a 1-hour consultation call to gain the clarity you need. Guaranteed.
​Book a Consultation Call Now »​

Fraud Strategy Action Plan - Is your Fintech struggling with balancing fraud prevention and growth? Are you thinking about adding new fraud vendors or even offering your own fraud product? Sign up for this 2-week program to get your tailored, high-ROI fraud strategy action plan so that you know exactly what to do next. ​
Sign-up Now »

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!

<