#43 - False positives cost you less than you think

How much are False Positives really costing your business?

The real number is much smaller than you’d think.

Good news? Bad news? Depends on who you ask.

A few weeks ago I published a Linkedin post that talked about the challenges CROs are facing when they need to defend their team’s performance in front of their board.

Ivan Stefanov, CEO of Noto360, added this thoughtful comment below it:

Ivan raised two important points that I think deserve some deep dive. More importantly, it served as a good reminder.

So let’s take a closer look.

Buyers behave differently than fraudsters

Shocking, I know.

And yet, when we take this simple truth and implement it in our daily work, we tend to think about behavioral patterns and how to separate them accurately.

What we often forget is to apply the same behavioral thinking to our high-level aggregated metrics.

Here’s what I mean:

Let’s say someone is trying to purchase a new iPhone in my shop and I decline them.

If they are a fraudster, they are likely to try again. Sometimes multiple times.

If they are a buyer, they are also likely to try again. Sometimes multiple times.

But what if I approved that transaction?

If they are a fraudster, they are most certainly going to try and transact again. Why wouldn’t they?

But if they are a buyer? It’s highly unlikely to see them place another order, especially an identical one.

So while we expect fraudsters and buyers to behave the same when they are blocked, we expect them to behave differently when they succeed.

Now let’s see how this would look like at scale:

In the table above we have a simple scenario that follows the above logic:

Declined buyers will attempt to transact 3 times, while approved buyers will attempt only once.

Declined fraudsters will attempt to transact 3 times while approved fraudsters will also continue to place purchases.

Side note: All the numbers above are just placeholders for the sake of example.

When considering the impact on the bottom line (profits), we see two things:

Firstly, the impact of single buyers is smaller than that of fraudsters due to the profit margin considerations. This makes sense and explains why most businesses are OK with false positives.

Secondly, while this difference is “only” 10x when we consider a single payment, when dealing with multiple attempts it rapidly grows. In this example, with only 3 attempts, it’s already at 30x.

And the reason is exactly what we’ve mentioned above–successful buyers will only purchase once.

Now let’s narrow down our view to focus only on the declined attempts in the example above:

While for the business it may look like we declined $300 in both cases, we actually see that not only the margin, but also the revenue lost is much smaller than we’d think.

All of this is to say that false positives are less impactful than how they seem at first glance.

Don’t get me wrong, I’m not saying they are to be ignored, just that we need to calculate their impact with the above in mind.

Fraud declines are not the only source of revenue loss

There is a case for claiming that some of the transactions we decline will not be completed anyway. Either because they were declined by a different actor (vendor, issuer, etc.) or because of user abandonment.

You could also claim that even successful payments can later result in refunds, non-fraud chargebacks, and cancellations.

Having said that, in most cases the revenue impact of this would be rather small, especially compared to the previous factor we’ve discussed.

Calculating the true cost of False Positives

So how do we determine the real impact of False Positives?

Supposedly the answer should be simple: deduplicate the transactions of unique legitimate buyers, and only count them once.

The challenge, as always, is figuring out which declined transactions are indeed False Positives and which are fraud attempts.

Fortunately, I already covered some ​techniques of identifying FPs​ in a previous TSFS issue.

Another method we can use that I haven’t mentioned in that article ties directly to what we’ve discussed today: looking at how many declined attempts belong to users that actually managed to complete a transaction.

If it was the last attempt from that user, we’re likely dealing with a legitimate buyer.

And if the user completed a transaction and continued making additional attempts? Odds are we’re looking at a fraudster.

OK, say we identified the false positives in our declined population, how do we know how to measure their true impact?

Yes, we can deduplicate the events, but can’t legitimate buyers be repeat buyers?

Yes, they can, so we usually deduplicate on customer_id+date, so we count all transactions that occurred on the same day from the same user as only one. Specifically, the first one.

And if they transacted again the day after? That would be counted as a separate transaction.

Side note: Instead of customer_id we can use any unique identifier we have for the buyer–account_id, email, SSN, etc.

Is this method 100% accurate? No.

We will likely miss False Positives, count some fraud as legitimate buyers, have some deduplication errors, etc.

But it will be much closer to the real number than just taking the numbers at face value.

Good news, bad news

I mentioned at the beginning that the implications of finding out the real impact of False Positives can be situational. Here’s why:

Are we worried we might be impacting the business’ top line? Good news, it’s probably lower than what everyone thinks it is.

Are we working on lowering the risk friction so we can grow more rapidly? Bad news, the impact might be smaller than what we expect.

Is a new fraud vendor promising +$X by increasing our approval rates? We actually need to verify they used the right methodology to make that claim.

But regardless–more accurate numbers build better predictions.

And better predictions build trust.

Have you seen this discrepancy playing out in your role? I would love to hear more such stories! Hit reply and share what you’ve seen and how you tackled it.

In the meantime, that’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Free Discovery Call - Unsure where to start or have a specific need? Schedule a 15-min call with me to assess if and how I can be of value.
​Schedule a Discovery Call Now »​

Consultation Call - Need expert advice on fraud? Meet with me for a 1-hour consultation call to gain the clarity you need. Guaranteed.
​Book a Consultation Call Now »​

Fraud Strategy Action Plan - Is your Fintech struggling with balancing fraud prevention and growth? Are you thinking about adding new fraud vendors or even offering your own fraud product? Sign up for this 2-week program to get your tailored, high-ROI fraud strategy action plan so that you know exactly what to do next. ​
Sign-up Now »

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!

<