#82 - Confidence sabotages your fraud investigation

Last Saturday I walked through how I run investigations: instead of collecting evidence, I try to disprove a theory. 

I first make a hypothesis and then ask specific questions that would try to knock it down. All in the first 60 seconds.

Sounds good in theory, right? But I bet a lot of readers remained skeptical.

It’s the kind of hand-wavey advice you’ll get without the practical means to actually implement it correctly.

So let’s run through a hypothetical case.

The setup

You open a new case in your ATO queue. The triggering event: an established account with a login from a new device, followed immediately by a primary email change.

Your fraud theory writes itself: someone got hold of the account credentials, logged in from their own device, and swapped the email to lock out the real owner.

Classic ATO opening move.

But you also know your queue runs at about 15% fraud rate. So instead of building a fraud case you start trying to kill one.

Side note: I’ve previously touched on the methodology behind ATO fraud detection in case you need a primer. The basic idea, though, is that we’re looking for (in)consistencies between the flagged session and the previous ones.

Step 1: Can I kill this quickly?

The first pass is always a fast one. You’re looking for something that makes the fraud story obviously false.

But this is obviously contextual, both to the use case (ATO) as well as to the actual suspicious event we’re investigating (new device + email).

In our case, let’s look at two examples for how this can look like.

The first is behavioral consistency: does the user still look like themselves? 

A quick check of the login IP can reveal it’s in the same subnet as three previous sessions over the past few months.

Alternatively, we might see that the user is behaving consistently across sessions, for example by checking their account history with a particular filter as the first thing they do after login. 

Another example is to prove the new assets on the account actually belong to the user. 

Say the new email matches their current company on their LinkedIn profile.

Or the new device shows up in a separate account with the same last name and home zip. Almost certainly a family member’s device.

Either of those is enough - case closed in under two minutes.

And more importantly, correctly closed.

Before moving on, I ask one question: could any evidence I find now convince me this is fraud, given what I’ve already seen?

But if the device belongs to the user’s spouse - what would fraud even look like? I can’t construct a reasonable question for that.

That’s the finishing condition. If I can’t think of what would flip my decision, I stop.

Step 2: Is this actually fraud?

Let’s say neither angle landed. No behavioral consistency signal and no obvious asset link. Before going deeper, I do one quick pass in the other direction: is there something blatantly incriminating?

I’m looking for hard fraud links.

Is the device fingerprint tied to a previous confirmed case? Is the new email on a block list or associated with a mule account? Is there already a large transfer pending to a new payee the user has never transacted with before?

But even here, the model holds. If the device is linked to a prior case, I ask: is this a shared device? A merchant terminal? A machine at an airport or internet cafe that appears in fraud cases simply because anyone can use it? 

A positive answer would likely change my decision, so these are worth asking and investigating.

In practice, steps 1 and 2 often blur together. While you’re scanning the account to understand what is the likely fraud story, you’re likely to notice these glaring red flags.

Side note: If you repeatedly find “easy to prove” fraud cases in your queue, that’s an issue. These should be resolved automatically to focus your investigations on true gray cases.

Step 3: Going deeper

Ok, but say there’s no quick exoneration and no obvious link to fraud. This is where the real investigation actually begins.

The question is still the same one: what would convince me this isn’t fraud?

One place I look is post-flag behavior. What has the user done since the event? 

If they paid $5 for a low-risk subscription service after the email change, that’s interesting. Fraudsters move fast toward high-value actions. A $5 charge doesn’t fit the profile. 

But I’m also looking at what they haven’t done. No new payees added. No address changes beyond the email. No large transfers queued. The absence of typical fraud follow-through is its own form of evidence.

Another angle can focus on whether this matches a fraud campaign I recognize? 

ATO attacks tend to cluster around the same behavior and infrastructure: consistent device patterns, email domains, velocity signatures, or IP networks.

If nothing here looks familiar, I ask: what are the odds this is the very first case of a new attack vector? It’s always a possibility, but would I bet money on it?

Step 4: When you run out of questions

This is where most people slip up.

At every point in this investigation, I need a clear question in front of me - one that, if answered a certain way, would flip my current ruling. Not a question that confirms what I already think. A question that would change it.

After the $5 payment, my working theory is that this is legitimate. What would change that? One scenario: it was a test transaction - a fraudster probing defenses before a bigger move. 

My next question then is specific: does this payment look like a probe? Is there a larger transfer pending? Is the payee linked to anything suspicious?

That’s my next move.

The question I never ask: “what would make me more confident?” That one has no finish line. There is always more evidence. 

That’s exactly how a four-minute case turns into a twenty-minute exercise.

Because investigations don’t end when you have 100% confidence in your decision. They end when you cannot think of a question that can change your decision.

That is the key.

A principle, not a full playbook

How many objections have you collected to yourself while reading this? Hopefully more than zero.

There’s obviously a lot of nuance here - your risk appetite, your queue’s fraud rate, your investigation capacity, and your team standardization.

Specifically regarding the last point, you obviously want to standardize the possible investigation paths by use case and the specific attack vectors that impact your business.

There’s still a lot we can cover here, so if you have specific questions or objections - hit the reply button and let me know. I might return with a 3rd part.

In the meantime, that’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Free Discovery Call - Unsure where to start or have a specific need? Schedule a 15-min call with me to assess if and how I can be of value.
​Schedule a Discovery Call Now »​

Consultation Call - Need expert advice on fraud? Meet with me for a 1-hour consultation call to gain the clarity you need. Guaranteed.
​Book a Consultation Call Now »​

Fraud Strategy Action Plan - Is your Fintech struggling with balancing fraud prevention and growth? Are you thinking about adding new fraud vendors or even offering your own fraud product? Sign up for this 2-week program to get your tailored, high-ROI fraud strategy action plan so that you know exactly what to do next. ​
Sign-up Now »

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!

<