This is an excerpt from FinAncIal: Helping Financial Executives Prepare for an Artificial World. Grab your copy here!
This case study covers the leading player in AI-based decline salvage for fraud prevention, FlexPay.
Stay tuned to the CU 2.0 blog while we feature more excerpts from our financial guide to AI!
“Wolf! There’s a wolf!”
Ring any bells?
How about, “The sky is falling! The sky is falling!”
Well, there are two things that the boy who cried wolf and Chicken Little have in common: first, both are trying to do their jobs despite being surrounded by danger. Second, both frustrate the heck out of people by identifying threats when there currently are none.
If you’ve ever dealt with false positives—card declines on legitimate purchases that you’ve made—then you can understand the townspeople’s ire in both stories. Few things are more frustrating than trying to pay and being unable to because Chicken Little—ahem, I mean the program in charge of detecting fraud—couldn’t tell the difference between your legitimate purchase and a bogus one. Worse, if you’re a retailer, a false decline on a card can cost you more than the sum of the transaction—it can cost you a customer.
The problem is that traditional methods of fraud detection are unable to keep up with modern consumer habits. Older methods of fraud detection are quick to cry wolf, much to the frustration of customers, retailers, financial institutions, and basically everyone involved. But there must be a solution, right?
Right.
FlexPay is an AI‑driven decline salvage program that uses machine learning to recover between 50–70% of declined transactions. FlexPay’s primary goal is to ensure that all transactions are accurately identified as either legitimate or fraudulent. In the end, it should mean smoother transactions, higher retailer revenue, and increased customer satisfaction. There’s a little something for everyone!
How Does FlexPay Work?
FlexPay uses machine learning models to analyze and understand financial transaction habits. As they accumulate increasing amounts of data, FlexPay is able to predict with a very high degree of accuracy whether a transaction is legitimate or fraudulent. But that’s not all. FlexPay also enhances transaction strategies to optimize Automated Clearing House interactions to reduce problems associated with ACH transactions.
FlexPay salvages false declines for vendors, recuperating the money they missed out on from failed transactions. To keep things fair, instead of charging flat or tiered rates for services, FlexPay takes a small portion of the decline salvage revenue. The more efficient FlexPay gets, the more revenue it generates. If they don’t help, they don’t get paid.
But how does FlexPay work? Primarily with machine learning. Their engine runs on the Azure platform, where it records and analyzes massive volumes of financial transaction data daily. From there, the models differentiate between real, intended transactions and false or fraudulent ones.
Similar services solve the same problem differently, though. As a result, similar services see lower real‑world efficacy—their fraud models can’t handle the breadth or depth of transactional intricacies that machine learning models can.
Why FlexPay Turned to Machine Learning
Traditionally, similar companies have used statistical modeling and sophisticated, rules‑based algorithms. In these models, people had to manually define which transactions to allow and which to decline. These algorithms relied on an “if this, then that” methodology.
FlexPay attempted to optimize strategies for various if/then scenarios at first. However, the amount of manual work involved to perfect if/then models is very high, and there’s a limit to their efficacy. Basically, they just aren’t as accurate as they should be, so they tend to fall short. Also, when FlexPay used if/then methods, some transactions fit under several “if” categories. Accordingly, their “then” outcomes were getting tangled, unpredictable, or sub‑optimal.
FlexPay updated their strategy to include a ranked order of if/then scenarios to counter the tangled “then” outcomes. For example, if an “if” scenario had three possible “then” outcomes, then the model would rank the “then” outcomes in order of desirability—the highest‑priority “then” outcome would be tried first, and subsequent “then” outcomes would occur only when the first “then” outcome didn’t work.
Still, FlexPay’s if/then methodologies were getting too complicated. Rules‑based algorithms were getting good results, but they were far from perfect. These basic algorithms also grouped transactions into different types, which were easier to write rules for, but lost a lot of nuance on the way.
For example, rules‑based algorithms often couldn’t differentiate between unique and recurring transactions. So, if someone went to Chipotle for lunch once or twice per week, a rules‑based algorithm would decide that the Chipotle purchases ticked all the boxes for a recurring transaction. But recurring transactions are a whole different story in the financial world. Just because something is frequent or matches a particular pattern, that doesn’t make it recurring. FlexPay wanted to identify actual recurring transactions, such as auto insurance, gym memberships, and Netflix subscriptions. While they were able to do that, they were also needlessly finding out that people were creatures of habit—and that some people eat entirely too much Chipotle. If FlexPay wanted to distinguish themselves from their peers, then they had to offer significantly better results.
FlexPay finally got around the problems with grouped transaction types by treating each transaction as its own individual unit instead of as transactions of a certain type. They incorporated individual habits into personal transactions, which required massive amounts of personal data, financial history, spending habits, qualities and characteristics of buyers, and so on.
To handle that data, FlexPay needed more powerful computers with better problem‑solving capabilities. They needed pattern‑matching and pattern‑mining capabilities to look for trends in billions of transaction records. The goal was to individually evaluate each transaction for validity. Machine learning was the only way to go from “good, but clunky,” to “great, and personalized.”
Growing Pains from Switching to Machine Learning
When moving from if/then algorithms, FlexPay experienced a few hiccups. Some of the assumptions in the rules‑based system were correlative rather than causative. One of the assumptions had to do with “friendly fraud.” Nearly 70% of fraudulent transactions are from customers who call their bank and say, “I didn’t order this shit!” In the industry, these people are called IDOTS. (No, you didn’t read that right, but yes, you got the right idea.)
Rules‑based algorithms incorporated IDOTS data, so FlexPay’s earliest machine learning models did as well. Consequently, FlexPay was essentially training their algorithms to recognize “friendly fraud” as actual fraud, which limited its effectiveness. Later, when IDOTS was accounted for, the sky stopped falling—FlexPay could filter for and predict similar activity. They went from a 33% rules‑based, if/then recovery rate to a 50–70% recovery rate by using machine learning. FlexPay’s machine learning model improved their recovery rate by around 35%.
If that doesn’t sound impressive, think about it this way: that’s roughly a 100% increase in efficiency over their previous rule‑based algorithms.
How Long Did It Take to Start Seeing Results?
To understand how long it took for FlexPay to start seeing results, it might first help to describe their methodology behind training machine learning models. (Cue the Rocky montage music.)
A lot of machine learning is pure trial and error. It starts with brute force learning, which basically means feeding data into the machine and letting it find patterns and connections. Those patterns and connections are the initial results that, in all honesty, rarely resemble their final product. With some initial patterns and connections in place, people can tweak, refine, reinforce, and improve their algorithms to get them closer to a desired final result.
But using only one machine learning model rarely works well. What if an adjustment to the model didn’t turn out right? Wouldn’t two models be twice as fast as one? What if you wanted to try five different tweaks at once for some serious A/B testing? Can’t you use multiple models at once?
Yes. You certainly can.
FlexPay started with about 15 machine learning models. They entered the necessary data and gave them all the same scenarios. Then, they let the models compete against each other and measured their results. Each time they measured results, they cut the bottom few performers to cull the models that didn’t show enough promise. Then, they instructed the models that produced the best results to train new models. They repeated this process ad nauseam.
FlexPay was able to adjust and refine their algorithms until their fraud detection models achieved a basic standard of accuracy. But getting further than that proved difficult. They were unable to increase its accuracy above certain levels without incorporating individualized data for each vendor. FlexPay found they needed different machine learning models for each client. To achieve a decline salvage rate above 70%, they assigned each vendor their own AI model to work exclusively with their own personal data sets.
Once the machine learning models start to incorporate fresh transactional data from a new client, they can make incremental improvements to problem‑solving. Most clients see the full results of FlexPay’s machine learning models after about 90 days of transaction history. The number of salvageable false declines caught using machine learning fraud models increases as new machine learning models develop. With more transactional data and more computing power, FlexPay will continue to get more accurate over time.
Don’t Call Them Regrets…
One thing that FlexPay wishes they knew before beginning their journey into machine learning is how many people were working on the same problem. Most of them were interested in helping and weren’t looking to compete.
The problem that FlexPay was solving wasn’t something that their peers were trying to address. Whether for lack of resources or interest, FlexPay didn’t have to deal with competitors. Still, worries about a competitive marketplace kept them secreted away for years to avoid tipping their hand to competitors.
Had they been more open about their project, they could have gotten more insight and expertise from others in their industry. And, if FlexPay had spoken with issuers, acquirers, and other actors in the payment system, they could have been better informed about how to improve their machine learning models, thus bringing their product along faster.
Why FlexPay’s Innovation Is Valuable to Banks & Credit Unions
One thing to note is that FlexPay’s technology isn’t critically important. They’re using machine learning to solve a problem that can be addressed without machine learning. Nevertheless, machine learning is a key differentiator. It already outperforms other approaches to solving false decline and transaction salvage issues. That trend will only continue.
While there are certainly other uses for the technology, FlexPay is focused on securing payments. Their introduction of machine learning to payment processing protects members, merchants, and institutions from false positives.
As they continue to push into new arenas of computer intelligence, they’re keeping an eye on their competitors and learning from them. They’re incorporating deep learning and neural networks to build better models trained on better, more relevant data. One of the ways FlexPay intends to improve their machine learning results is by improving the quality of their data. Distilling their data into the most important and impactful might provide as much insight as the broad, expansive data with which they’ve been working.
Although children’s stories are fun, you can only hear someone cry “wolf” so many times before it gets old. FlexPay wants to make sure that when you hear someone cry “wolf,” that there’s actually a wolf—or maybe it’s just a piece of falling sky in wolf’s clothing.
