For 18 months, we built our own fraud detection system.
The reasoning seemed sound.
Fraud was important.
Payments were core to the business.
We had strong engineers.
We had access to transaction data.
And building the capability ourselves would give us complete control.
So we built.
We created data pipelines.
We built feature extraction.
We trained models.
We created scoring services.
We built dashboards.
We added rules.
We tuned thresholds.
We created feedback loops.
We integrated the system into the payment flow.
Technically, it worked.
But there was a problem.
We were still losing the fraud arms race.
Our fraud rate wasn’t improving as quickly as we expected.
Our engineers were spending increasing amounts of time maintaining models and infrastructure.
And every improvement required another round of experimentation.
Eventually, we had to ask a difficult architectural question:
Were we building a strategic capability, or were we building infrastructure that someone else could provide better?
The answer changed our architecture.
1. The Problem We Created Ourselves
The original decision looked reasonable.
We wanted control over fraud detection, so we decided to build it internally.
The architecture looked roughly like this:
Payment Request
│
▼
Payment Service
│
▼
Fraud Detection
┌─────┴─────┐
│ │
▼ ▼
Rules Engine ML Models
│ │
└─────┬─────┘
▼
Risk Decision
│
┌─────┴─────┐
▼ ▼
Allow Block
Around that simple diagram was a surprising amount of infrastructure.
We needed:
- Transaction data pipelines
- Feature engineering
- Model training
- Model deployment
- Model monitoring
- Fraud labeling
- Threshold management
- Rule management
- Data quality checks
- Feedback mechanisms
- Performance monitoring
- Incident handling
The fraud detector became a product of its own.
And that created the first warning sign.
2. The Hidden Cost of Building
The initial estimate focused on implementation.
How long would it take to build the service?
That wasn’t the real question.
The real question was:
How much would it cost to operate this capability well for the next five years?
Fraud detection isn’t a system you build once.
Fraudsters adapt.
Transaction patterns change.
Attack techniques evolve.
False positives need investigation.
Models need retraining.
Features need maintenance.
Data quality changes.
New payment methods appear.
New markets create new patterns.
Regulatory expectations evolve.
So the actual lifecycle looked more like:
Build
↓
Deploy
↓
Monitor
↓
Detect Drift
↓
Retrain
↓
Tune
↓
Investigate
↓
Repeat
The engineering investment never stopped.
We hadn’t purchased a feature.
We had taken ownership of an entire capability.
3. The Question We Should Have Asked Earlier
For a long time, the conversation was:
“Can we build fraud detection ourselves?”
The answer was obviously yes.
We had capable engineers.
We had the technology.
We had the data.
But that was the wrong question.
The better question was:
“Should fraud detection be one of the capabilities we build and operate ourselves?”
Those questions have completely different answers.
Almost anything can be built.
That doesn’t mean it should be.
4. We Evaluated Our Core Competency
We stepped back and looked at where the company actually created differentiated value.
Our competitive advantage was around:
- Payment reliability
- Payment experience
- Merchant integrations
- Transaction workflows
- Settlement
- Financial operations
- Customer experience
Fraud detection was critical.
But it wasn’t where we believed we could build a durable competitive advantage through proprietary ML infrastructure.
That distinction mattered.
A capability can be:
Important
without being:
Strategically differentiating.
Those aren’t the same thing.
5. The Strategic Capability Test
We started evaluating systems using a simple set of questions.
Is this core to our competitive advantage?
If yes, building may make sense.
Does owning the capability create meaningful differentiation?
If yes, building becomes more attractive.
Do we have unique data or expertise that gives us an advantage?
If yes, internal development may be valuable.
Is the capability changing rapidly?
If yes, maintaining it internally can become expensive.
Are specialized vendors operating at much larger scale?
If yes, buying may provide capabilities we would struggle to reproduce.
Does building distract engineers from our core product?
If yes, the opportunity cost matters.
This changed the discussion from:
“Build vs. buy.”
to:
“Where should we spend our engineering complexity budget?”
6. The 18-Month Reality Check
After 18 months, we had a capable fraud system.
But capability wasn’t the same as competitiveness.
We compared our internal results with what specialized providers could offer.
The gap was uncomfortable.
A specialized provider had:
- Larger transaction datasets
- More fraud signals
- More customers generating feedback
- Specialized fraud research
- Dedicated model teams
- Continuous model updates
- Global fraud patterns
- Mature operational infrastructure
We were maintaining a specialized ML platform for one business.
They were operating a specialized fraud platform as their business.
That difference in scale mattered.
7. The Network Effect We Couldn’t Easily Recreate
Fraud detection has an unusual characteristic.
Patterns seen in one environment can help identify patterns elsewhere, subject to appropriate privacy, contractual, and legal constraints.
A specialized provider may observe attack patterns across many merchants and payment environments.
We saw only our own traffic.
Conceptually:
Our Internal System
Transactions
│
▼
Our Data
│
▼
Our Models
Whereas a specialized provider could potentially operate with a much broader signal environment:
Multiple Customers
│
▼
Broader Signal Set
│
▼
Specialized Models
│
▼
Risk Decision
That scale advantage was difficult for us to reproduce economically.
8. The Three Options We Considered
We considered three paths.
Option 1: Keep Building
We could continue investing internally.
More engineers.
More models.
More features.
More data.
More experimentation.
This would give us maximum control.
But it also meant continuing to invest heavily in a capability that wasn’t our primary differentiator.
Option 2: Buy Everything
We could completely replace our internal fraud system with a third-party service.
That would reduce our internal engineering burden.
But it would create a different problem.
We would become heavily dependent on an external decision engine.
We could lose control over:
- Risk policy
- Integration behavior
- Latency
- Availability
- Explainability
- Vendor changes
- Pricing
- Data handling
That wasn’t acceptable for a critical payment path.
Option 3: Buy the Commodity, Own the Decision
This became the most interesting option.
We didn’t have to choose between:
Build everything
and:
Buy everything
We could separate the capability into layers.
The architecture became:
Payment Request
│
▼
Payment Service
│
▼
Risk Evaluation
┌─────┴─────┐
│ │
▼ ▼
Internal Rules External
Fraud Signal
│ │
└─────┬─────┘
▼
Risk Decision
│
┌─────┴─────┐
▼ ▼
Allow Review/Block
We bought specialized fraud intelligence.
We retained ownership of the business decision.
That distinction was important.
9. We Designed a Provider Boundary
We didn’t want the payment system to depend directly on a vendor-specific API everywhere.
Instead, we introduced an internal abstraction.
Conceptually:
Payment Service
│
▼
Fraud Evaluation Interface
│
├───────────────┐
▼ ▼
Provider A Provider B
The payment domain knew about:
RiskEvaluation
not:
VendorSpecificFraudAPI
This reduced coupling.
It also gave us the ability to change providers later without rewriting the payment workflow.
The vendor became an implementation detail.
The business capability remained ours.
10. We Didn’t Outsource the Business Decision
This was one of the most important design choices.
The external service could provide signals such as:
Risk Score
Device Signal
Transaction Signal
Behavioral Signal
Velocity Signal
But our system still applied business policy.
For example:
External Risk Signal
+
Transaction Context
+
Merchant Policy
+
Payment Rules
+
Account State
↓
Internal Decision
That allowed us to retain control over the financial workflow.
The provider helped answer:
“How risky does this transaction look?”
Our system remained responsible for:
“What should our business do about it?”
11. We Designed for Failure
Once we introduced a third-party dependency, we had to take reliability seriously.
A fraud service is now part of the payment architecture.
What happens if it becomes slow?
What happens if it is unavailable?
What happens if it returns an unexpected response?
What happens if the network fails?
What happens if the provider changes behavior?
We couldn’t simply assume:
Fraud Service
↓
Always Available
Instead:
Payment
│
▼
Risk Evaluation
│
├── Success → Decision
│
├── Timeout → Fallback Policy
│
├── Failure → Fallback Policy
│
└── Unknown → Review / Safe Handling
The exact fallback depends on the risk tolerance and payment operation.
The important architectural principle was:
Buying a capability doesn’t remove the need to design for its failure.
It moves the failure boundary.
12. We Added Timeouts
One of the simplest but most important controls was a strict timeout.
A payment request shouldn’t wait indefinitely for an external risk provider.
Without a timeout:
Payment
↓
Fraud Service
↓
Waiting...
↓
Waiting...
↓
Waiting...
A problem in the fraud service can become a payment outage.
With a bounded timeout:
Payment
↓
Fraud Service
↓
Timeout
↓
Fallback Policy
The system remains predictable.
Timeouts don’t solve dependency failure.
They prevent dependency failure from consuming unlimited resources.
13. We Added Observability Around the Vendor Boundary
The external dependency also became an important observability boundary.
We tracked things such as:
- Fraud evaluation latency
- Timeout rate
- Error rate
- Decision distribution
- Provider availability
- Fallback frequency
- Payment impact
- Fraud outcomes
We wanted to know not only:
“Is the vendor up?”
but:
“Is the vendor still helping us make good decisions?”
A provider can have 99.99% availability and still perform poorly from a business perspective if its decisions become less useful.
14. The Contract Became More Important Than the API
Vendor integration isn’t simply:
HTTP request
↓
HTTP response
For a critical financial capability, the real contract includes:
Availability
Latency
Data handling
Security
Failure behavior
Versioning
Support
Auditability
Service levels
Exit strategy
We therefore evaluated the provider as part of the architecture, not merely as a library or API.
That distinction matters.
When you outsource a critical capability, you’re creating a dependency.
Dependencies need architecture.
15. We Also Considered Data Ownership
Fraud detection often requires sensitive information.
That meant we couldn’t treat data sharing casually.
We had to consider:
- What data leaves our environment?
- Why is it required?
- How is it protected?
- Where is it processed?
- How long is it retained?
- What contractual controls apply?
- What regulatory requirements apply?
- Can we delete or retrieve it?
- Who can access it?
Buying software doesn’t transfer responsibility for protecting customer data.
The architecture still needs clear data boundaries.
16. The Unexpected Benefit: Engineers Returned to the Core Product
The most visible change wasn’t actually the fraud service.
It was what happened to the engineering team.
Before:
Fraud Infrastructure
│
├── Models
├── Pipelines
├── Training
├── Monitoring
├── Feature Engineering
└── Maintenance
After:
External Fraud Capability
│
▼
Payment Platform
│
├── Payment UX
├── Reliability
├── Merchant Experience
├── Settlement
└── Core Product
The team could spend more time improving the capabilities that differentiated the business.
That was the real opportunity cost we had underestimated.
17. The Results
After the transition, the results were significant.
Fraud losses dropped by approximately 40%.
The engineering team’s velocity roughly doubled.
More importantly, we stopped treating fraud infrastructure as a project that needed to be continuously rebuilt.
It became a managed dependency with explicit boundaries.
We could focus on:
Payment Experience
↓
Reliability
↓
Merchant Capabilities
↓
Financial Workflows
Instead of spending the majority of our effort maintaining an ML platform.
18. But Buying Wasn’t Free
It would be misleading to say that buying solved everything.
We traded one set of problems for another.
We now had:
- Vendor dependency
- Contractual risk
- Integration complexity
- External availability risk
- Pricing exposure
- Data governance requirements
- Migration concerns
- Vendor lock-in risk
That is the fundamental trade-off.
Build gives you more control but more ownership.
Buy reduces implementation burden but introduces dependency.
The right architecture manages that trade-off deliberately.
19. The Dangerous “Build Everything” Mentality
Engineering organizations sometimes associate building with technical excellence.
If we can build it ourselves, why wouldn’t we?
Because engineering capacity is finite.
Every internal system has a cost.
Build
↓
Maintain
↓
Operate
↓
Secure
↓
Monitor
↓
Upgrade
↓
Eventually Replace
The question isn’t whether your engineers are capable of building something.
The question is whether that capability deserves your engineers’ time.
20. When Building Is the Better Choice
Buying isn’t automatically better.
There are situations where building is clearly the right decision.
Build when the capability:
- Differentiates your product
- Creates proprietary advantage
- Requires unique business logic
- Depends heavily on unique internal data
- Needs specialized behavior vendors cannot provide
- Is strategically important to control
- Has unacceptable external dependency risk
For example, we would be much more cautious about outsourcing a capability that directly defines our competitive advantage.
The decision depends on the architecture and the business.
21. When Buying Is the Better Choice
Buying becomes attractive when:
- The capability is mature in the market
- Specialized vendors have significant scale advantages
- The capability isn’t a core differentiator
- Maintaining it internally requires specialized expertise
- The vendor can provide better economics
- The integration boundary can be controlled
- The dependency can fail safely
- An exit strategy is possible
The key phrase is:
Can we buy this capability without surrendering control of the business-critical decision?
If yes, buying can be powerful.
22. The Questions I Ask Now
Before building a major platform capability, I ask:
Is this core to our competitive advantage?
Then:
Would owning this capability create meaningful differentiation?
Then:
Do we have unique expertise or data that makes us unusually good at it?
Then:
How much operational ownership are we taking on?
Then:
Are specialized providers structurally better positioned to solve this problem?
And finally:
If we buy it, can we create a clean boundary and maintain an exit strategy?
These questions prevent “build because we can” from becoming an architecture strategy.
23. The Bigger Architectural Lesson
Architecture isn’t about maximizing the amount of software you own.
It’s about deciding which capabilities you should own.
A mature architecture might look like:
Business
│
┌────────────┴────────────┐
│ │
▼ ▼
Core Capability Specialized Capability
│ │
▼ ▼
Build Buy
│ │
└────────────┬────────────┘
▼
Clean Boundary
The best systems aren’t necessarily the ones with the most internally built components.
They’re the ones where ownership aligns with strategy.
24. The Principle We Took Away
The biggest lesson wasn’t:
“Buy fraud detection.”
That would be too simplistic.
The lesson was:
Don’t confuse technical capability with strategic capability.
We could build fraud detection.
We did build it.
But building it didn’t make it our competitive advantage.
It made it our responsibility.
Those are very different things.
25. Final Thought
For 18 months, we thought the engineering challenge was:
“How do we build a better fraud detection system?”
The better question was:
“Is building a fraud detection system how we should be spending our engineering capacity?”
Once we asked that question, the architecture changed.
We bought a specialized capability.
We kept ownership of the business decision.
We created a clean integration boundary.
We designed for provider failure.
We protected the data boundary.
And we redirected our engineering effort toward the parts of the payment platform that actually differentiated us.
There is a certain maturity in knowing how to build something.
There is another kind of maturity in knowing when not to.
Because software architecture isn’t about building everything yourself.
It’s about deciding what must be yours.
Not everything you can build is worth building.