One of the first principles we adopted after decomposing the system was simple:
Every service should own its own data.
It sounded right.
The architecture became cleaner.
Teams could deploy independently.
Services no longer needed direct access to another service’s tables.
Ownership was explicit.
But then reality arrived.
The reporting team needed data from three services.
The payment service needed information owned by the customer service.
The ledger needed transaction context from the payment system.
Operations wanted a single view of a customer’s financial activity.
And suddenly, the question wasn’t:
“Should every service have its own database?”
It was:
“What does data ownership actually mean in a distributed system?”
That turned out to be a much harder architectural question.
1. The Shared Database Wasn’t Always the Problem
Before decomposition, the architecture looked roughly like this:
Application
│
▼
┌─────────────┐
│ Database │
└─────────────┘
/ | \
/ | \
▼ ▼ ▼
Payments Orders Customers
Every module could access the same database.
That had obvious advantages.
A cross-domain query was easy.
SELECT ...
FROM payments
JOIN customers ...
JOIN orders ...
Transactions across related data were straightforward.
Reporting was relatively simple.
There was one source of truth.
But there was a serious downside.
The database had effectively become a shared integration layer.
Multiple teams depended on:
- Tables
- Columns
- Foreign keys
- Indexes
- Stored procedures
- Transaction behavior
- Database-specific behavior
A schema change in one area could affect several teams.
The database became a coupling mechanism.
2. Then We Introduced Database Per Service
As the system became more distributed, we moved toward:
Payment Service
│
▼
Payment DB
Order Service
│
▼
Order DB
Customer Service
│
▼
Customer DB
Ledger Service
│
▼
Ledger DB
The principle was attractive:
A service owns its data and controls how that data changes.
The Payment Service didn’t directly modify the Order database.
The Order Service didn’t query the Customer database directly.
Instead, services communicated through APIs or events.
This created a much stronger architectural boundary.
But it also created a new problem.
Data that used to be joined inside one database now had to be connected across systems.
3. Ownership Became a Real Architectural Concept
We had to define what “ownership” actually meant.
For example:
Customer Service
└── Customer identity
Payment Service
└── Payment state
Order Service
└── Order state
Inventory Service
└── Inventory state
Ledger Service
└── Financial ledger state
The owner became responsible for:
- Validating changes
- Maintaining invariants
- Controlling writes
- Managing schema evolution
- Publishing relevant events
- Maintaining data quality
Other services could consume information.
But they shouldn’t silently become another writer.
This distinction was important.
A copy of data is not the same thing as ownership.
4. The First Problem: Duplication
Once services stopped sharing tables, we inevitably duplicated some information.
For example:
Customer Service
│
└── Customer ID
└── Customer Name
└── Customer Status
The Payment Service might need some of that information.
So we could either call Customer Service every time:
Payment Service
│
▼
Customer Service
or maintain a local representation:
Customer Service
│
│ CustomerUpdated
▼
Payment Service
│
▼
Local Customer View
The second approach reduced runtime coupling.
But now we had duplicated data.
And duplicated data creates a consistency problem.
5. Duplication Isn’t Automatically Bad
This was one of the lessons that took some time to internalize.
We initially treated duplication as something to avoid.
But in distributed systems, duplication can be intentional.
Suppose the Payment Service needs:
customer_id
customer_status
risk_category
It may be reasonable to maintain a local projection of those fields.
The important question isn’t:
“Is this data duplicated?”
The better question is:
“Which system owns the authoritative version, and what consistency does the copy require?”
For example:
Customer Service
│
│ authoritative
▼
Customer Data
Payment Service
│
│ derived copy
▼
Payment Customer View
Now the architecture has an explicit ownership model.
6. Synchronization Became Part of the Architecture
Once data was duplicated, we needed a synchronization mechanism.
One common pattern was events:
Customer Service
│
▼
CustomerUpdated
│
▼
Message Broker
│
├────────► Payment Service
│
├────────► Order Service
│
└────────► Reporting
Each consumer could update its own local representation.
This reduced synchronous dependencies.
But it introduced eventual consistency.
For a short period:
Customer DB
Status = ACTIVE
Payment DB
Status = PENDING
The two systems might temporarily disagree.
That isn’t necessarily a bug.
It is a consequence of choosing distributed data ownership.
The important thing is that the business can tolerate the difference.
7. Not Every Data Item Needs the Same Consistency
This became one of the most important design questions.
Consider customer display information.
A slight delay might be acceptable.
Customer name
Customer profile
Marketing preferences
But other information may require much stronger guarantees.
For example:
Available balance
Ledger entry
Payment settlement state
Account ownership
A stale balance can have serious consequences.
So we stopped asking:
“Should our system use eventual consistency?”
That question is too broad.
We started asking:
“Where can eventual consistency be tolerated, and where must the business state remain strongly consistent?”
Consistency became a property of individual business invariants rather than a single system-wide setting.
8. The Cross-Service Query Problem
Then reporting arrived.
Someone wanted:
“Show me every payment, the customer, the order, the settlement status, and the ledger entry.”
In a shared database:
SELECT ...
FROM payments
JOIN customers
JOIN orders
JOIN ledger
Easy.
With database-per-service:
Payment DB
Order DB
Customer DB
Ledger DB
There was no simple database join anymore.
We had to decide what architecture should provide that view.
9. The First Wrong Solution: Let Everyone Query Everyone
One tempting solution was:
Payment Service
│
├──► Customer DB
├──► Order DB
└──► Ledger DB
It solved the immediate problem.
But it destroyed the boundary we had created.
Now the Payment Service depended on:
- Customer schema
- Order schema
- Ledger schema
A database migration in another service could break it.
The database had become a distributed API without an explicit contract.
We quickly realized:
Database-per-service doesn’t work if services are still directly querying each other’s databases.
You have database separation without data ownership.
10. The Better Solution: Explicit Read Models
For cross-service reporting, we created purpose-built read models.
Conceptually:
Payment Events ──────┐
│
Order Events ────────┤
▼
Customer Events ─────► Reporting Pipeline
│
Ledger Events ───────┘
│
▼
Reporting Store
The reporting system could build a denormalized representation optimized for queries.
Instead of asking five services for information every time, it could query its own data store.
This created another copy of the data.
But this time the duplication had a purpose.
The reporting database wasn’t the owner of the underlying business state.
It was a derived read model.
11. Reporting Is Not the Same as Source of Truth
This distinction became critical in financial systems.
Suppose the reporting database says:
Payment = SETTLED
That doesn’t necessarily make the reporting database authoritative.
The authoritative state may belong to the Payment or Ledger domain.
The reporting store is derived from those sources.
If something goes wrong:
Source of Truth
│
▼
Events
│
▼
Reporting Projection
we can rebuild the projection.
That is a very different relationship from:
Reporting DB
│
▼
Authority
The architecture needs to make that distinction explicit.
12. The Problem With “One Source of Truth”
We often heard:
“We need one source of truth.”
That’s a useful phrase, but it can become misleading.
A large distributed system may have multiple authoritative datasets.
For example:
Customer Service
→ customer identity
Payment Service
→ payment lifecycle
Ledger Service
→ financial postings
Order Service
→ order lifecycle
There isn’t necessarily one giant database containing every authoritative fact.
Instead, there can be multiple domain-specific sources of truth.
The important question becomes:
Who owns each fact?
That is much more useful than simply asking where the data is stored.
13. Ownership Must Include Writes
We eventually made one rule particularly strict:
Only the owning service writes authoritative state.
For example:
Payment Service
│
▼
Payment State
Other services could request a payment operation:
Order Service
│
▼
Payment API
│
▼
Payment State
But the Order Service shouldn’t directly modify:
payment_status
inside the Payment database.
That would create two owners.
And two owners usually means ambiguous authority.
14. The “Shared Database” Can Exist for the Right Reasons
This doesn’t mean that a shared database is always wrong.
There are situations where a shared database can be the right architecture.
For example, if several modules:
- Require strong transactional consistency
- Share tightly coupled invariants
- Are owned by the same team
- Evolve together
- Have similar availability requirements
then keeping them inside one database may be completely reasonable.
The mistake is treating:
database-per-service
as a mandatory architectural rule.
The database boundary should follow the business and consistency boundary.
Not fashion.
15. The Real Cost of Database Per Service
Database-per-service provides benefits:
- Clear ownership
- Independent schema evolution
- Reduced coupling
- Independent scaling
- Better service autonomy
But it also introduces costs:
- Data duplication
- Event propagation
- Eventual consistency
- More operational infrastructure
- Cross-service reporting complexity
- More complicated debugging
- Reconciliation requirements
- More difficult transactional workflows
The architecture isn’t automatically better.
It has simply moved complexity.
16. We Started Thinking in Consistency Boundaries
The most useful architectural question became:
Which data must change atomically together?
Suppose:
Payment
Ledger
must always satisfy a strict business invariant.
That relationship may deserve a strong consistency boundary.
But:
Payment
Customer Display Name
may not require the same level of coupling.
Similarly:
Ledger
Reporting Dashboard
can often tolerate asynchronous propagation.
This gave us a more useful model:
Business Invariant
│
▼
Consistency Boundary
│
▼
Data Ownership Boundary
│
▼
Service Boundary
The boundaries should reinforce one another.
17. The Database Boundary Should Follow the Business Boundary
One of the mistakes we made early was designing services around technical components.
For example:
Database Team
Payment Team
Reporting Team
API Team
Then trying to determine which database each service should own.
The better approach was to start with business capabilities:
Payments
Orders
Customers
Ledger
Settlement
Then ask:
Which data and invariants belong to each capability?
That produced much more meaningful ownership.
18. The Financial Systems Exception
Financial systems make these decisions more sensitive.
Consider a ledger.
You don’t want several services independently modifying financial state.
For example:
Payment Service ──► Ledger DB
Order Service ────► Ledger DB
Refund Service ───► Ledger DB
Admin Service ────► Ledger DB
That creates ambiguous authority.
Instead, a ledger service should own the ledger state and expose controlled operations.
Payment Service
│
▼
Ledger Service
│
▼
Ledger DB
Other systems can consume ledger events.
But they shouldn’t become alternate writers.
The architectural boundary protects the financial invariant.
19. What We Actually Changed
The final architecture wasn’t:
“Everything must have its own database.”
It was more nuanced.
We established explicit ownership.
Customer Service
│
└── Customer Data
Payment Service
│
└── Payment Data
Order Service
│
└── Order Data
Ledger Service
│
└── Ledger Data
Then we created controlled mechanisms for sharing information:
Events
│
┌────────────┼────────────┐
▼ ▼ ▼
Reporting Analytics Other Services
For queries requiring data from multiple domains:
Services
│
▼
Events
│
▼
Derived Read Model
And for operations that changed authoritative state:
Consumer
│
▼
Owning Service
│
▼
Authoritative Data
This preserved ownership while still allowing the broader system to work with shared information.
20. What We Would Not Do Now
We wouldn’t automatically create a database for every service.
We wouldn’t allow services to query each other’s databases directly.
We wouldn’t duplicate authoritative data without defining who owns it.
We wouldn’t assume eventual consistency is acceptable for every business operation.
We wouldn’t build reporting queries by synchronously calling ten services.
We wouldn’t treat a reporting database as the financial source of truth.
And we wouldn’t choose database boundaries before understanding business ownership and consistency requirements.
21. The Questions I Ask Before Giving a Service Its Own Database
Before creating a database boundary, I ask:
Who owns this data?
Then:
Who is allowed to change it?
Then:
Which business invariants depend on it?
Then:
Which consumers need the data?
Then:
Can those consumers tolerate eventual consistency?
Then:
How will reporting work?
And finally:
What happens when the source and the derived copy disagree?
If those questions don’t have clear answers, the database boundary probably isn’t ready.
22. The Bigger Architectural Lesson
The debate is often presented as:
Shared Database
vs
Database Per Service
But that framing is too simplistic.
The real decision is about:
Ownership
↓
Consistency
↓
Coupling
↓
Business Boundaries
↓
Data Architecture
A shared database gives you easier joins and stronger transactional coordination.
Database-per-service gives you stronger ownership and service autonomy.
Both have costs.
The right architecture depends on which constraints matter most.
23. Final Thought
At first, we thought database ownership was a technical decision.
Which database should this service connect to?
Which schema should it use?
Which tables should it access?
Eventually, we realized the more important question was:
Who owns this piece of business state?
Once that was clear, many other decisions became easier.
The owning service controls authoritative writes.
Other services consume information through explicit contracts.
Derived copies are allowed when they solve a real problem.
Reporting gets its own read models.
Strong consistency is reserved for invariants that actually require it.
And the architecture becomes much easier to reason about.
The lesson was simple:
Database ownership is an architectural boundary.
It defines more than where data lives.
It defines who is responsible for its correctness, who can change it, and what the rest of the system must do when it needs that information.
In financial systems, that boundary matters even more.
Because when the question is “Who owns this data?”, you’re often really asking:
“Who is responsible for getting the financial state right?”