Philosopher Ping
philosopherping.com
Philosopher Ping
@philosopherping.com
Explorations of software, philosophy, work, life, the universe and everything else.
You can see the full post, along with all diagrams, here: philosopherping.com/hybrid-store...
Hybrid Store Design Pattern
🔂
philosopherping.com
May 5, 2025 at 4:08 PM
Hybrid Stores are a way to implement Lambda Architectures, but by merging at write-time, rather than read-time. They can also support Kappa Architectures.

@venicedb.org is an open source database implementing the Hybrid Store design pattern, and other proprietary systems do so as well.
May 5, 2025 at 4:08 PM
- After the buffer replay is caught up, switch the current version pointer from N to N+1. Version N can be kept as a backup. There is no future version anymore (until the next batch job).
May 5, 2025 at 4:08 PM
- Replay recent data from the real-time buffer onto the future version N+1.
May 5, 2025 at 4:08 PM
When the batch data source runs and produces a new version of the dataset, the Hybrid Store design pattern then does the following:

- Create a new version N+1 of the dataset (called a future version) and load the batch data into it.
May 5, 2025 at 4:08 PM
The batch data source is a job that produces new versions of the dataset. The real-time data source, on the other hand, appends data to a real-time buffer, and this data eventually gets written to all versions of the dataset.
May 5, 2025 at 4:08 PM
The Coupling Cost of Leverage
⛓️
philosopherping.com
January 31, 2025 at 3:56 AM
This maintenance gain could be especially significant if system B offers a correct solution to a difficult problem. That being said, some nuances are missing.
The Coupling Cost of Leverage
⛓️
philosopherping.com
January 31, 2025 at 3:56 AM
The case in favor of A leveraging B was that it would decrease the maintenance cost of system A, since more functionality would be “outsourced” to system B (owned by another team).
The Coupling Cost of Leverage
⛓️
philosopherping.com
January 31, 2025 at 3:56 AM