A step-by-step guide to building a key-value database from scratch.

AgentDev, toolset and UI for Git Worktrees and agent sessions - My opinionated, more flexible version of Cursor 2.0

There's no money left in real-time event streaming. A major consolidation wave is beginning.

From NervesConf US 2024

A deep dive into how Linux manages process memory, page tables, and virtual address spaces

It is 1995.

Subscribe • Previous Issues

Conference highlights.

From GPU architecture and PTX/SASS to warp-tiling and deep asynchronous tensor core pipelines.

Iceberg v3 gave CDC identity and precision, v4 makes change detection light and metadata compact. Streaming icy edges finally start to…

Corrosion is distributed service discovery based on Rust, SQLite, and CRDTs.

See how Husky enables interactive querying across 100 trillion events daily by combining caching, smart indexing, and query pruning.

Interactive features without events

R2 SQL provides a built-in, serverless way to run ad-hoc analytic queries against your R2 Data Catalog. This post dives deep under the Iceberg into how we built this distributed engine, from its metadata-driven planner to its parallel execution model.

2 Months of Banging My Head Against Send/Sync, Explained in One Coffee Break

The rapid development of large language models (LLMs) has led to the widespread deployment of LLM agents across diverse industries, including customer service, content generation, data analysis, and even healthcare. However, as more LLM agents are deployed, a major issue has emerged: there is no standard way for these agents to communicate with external tools or data sources. This lack of standardized protocols makes it difficult for agents to work together or scale effectively, and it limits their ability to tackle complex, real-world tasks. A unified communication protocol for LLM agents could change this. It would allow agents and tools to interact more smoothly, encourage collaboration, and triggering the formation of collective intelligence. In this paper, we provide the first comprehensive analysis of existing agent protocols, proposing a systematic two-dimensional classification that differentiates context-oriented versus inter-agent protocols and general-purpose versus domain-specific protocols. Additionally, we conduct a comparative performance analysis of these protocols across key dimensions such as security, scalability, and latency. Finally, we explore the future landscape of agent protocols by identifying critical research directions and characteristics necessary for next-generation protocols. These characteristics include adaptability, privacy preservation, and group-based interaction, as well as trends toward layered architectures and collective intelligence infrastructures. We expect this work to serve as a practical reference for both researchers and engineers seeking to design, evaluate, or integrate robust communication infrastructures for intelligent agents.

This Berkeley systems group paper opens with the thesis that LLM agents will soon dominate data system workloads. These agents, acting on b...

We launched a new product called WarpStream Tableflow that is the easiest, cheapest, and most flexible way to convert Kafka topic data into Iceberg tables with low latency, and keep them compacted.

Hummock is a cloud-native LSM storage engine developed by us. In this article, we will introduce some optimizations that Hummock has made for streaming.

<img src="https://media.licdn.com/dms/image/v2/D4D08AQFRkRFMl4JfiA/croft-frontend-shrinkToFit767/B4DZhTZuRuHsAQ-/0/1753745911803?e=2147483647&v=beta&t=ZSK-8rwkoizGzDG3CIDTLfiaj9_mOqlL91DuqOSg43M" /><br/>

Open-source search and retrieval database for AI applications. - chroma-core/chroma

wal3, Chroma's write-ahead log, combining a 30-year-old lock-free algorithm with S3's newest conditional writes feature.

This is a data system internals blog post. So if you enjoyed my table formats internals blog posts , or writing on Apache Kafka internals or Apache BookKeeper internals , you might enjoy this one. But beware, it’s long and detailed. Also note that I work for Confluent, which also runs Apac

Embedding sizes and architectures have changed remarkably over the past 5 years