Frank McSherry
@frankmcsherry.bsky.social
What a difference an allocator makes!
This is the same Rust program first using the system allocator, and then using mimalloc. About 100MB of working set in both cases, just .. apparently it pilots the system allocator to some horrible behavior.
Obviously going to start using mimalloc from now on.
This is the same Rust program first using the system allocator, and then using mimalloc. About 100MB of working set in both cases, just .. apparently it pilots the system allocator to some horrible behavior.
Obviously going to start using mimalloc from now on.
September 30, 2025 at 10:20 AM
What a difference an allocator makes!
This is the same Rust program first using the system allocator, and then using mimalloc. About 100MB of working set in both cases, just .. apparently it pilots the system allocator to some horrible behavior.
Obviously going to start using mimalloc from now on.
This is the same Rust program first using the system allocator, and then using mimalloc. About 100MB of working set in both cases, just .. apparently it pilots the system allocator to some horrible behavior.
Obviously going to start using mimalloc from now on.
The datafrog numbers, with the same plan (we think) hardcoded in, and explicit u32 types everywhere, gives us ...
August 14, 2025 at 11:10 PM
The datafrog numbers, with the same plan (we think) hardcoded in, and explicit u32 types everywhere, gives us ...
The datatoad numbers have lightly improved while we have been working the tries for the Galen queries, now at around 10s (formerly, like .. ten minutes? oh those days of easy improvements).
August 14, 2025 at 11:08 PM
The datatoad numbers have lightly improved while we have been working the tries for the Galen queries, now at around 10s (formerly, like .. ten minutes? oh those days of easy improvements).
Word is getting around.
July 8, 2025 at 11:51 AM
Word is getting around.
Happy to report that 5x of the 10x has now shipped, and should be live in the repo. What was once 50GB is now under 1GB. We have another 2x to go, but it's some new trie-based data structures (moving away from "sorted list" technology).
July 2, 2025 at 3:53 PM
Happy to report that 5x of the 10x has now shipped, and should be live in the repo. What was once 50GB is now under 1GB. We have another 2x to go, but it's some new trie-based data structures (moving away from "sorted list" technology).
Current side project: an interactive Datalog in Rust.
Should be a few blog posts about it coming, once I make sure it isn't utterly terrible. It may be a while though, with that constraint.
Should be a few blog posts about it coming, once I make sure it isn't utterly terrible. It may be a while though, with that constraint.
June 7, 2025 at 9:52 PM
Current side project: an interactive Datalog in Rust.
Should be a few blog posts about it coming, once I make sure it isn't utterly terrible. It may be a while though, with that constraint.
Should be a few blog posts about it coming, once I make sure it isn't utterly terrible. It may be a while though, with that constraint.
Hat tip to dzfrias.dev/blog/rust-wa..., which resulted in a correspondingly minimal Rust+Wasm+Timely set-up. The whole story can be told with a few screenshots (plus reading the link for the wasm-pack command).
May 7, 2025 at 11:21 PM
Hat tip to dzfrias.dev/blog/rust-wa..., which resulted in a correspondingly minimal Rust+Wasm+Timely set-up. The whole story can be told with a few screenshots (plus reading the link for the wasm-pack command).
Fun dataflow news: prompted initially by @oliwessels.bsky.social, we've finally merged changes into dataflows timely and differential that allow programs that use them to be built into WASM, and run in the browser.
May 7, 2025 at 11:14 PM
Fun dataflow news: prompted initially by @oliwessels.bsky.social, we've finally merged changes into dataflows timely and differential that allow programs that use them to be built into WASM, and run in the browser.
The paper has as a goal, I think, the introduction of "a leaky IVM implementation" (dynamic tables) that starts from the premise that they will *not* plan to keep up with with changes to their inputs.
April 17, 2025 at 10:06 AM
The paper has as a goal, I think, the introduction of "a leaky IVM implementation" (dynamic tables) that starts from the premise that they will *not* plan to keep up with with changes to their inputs.
Summertime, but otherwise the same (Richmond vs Charlotte)!
January 17, 2025 at 10:16 PM
Summertime, but otherwise the same (Richmond vs Charlotte)!
Today's problem was a nice warm up, and for me a sudden introduction to the differences between RANK(), DENSE_RANK(), and ROW_NUMBER(). I tried them in that order. T.T
December 1, 2024 at 3:33 PM
Today's problem was a nice warm up, and for me a sudden introduction to the differences between RANK(), DENSE_RANK(), and ROW_NUMBER(). I tried them in that order. T.T
Temporal filters are WHERE clauses in which you reference "time", and involve it in inequalities. Materialize can back solve for the precise moments at which each record should enter and ultimately leave the result set. In this example, Materialize will schedule retractions at exactly expire_ts.
November 27, 2024 at 7:35 PM
Temporal filters are WHERE clauses in which you reference "time", and involve it in inequalities. Materialize can back solve for the precise moments at which each record should enter and ultimately leave the result set. In this example, Materialize will schedule retractions at exactly expire_ts.
Not the usual content I put out, but pretty pleased with it.
November 12, 2024 at 7:50 PM
Not the usual content I put out, but pretty pleased with it.