Aviv Dozorets
@themoah.bsky.social
Data engineer.
Anything less than PB scale isn’t big 😬
Anything less than PB scale isn’t big 😬
Lesson learned: don't watch Severance before sleep. Had craziest dreams.
May 4, 2025 at 12:32 PM
Lesson learned: don't watch Severance before sleep. Had craziest dreams.
Lets try Google Firebase studio.
Asked to generate simple api serverless app, that would serve as endpoint to send logs and metrics.
Asked to do it in JS and target CloudFlare workers runtime.
What did I get?
UI application in TS, that returns hardcoded JWT.
Asked to generate simple api serverless app, that would serve as endpoint to send logs and metrics.
Asked to do it in JS and target CloudFlare workers runtime.
What did I get?
UI application in TS, that returns hardcoded JWT.
April 17, 2025 at 7:44 AM
Lets try Google Firebase studio.
Asked to generate simple api serverless app, that would serve as endpoint to send logs and metrics.
Asked to do it in JS and target CloudFlare workers runtime.
What did I get?
UI application in TS, that returns hardcoded JWT.
Asked to generate simple api serverless app, that would serve as endpoint to send logs and metrics.
Asked to do it in JS and target CloudFlare workers runtime.
What did I get?
UI application in TS, that returns hardcoded JWT.
How to make money online 101: OpenAI version.
March 12, 2025 at 3:34 PM
How to make money online 101: OpenAI version.
- Sell me this pen.
- It's AI-powered.
- It's AI-powered.
March 11, 2025 at 7:21 PM
- Sell me this pen.
- It's AI-powered.
- It's AI-powered.
Hm.. So I've asked multiple LLMs same prompt - "tell me something I don't know".
Both grok-3 and chatgpt-4o responded with same answer.
Qwen-32b and llama3.2 had almost same response.
Both grok-3 and chatgpt-4o responded with same answer.
Qwen-32b and llama3.2 had almost same response.
March 11, 2025 at 11:52 AM
Hm.. So I've asked multiple LLMs same prompt - "tell me something I don't know".
Both grok-3 and chatgpt-4o responded with same answer.
Qwen-32b and llama3.2 had almost same response.
Both grok-3 and chatgpt-4o responded with same answer.
Qwen-32b and llama3.2 had almost same response.
I don't understand: it's 2025 and still no one sells computer monitors with built in level (bubble level) ?
🤯
🤯
February 17, 2025 at 9:33 AM
I don't understand: it's 2025 and still no one sells computer monitors with built in level (bubble level) ?
🤯
🤯
Amen to that.
Especially on humping part. Still being curious is fundamental.
And I'm learning Rust.
antirez.com/news/145
Especially on humping part. Still being curious is fundamental.
And I'm learning Rust.
antirez.com/news/145
We are destroying software - <antirez>
antirez.com
February 16, 2025 at 1:50 PM
Amen to that.
Especially on humping part. Still being curious is fundamental.
And I'm learning Rust.
antirez.com/news/145
Especially on humping part. Still being curious is fundamental.
And I'm learning Rust.
antirez.com/news/145
This is amazing. I should really consider it.
Just like you'd use emr or any other managed compute platform.
Hopefully quality is there.
Just like you'd use emr or any other managed compute platform.
Hopefully quality is there.
We are happy to share more about what we are building and our goal to run Polars on any dataset size!
A managed Distributed Polars compute cluster to ensure a single DataFrame API for all your needs.
pola.rs/posts/polars...
A managed Distributed Polars compute cluster to ensure a single DataFrame API for all your needs.
pola.rs/posts/polars...
Polars Cloud; the distributed Cloud Architecture to run Polars anywhere
DataFrames for the new era
pola.rs
February 13, 2025 at 10:04 AM
This is amazing. I should really consider it.
Just like you'd use emr or any other managed compute platform.
Hopefully quality is there.
Just like you'd use emr or any other managed compute platform.
Hopefully quality is there.
I've pulled the deepseek model (thanks @ollama) and asked it a simple question (tried it with different temperature):
I’m in London and facing west, is Edinburgh to my left or my right?
And it failed to answer it.
o1 and o1-mini nailed it.
Yes, deepseek was faster, but wrong.
I’m in London and facing west, is Edinburgh to my left or my right?
And it failed to answer it.
o1 and o1-mini nailed it.
Yes, deepseek was faster, but wrong.
January 27, 2025 at 3:59 PM
I've pulled the deepseek model (thanks @ollama) and asked it a simple question (tried it with different temperature):
I’m in London and facing west, is Edinburgh to my left or my right?
And it failed to answer it.
o1 and o1-mini nailed it.
Yes, deepseek was faster, but wrong.
I’m in London and facing west, is Edinburgh to my left or my right?
And it failed to answer it.
o1 and o1-mini nailed it.
Yes, deepseek was faster, but wrong.
Writing today internal doc on evaluation of llm/rag evaluation libraries. Or should I use LLM to evaluate the output of it? And then pick some random (==42) metric to pick one ?
🤓
🤓
January 23, 2025 at 3:29 PM
Writing today internal doc on evaluation of llm/rag evaluation libraries. Or should I use LLM to evaluate the output of it? And then pick some random (==42) metric to pick one ?
🤓
🤓
Stargate name wasn't copyrighted? Hm, so how much of $500B will go to building coal energy plants?
January 22, 2025 at 8:53 AM
Stargate name wasn't copyrighted? Hm, so how much of $500B will go to building coal energy plants?
I don’t get the hype about cursor. “It’s the best IDE ever!”
No, it’s not. Debugging is a mess. Simple rename refactor is a slow op.
Code completion in tests is very nice, but go try IntelliJ.
No, it’s not. Debugging is a mess. Simple rename refactor is a slow op.
Code completion in tests is very nice, but go try IntelliJ.
January 22, 2025 at 4:41 AM
I don’t get the hype about cursor. “It’s the best IDE ever!”
No, it’s not. Debugging is a mess. Simple rename refactor is a slow op.
Code completion in tests is very nice, but go try IntelliJ.
No, it’s not. Debugging is a mess. Simple rename refactor is a slow op.
Code completion in tests is very nice, but go try IntelliJ.
TIL that today is the third day that MLK day happens on the same day as us president inauguration.
Both first times were during second terms inauguration of Bill Clinton and Barack Obama.
Odds of MLK happen on January 20 - 1/7.
Odds of second term - 1/2.
General odds of overlap - 1/14.
Both first times were during second terms inauguration of Bill Clinton and Barack Obama.
Odds of MLK happen on January 20 - 1/7.
Odds of second term - 1/2.
General odds of overlap - 1/14.
January 20, 2025 at 3:06 PM
TIL that today is the third day that MLK day happens on the same day as us president inauguration.
Both first times were during second terms inauguration of Bill Clinton and Barack Obama.
Odds of MLK happen on January 20 - 1/7.
Odds of second term - 1/2.
General odds of overlap - 1/14.
Both first times were during second terms inauguration of Bill Clinton and Barack Obama.
Odds of MLK happen on January 20 - 1/7.
Odds of second term - 1/2.
General odds of overlap - 1/14.
.yaml or .yml ?
Do you stick to only one option or don’t care?
Do you stick to only one option or don’t care?
January 9, 2025 at 11:05 AM
.yaml or .yml ?
Do you stick to only one option or don’t care?
Do you stick to only one option or don’t care?
Dear Houthis, your rockets are useless, but I’d love to sleep at night.
December 30, 2024 at 9:16 PM
Dear Houthis, your rockets are useless, but I’d love to sleep at night.
Reading on zero-shot vs few-shot prompting.
Internal voice: "If you have one shot or one opportunity to seize everything you ever wanted in one moment”.
On another thought: I’m wondering how lyrics writers use genai.
Internal voice: "If you have one shot or one opportunity to seize everything you ever wanted in one moment”.
On another thought: I’m wondering how lyrics writers use genai.
December 11, 2024 at 10:37 AM
Reading on zero-shot vs few-shot prompting.
Internal voice: "If you have one shot or one opportunity to seize everything you ever wanted in one moment”.
On another thought: I’m wondering how lyrics writers use genai.
Internal voice: "If you have one shot or one opportunity to seize everything you ever wanted in one moment”.
On another thought: I’m wondering how lyrics writers use genai.
I had 0 meetings today.
Zero context switches.
Omg, I’ve worked half through a feature that I thought would take at least 4 days.
Zero context switches.
Omg, I’ve worked half through a feature that I thought would take at least 4 days.
December 9, 2024 at 4:23 PM
I had 0 meetings today.
Zero context switches.
Omg, I’ve worked half through a feature that I thought would take at least 4 days.
Zero context switches.
Omg, I’ve worked half through a feature that I thought would take at least 4 days.
Couple weeks ago I’ve picked a Damascus station novel to read (was recommended by The Economist, great book by David McCloskey) and reading news coming from Syria feel like eerie dejavu.
December 8, 2024 at 8:51 AM
Couple weeks ago I’ve picked a Damascus station novel to read (was recommended by The Economist, great book by David McCloskey) and reading news coming from Syria feel like eerie dejavu.
For the first time in my career I’ve written couple Postgres functions, that actually replaced a microservice (and return json).
Yes, maybe it’s not fun long sql code, that isn’t easy to debug, but complexity reduction is huge.
Yes, maybe it’s not fun long sql code, that isn’t easy to debug, but complexity reduction is huge.
December 3, 2024 at 3:11 PM
For the first time in my career I’ve written couple Postgres functions, that actually replaced a microservice (and return json).
Yes, maybe it’s not fun long sql code, that isn’t easy to debug, but complexity reduction is huge.
Yes, maybe it’s not fun long sql code, that isn’t easy to debug, but complexity reduction is huge.
So if you accumulate 10PB of data annually in your datalake, that means that average rate is about 320 mbps, which is not a lot.
But 10PB is.
Kafka cluster will need couple brokers on commodity servers. Or Flink with couple task managers. (Yes of course depends on kind of applied transformations).
But 10PB is.
Kafka cluster will need couple brokers on commodity servers. Or Flink with couple task managers. (Yes of course depends on kind of applied transformations).
December 2, 2024 at 7:14 PM
So if you accumulate 10PB of data annually in your datalake, that means that average rate is about 320 mbps, which is not a lot.
But 10PB is.
Kafka cluster will need couple brokers on commodity servers. Or Flink with couple task managers. (Yes of course depends on kind of applied transformations).
But 10PB is.
Kafka cluster will need couple brokers on commodity servers. Or Flink with couple task managers. (Yes of course depends on kind of applied transformations).
I’m really happy that Polars exists.
Instead of building complex infra with Spark for day-to-day work for data analysis and research usage I can just use polars and stop worrying whether asset will fit the machine. And it’s super fast too !
Magic 👏
Instead of building complex infra with Spark for day-to-day work for data analysis and research usage I can just use polars and stop worrying whether asset will fit the machine. And it’s super fast too !
Magic 👏
November 26, 2024 at 4:38 AM
I’m really happy that Polars exists.
Instead of building complex infra with Spark for day-to-day work for data analysis and research usage I can just use polars and stop worrying whether asset will fit the machine. And it’s super fast too !
Magic 👏
Instead of building complex infra with Spark for day-to-day work for data analysis and research usage I can just use polars and stop worrying whether asset will fit the machine. And it’s super fast too !
Magic 👏
Me before kids: Oh, Black Friday, I’m gonna buy some gadget I might use in the future.
Now: it’s time to upgrade dishwasher.
Now: it’s time to upgrade dishwasher.
November 24, 2024 at 12:50 PM
Me before kids: Oh, Black Friday, I’m gonna buy some gadget I might use in the future.
Now: it’s time to upgrade dishwasher.
Now: it’s time to upgrade dishwasher.
My thoughts after the call with graph database vendor: “can Postgres do it?”.
Apparently yes (via extension ofc), but it’s still some serious learning curve and bit of a headache.
So far, looks like upcoming days will be very busy.
Apparently yes (via extension ofc), but it’s still some serious learning curve and bit of a headache.
So far, looks like upcoming days will be very busy.
November 13, 2024 at 9:08 PM
My thoughts after the call with graph database vendor: “can Postgres do it?”.
Apparently yes (via extension ofc), but it’s still some serious learning curve and bit of a headache.
So far, looks like upcoming days will be very busy.
Apparently yes (via extension ofc), but it’s still some serious learning curve and bit of a headache.
So far, looks like upcoming days will be very busy.