Jason Nochlin
@nochlin.com
Just wrote a blog post that covers two of my favorite, underutilized software engineering topics: bloom filter variants and using mathematical insights to optimize data structures.
The post summarizes a 2012 paper "The Variable-Increment Counting Bloom Filter"
nochlin.com/blog/variabl...
The post summarizes a 2012 paper "The Variable-Increment Counting Bloom Filter"
nochlin.com/blog/variabl...
January 10, 2025 at 8:25 PM
Just wrote a blog post that covers two of my favorite, underutilized software engineering topics: bloom filter variants and using mathematical insights to optimize data structures.
The post summarizes a 2012 paper "The Variable-Increment Counting Bloom Filter"
nochlin.com/blog/variabl...
The post summarizes a 2012 paper "The Variable-Increment Counting Bloom Filter"
nochlin.com/blog/variabl...
Here's an example of a simple pipeline configured with Dagster
![from dagster import job, op, Definitions, SkipReason, schedule
from datetime import datetime
import requests
@op(retry_policy={
"max_retries": 3,
"delay": 60 # 1 minute between retries
})
def check_api_state():
response = requests.get("https://api.example.com/status")
data = response.json()
# Return None if conditions aren't met, which will skip downstream ops
if data["status"] != "READY" or data["queue_size"] > 100:
return None
return data
@op
def process_data(context, api_state):
# This won't run if api_state is None
if not api_state:
context.log.info("Conditions not met, skipping processing")
return
# Your processing logic here
context.log.info(f"Processing data with state: {api_state}")
@job
def conditional_job():
state = check_api_state()
process_data(state)
@schedule(
job=conditional_job,
cron_schedule="0 * * * *" # Run hourly
)
def hourly_schedule():
return {}
defs = Definitions(
jobs=[conditional_job],
schedules=[hourly_schedule]
)](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:zxykpb36liefxnp4bmax4ro6/bafkreifdeqkkjrmlic66yodruhneeuwj3hzpyzwizostb7ff65yj47k2ji@jpeg)
December 24, 2024 at 9:25 PM
Here's an example of a simple pipeline configured with Dagster
1. Quickly prototype data transformation in Python using DuckDB + Python Functions
2. Use LLM to rewrite Python Functions to C
3. Enjoy 200x performance improvement 🚀
2. Use LLM to rewrite Python Functions to C
3. Enjoy 200x performance improvement 🚀
November 30, 2024 at 5:33 PM
1. Quickly prototype data transformation in Python using DuckDB + Python Functions
2. Use LLM to rewrite Python Functions to C
3. Enjoy 200x performance improvement 🚀
2. Use LLM to rewrite Python Functions to C
3. Enjoy 200x performance improvement 🚀
@joshuawood.net thanks for the awesome sticker! Goes great with my jiu jitsu griffin 🔥
October 21, 2024 at 8:35 PM
@joshuawood.net thanks for the awesome sticker! Goes great with my jiu jitsu griffin 🔥