wireless_dev
@wireless-dev.bsky.social
stealth (50K DAU) | previously ML @ google, competitive programmer
i think typescript is also one of the only languages where there can be two libraries with the same api shape, but different typings
there could theoretically be (much worse) versions of drizzle, aisdk, etc. with the same functions exposed but none of the typed goodness
there could theoretically be (much worse) versions of drizzle, aisdk, etc. with the same functions exposed but none of the typed goodness
January 23, 2025 at 8:26 PM
i think typescript is also one of the only languages where there can be two libraries with the same api shape, but different typings
there could theoretically be (much worse) versions of drizzle, aisdk, etc. with the same functions exposed but none of the typed goodness
there could theoretically be (much worse) versions of drizzle, aisdk, etc. with the same functions exposed but none of the typed goodness
side note: one approach which we'd completely overlooked was planetscale's sharding features. given our concerns on cost at the time, offloading was a more suitable fit for us. but we would be dumb not to combine offloading + sharding in the future. shows how much we still have to learn!
December 10, 2024 at 8:41 AM
side note: one approach which we'd completely overlooked was planetscale's sharding features. given our concerns on cost at the time, offloading was a more suitable fit for us. but we would be dumb not to combine offloading + sharding in the future. shows how much we still have to learn!
in hindsight, this is a little obvious, but it's the kind of thing that you may never realize without experience in building these systems before. it also made me realize how little you actually learn about scaling at big companies.
December 10, 2024 at 8:41 AM
in hindsight, this is a little obvious, but it's the kind of thing that you may never realize without experience in building these systems before. it also made me realize how little you actually learn about scaling at big companies.
with more granular offloading strategies of user objects, we found that we could cut our db size down dramatically (nearly 90%). we'd need a system to handle the process of offloading/restoring data, but that is a tiny amount of additional complexity for the amount of scalability we've unlocked.
December 10, 2024 at 8:41 AM
with more granular offloading strategies of user objects, we found that we could cut our db size down dramatically (nearly 90%). we'd need a system to handle the process of offloading/restoring data, but that is a tiny amount of additional complexity for the amount of scalability we've unlocked.
the reality is, when you really think about it, only a fraction of the data in your database is truly "active". this is especially true of b2c apps where 90% of user content is never touched again after a week (or even a day) of being created.
December 10, 2024 at 8:41 AM
the reality is, when you really think about it, only a fraction of the data in your database is truly "active". this is especially true of b2c apps where 90% of user content is never touched again after a week (or even a day) of being created.
we weren't making any stupid mistakes (using our DB for images/videos/huge blobs of text or data), so the idea of even more granular offloading never crossed our minds. relatedly, while we all come from big companies, problems like these were abstracted away to dedicated teams.
December 10, 2024 at 8:41 AM
we weren't making any stupid mistakes (using our DB for images/videos/huge blobs of text or data), so the idea of even more granular offloading never crossed our minds. relatedly, while we all come from big companies, problems like these were abstracted away to dedicated teams.
around the same time, i stumbled upon a message in the Replicache discord about limiting your database size to 100GB. that's when I started to think more about what we truly need to store in our database.
December 10, 2024 at 8:41 AM
around the same time, i stumbled upon a message in the Replicache discord about limiting your database size to 100GB. that's when I started to think more about what we truly need to store in our database.
essentially, we needed a way to unlock scalability without resorting to techniques that are only maintainable by companies hundreds of times larger than us.
December 10, 2024 at 8:41 AM
essentially, we needed a way to unlock scalability without resorting to techniques that are only maintainable by companies hundreds of times larger than us.
but as we were going deeper and deeper down the rabbit hole, in the back of our minds, we knew that this was just going to cause us even more trouble than what we'd started with.
December 10, 2024 at 8:41 AM
but as we were going deeper and deeper down the rabbit hole, in the back of our minds, we knew that this was just going to cause us even more trouble than what we'd started with.
our first instinct was to start researching how massive b2c apps like twitter or whatsapp are architected. we started drawing up complicated plans on migrating certain tables to Dynamo/Scylla, doing all sorts of denormalization, and so on.
December 10, 2024 at 8:41 AM
our first instinct was to start researching how massive b2c apps like twitter or whatsapp are architected. we started drawing up complicated plans on migrating certain tables to Dynamo/Scylla, doing all sorts of denormalization, and so on.
none of these is individually fatal, but together they eventually became too much to bear, and we started to look for solutions.
December 10, 2024 at 8:41 AM
none of these is individually fatal, but together they eventually became too much to bear, and we started to look for solutions.
- deploying db branch changes take 13+ hrs, if they even work at all
- backups take forever (1-2hrs+)
- restoring from backups takes forever
- costs for storage/backups skyrocket
- backups take forever (1-2hrs+)
- restoring from backups takes forever
- costs for storage/backups skyrocket
December 10, 2024 at 8:41 AM
- deploying db branch changes take 13+ hrs, if they even work at all
- backups take forever (1-2hrs+)
- restoring from backups takes forever
- costs for storage/backups skyrocket
- backups take forever (1-2hrs+)
- restoring from backups takes forever
- costs for storage/backups skyrocket
- at scale, random things become unusable/unavailable, e.g., Planetscale support told us to stop using Boost
- weekly mysql version upgrades start to cause a noticeable downtime blip, requiring special retry handling
- schema migrations become more complex operationally
December 10, 2024 at 8:41 AM
- at scale, random things become unusable/unavailable, e.g., Planetscale support told us to stop using Boost
- weekly mysql version upgrades start to cause a noticeable downtime blip, requiring special retry handling
- schema migrations become more complex operationally
well, we quickly started to realize that was a mistake. while our queries were blazing fast as ever (thanks to Planetscale), we ran into a number of other problems:
December 10, 2024 at 8:41 AM
well, we quickly started to realize that was a mistake. while our queries were blazing fast as ever (thanks to Planetscale), we ran into a number of other problems:
for a mid-sized company, this is just another day at the office; however, for a team of 3 and a product less than a year old, this can come with challenges.
at first, we didn't exercise much caution. we were using PlanetScale, which scales ~infinitely, so we figured, what's the harm?
at first, we didn't exercise much caution. we were using PlanetScale, which scales ~infinitely, so we figured, what's the harm?
December 10, 2024 at 8:41 AM
for a mid-sized company, this is just another day at the office; however, for a team of 3 and a product less than a year old, this can come with challenges.
at first, we didn't exercise much caution. we were using PlanetScale, which scales ~infinitely, so we figured, what's the harm?
at first, we didn't exercise much caution. we were using PlanetScale, which scales ~infinitely, so we figured, what's the harm?
One thing that kinda sucks is that animated JXL support is like another order of magnitude worse than still JXL support.
December 3, 2024 at 12:21 AM
One thing that kinda sucks is that animated JXL support is like another order of magnitude worse than still JXL support.
ah yea makes sense. was thinking in the context of an Expo app that is mobile + web.
December 3, 2024 at 12:20 AM
ah yea makes sense. was thinking in the context of an Expo app that is mobile + web.
Is there a good way to render them? Around a week ago was coincidentally looking to add JXL to a mobile app I'm working on but couldn't find anything viable. Best cross-platform option seems to be webview + WASM polyfill.
December 2, 2024 at 11:54 PM
Is there a good way to render them? Around a week ago was coincidentally looking to add JXL to a mobile app I'm working on but couldn't find anything viable. Best cross-platform option seems to be webview + WASM polyfill.