Don't forget the cut on every marketplace transaction, even if it's to take two cents out of every three cents transaction.

(It's roughly a third of the lowest value mentioned in this thread so far)

I think I won't disclose how much I'm paying for a three letter domain (@tny.im), should they realize they're undercharging.

From what I've read and observed on a couple PDS, if you do that it won't take long for bots to begin creating accounts en masse in your PDS. Most PDS intended for public use require additional verification methods (e.g. email) for that reason.

on responding to security/compliance concerns more so than any technology- or implementation-specific concerns.

Because if it were something more fundamental about how LLMs work, the isolation would not be at the organization level but at a smaller level, as users probably wouldn't want "crosstalk" between unrelated requests within each organization, either. That passage in the docs seems especially focused

dealing with keys and/or values that could contain sensitive information or which could help disclose sensitive information when coupled with one or two other side-channels.

It probably isn't "bad," beyond protecting against the more "in general" bugs and side-channels. It makes it easier to reason about security and data privacy compliance. What I mean is that the reason they're doing that is probably not LLM specific, just something one would do in general when

Then you're relying on specifics of how the keys and values are used to trust that nothing will go wrong, assuming that their use case will remain constant forever, and that there is no hidden detail that could "get you", and that's not how defense in depth works...

it's easier to ensure nothing weird like that ever happens by simply not sharing the cache across tenants.

But if you have a bug in that derivation process that caused the wrong values to be pulled every now and then, that will surely influence the response in an incorrect way, and in a way which could directly or indirectly leak information about other requests being made. So to be on the safe side,

tenant, or make it so that the tenant ID is always part of the cache key no matter what, you prevent not just the timing attacks but also information leaking due to the cache being accessed with keys that don't encapsulate all of the information they should have.

process to derive a key from the prompt is bugged, e.g. someone forgot to take into account some setting like the system prompt or the temperature, and now the keys being used to access the cache are the same even though the requests are different. If you use an actually separate cache structure per

additional data that influences the value and which you forgot to take into account as part of the key, and now the cache is responding with values that aren't actually the correct ones for the request being made, and which could be polluted with sensitive data from other tenants. Or maybe that

Consider a cache that's implemented with a dictionary/map. To retrieve a value from a cache you usually need a key, which in this case would be the prompt (or something derived from it), to identify the value to retrieve (the value would be the response to the prompt). But there could be some

That's usually the main reason to isolate caches per tenant. It also makes it more difficult for data to accidentally leak between tenants (e.g. because cache keys could turn out not to be as unique as the developer assumed...). It could also naturally arise as a side-effect of sharding.

need a gta style game where the plot is keeping gta6 from being finished and released by any means necessary

And, once again, a good time to bring this back from the archive:

(via www.gq-magazine.co.uk/article/red-... - dated October 2018. Dan Houser left Rockstar in 2020)

You made me double-check my open programs, even if I haven't used slsk in ages...