At the opening ceremony in Singapore they said that the city is still unclear.

Agreed that it's important to consider depth in addition to work. But for work to be negligible, the hardware would need to be infinitely scalable with input size because the asymptotic runtime is O(work/workers + depth). I doubt that this will change for future hardware due to physical limitations.

You could have a look at litmaps. They allow to explore new papers from a set of starting nodes. Also nice ways of sorting in time, relevancy, clustering etc.

It's not fake:
bsky.app/profile/yann...

Congrats Anastasia!!

You can use @bookmarks.bluecanary.dev to send messages to or report feature which are stored in a feed of your bookmarks

I also wonder how one would go about normalizing the sum in a variable sequence length mode.. How are you handling that at the moment for video? One potential solution could be a geometric decay on top of the mask, but that would lead to somewhat of an SSMs formulation? Interesting food for thought!

Interesting idea! This reminds me of the Symmetric Power Transformer (manifestai.com/articles/symmetric-power-transformers), but in your case the values are subject to the polynomial kernel instead of queries/keys while the selection resembles a Gated Linear Unit which is used in many backbones:)