How well do Multimodal LLMs consider visual information when creating plans to complete household activities? To answer this, we put a few multimodal LLMs on a pair of smart glasses and had participants try to solve cooking tasks while taking instructions from them.

For those of you attending #NeurIPS2024 in person: I'm from Vancouver and I made an extensive list of restaurants, bars, bookstores, etc., that I used to frequent when I still lived there. Enjoy!
dippedrusk.com/posts/2024-0...

Kicking off our TUM AI - Lecture Series tomorrow with none other than Jiaming Song, CSO @LumaLabsAI.

He'll be talking about "Dream Machine: Emergent Capabilities from Video Foundation Models".

Live stream: youtu.be/oilWwsXZamA
7pm GMT+1 / 10am PST (Mon Dec 2nd)

I've created a startepack on Generative Modeling: go.bsky.app/Hd9ykTw

Streaming Deep Reinforcement Learning Finally Works, by
M. Elsayed, G. Vasan, A. R. Mahmood, is one of those papers I wish I had written 😅

This paper seems to allow us to do RL with NNs as it should have always been done. Everyone should read it!

arxiv.org/abs/2410.14606

authors doing their best addressing reviewer’s concerns

youtu.be/FN2RM-CHkuI?...

Dear reviewers:

As you react/respond to the author rebuttal can you please articulate the answers to these questions in 1-2 sentences each?

1. Why not a lower score
2. Why not a higher score

This significantly helps bring everyone (authors/reviewers/AC/SAC) on the same page.

Test of Time Paper Awards are out! 2014 was a wonderful year with lots of amazing papers. That's why, we decided to highlight two papers: GANs (@ian-goodfellow.bsky.social et al.) and Seq2Seq (Sutskever et al.). Both papers will be presented in person 😍

Link: blog.neurips.cc/2024/11/27/a...

NeurIPS Test of Time Awards:

Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio

Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le

Very happy to announce that we could convince Andy Wood (staff.ucar.edu/users/andywood) to give a solicited talk in our session.

Small yet mighty! 💫

We are releasing SmolVLM: a new 2B small vision language made for on-device use, fine-tunable on consumer GPU, immensely memory efficient 🤠

We release three checkpoints under Apache 2.0: SmolVLM-Instruct, SmolVLM-Synthetic and SmolVLM-Base huggingface.co/collections/...

Reviewed for #ICLR?

Please take a moment to read authors' rebuttal, other reviews, and ask clarifying questions or request for further evidence that is still missing.

Many (junior) authors have put a ton of effort into this and may get discouraged by lack of engagement!

Time sink 😡

Hello BlueSky! Joao Henriques (joao.science) and I are hiring a fully funded PhD student (UK/international) for the FAIR-Oxford program. The student will spend 50% of their time @UniofOxford and 50% @MetaAI (FAIR) in London, while completing a DPhil (Oxford PhD). Deadline: 2nd of Dec AOE!!

This article really spoke to me; all the science I've enjoyed and that I thought came out well has been done with a colleague that I was talking to every day and almost every couple of hours

I have become a fan of the game-theoretic approaches to RLHF, so here are two more papers in that category! (with one more tomorrow 😅)

1. Self-Play Preference Optimization (SPO).

2. Direct Nash Optimization (DNO).

🧵 1/3.