Paging @cosmicskeptic.bsky.social . The wine glass is now full ! www.reddit.com/r/singularit...

Context: www.youtube.com/watch?v=160F...

Many more cool examples that people are finding in this reddit thread (including visual story generation, funky editing and style changes, recontexualization and more !):

www.reddit.com/r/singularit...

An example of reasoning in 'pixel space':

Notably we pass the 'room without an elephant' test (medium.com/@avanib28264...)

Image quality is not quite as high as our SOTA Imagen 3 model (see previous post :) ) but the ability to do reasoning in a combination of text and pixel space unlocks some amazing new capabilities like interleaved generation of text and images and just jamming crazy creative ideas with Gemini.

This is such amazing work by @abeirami.bsky.social and collaborators. A deep investigation of a simple and practically important idea. Highly relevant to our own work and anywhere else RL is used for Gen AI.

Lol....u'll always be my #1 Nathan. But your post did remind me that some collaborators of mine at Google did some research on content ecosystems :https://arxiv.org/abs/2309.06375

For example, I've been really enjoying following this guys math channel : youtube.com/@oceansofmat... not as slick as 3b1b but still quite informative.

We even show you can do this without a specialized heatmap model if you have a good classifier for the badness you want to eliminate by fine-tuning. Simply use a pixel attribution technique like GRADCAM to generate the heatmap !

Surprisingly effective. The problematic parts are changed but everything else remains the same in the fine-tuned model. This is different from an editing model, where 2 rounds of inference are needed to fix the problematic parts.

Then you fine-tune using a combination of DRAFT (arxiv.org/html/2309.17...) and our custom region-aware fine-tuning objective.

you generate a heatmap highlighting the problematic region (e.g. using our previous work on Rich Human Feedback for T2I): arxiv.org/pdf/2312.10240

The idea is simple. If the image from the base model has a region that's (say) NSFW: