Same

Finally, huge shout out to our team — @kartikchandra.bsky.social, Katie Collins, Anne Harrington, Arnav Verma, Sean Anderson, Chris Kymn, and @judithfan.bsky.social
Everyone's been been hard at work for almost 6 months (!!) now, planning a truly one-of-a-kind workshop! :)

🎨 Last but not least, on visual expression (7/16, 1-2pm ET):
@aaronhertzmann.bsky.social, Sarah Schwettmann, Yael Vinker, and Cassidy Curtis

⚙️ On visual reasoning (7/3, 3-4pm ET):
Kelsey Allen, Gilbert Bernstein, Felix Binder, Daniel Ritchie, and @tomerullman.bsky.social

👁️ On visual perception (6/17, 10-11am ET):
Zachary Ferguson, Wenzel Jacob, Tzu-Mao Li, @vivianpaulun.bsky.social, and Ilker Yildirim

🗣️ On visual communication (6/12, 3-4pm ET): Gabriel Greenberg, @magrawala.bsky.social, Cindy Xiong, and Zoya Bylinskii

Hi Bluesky! Crossposting the announcement of a new summer workshop on the emerging interface between cognitive science 🧠 and computer graphics 🫖 spearheaded by @kartikchandra.bsky.social!

We're calling it: COGGRAPH! coggraph.github.io

June – July 2024, free & open to all!
🧶

When we took a closer look, we noticed that the distribution of guesses people generated when presented with human/CLIPasso sketches were most aligned for the most detailed sketches — but these distributions diverged dramatically for the sparser sketches. 9/11

Turns out CLIPasso sketches were about as recognizable to people as human-made ones at each level of detail! 8/11

In the meantime, we already have impressive generative AI systems – like CLIPasso (clipasso.github.io/clipasso/) – that can create sketches varying in visual abstraction. How similar are its sketches to the ones in our human sketch benchmark? 7/11

But! We found huge gaps in human-model ALIGNMENT for even the most performant models (like CLIP) — meaning there is still a long way to go to emulate *human-like* sketch understanding! 6/11

Some models did better than others, but basically every model we tested had an easier time recognizing the more detailed sketches (higher recog accuracy, less variable response distributions, more reasonable errors). Phew! 5/11

Second, a MODEL SUITE that is broadly representative of current SOTA vision algorithms. And third, an EVALUATION PROTOCOL to measure to what degree models achieve *human-level* recognition performance AND generate *human-like* response patterns when presented with a… well, sketchy looking lion. 4/11

First, a SKETCH DATASET spanning a wide range of concepts with systematic variation in their level of detail. We collected 90K human sketches of 128 visual concepts under different time constraints (4s-32s) + 8K CLIPasso (clipasso.github.io/clipasso/) sketches (4-32 strokes). 3/11

While current vision models achieve human-lvl performance on some visual benchmarks (e.g., object detection), it's less clear whether they grasp semantic ambiguity in sketches the way people do. To test human-model alignment on sketch understanding we needed *3 ingredients*— 2/11

Super excited to share that the 𝗦𝗘𝗩𝗔 dataset and benchmark will be presented at #NeurIPS2023!✏️🧠🤖

🌐 Website - seva-benchmark.github.io
📜 Paper - arxiv.org/abs/2312.03035
💻 Code + Materials - github.com/cogtoolslab/...

Thread below 🧵-

自分の足でTRY!