Akhila Yerukola
@akhilayerukola.bsky.social
PhD student at CMU LTI; Interested in pragmatics and cross-cultural understanding;
intern @ Allen Institute for AI |Prev: Senior Research Engineer @ Samsung Research America | Masters @ Stanford
https://akhila-yerukola.github.io/
intern @ Allen Institute for AI |Prev: Senior Research Engineer @ Samsung Research America | Masters @ Stanford
https://akhila-yerukola.github.io/
Also, this work began while I interned with Nanyun Peng and @skgabrie.bsky.social at sunny UCLA under the guidance of my advisor @maartensap.bsky.social ! Grateful for their mentorship throughout! 🙌
February 26, 2025 at 6:30 PM
Also, this work began while I interned with Nanyun Peng and @skgabrie.bsky.social at sunny UCLA under the guidance of my advisor @maartensap.bsky.social ! Grateful for their mentorship throughout! 🙌
Special thanks to: @sunipadev.bsky.social @841io.bsky.social , @nouhadziri.bsky.social , Jocelyn Shen, @shaily99.bsky.social , @simi97k.bsky.social ,
@vijaytarian.bsky.social , @apratapa.xyz for helpful discussions and feedback on this work!
@vijaytarian.bsky.social , @apratapa.xyz for helpful discussions and feedback on this work!
February 26, 2025 at 4:23 PM
Special thanks to: @sunipadev.bsky.social @841io.bsky.social , @nouhadziri.bsky.social , Jocelyn Shen, @shaily99.bsky.social , @simi97k.bsky.social ,
@vijaytarian.bsky.social , @apratapa.xyz for helpful discussions and feedback on this work!
@vijaytarian.bsky.social , @apratapa.xyz for helpful discussions and feedback on this work!
Huge shoutout to my amazing collaborators: @skgabrie.bsky.social, Nanyun (Violet) Peng, @maartensap.bsky.social!!
February 26, 2025 at 4:23 PM
Huge shoutout to my amazing collaborators: @skgabrie.bsky.social, Nanyun (Violet) Peng, @maartensap.bsky.social!!
🚀 I'm passionate about developing culturally contextual safety guardrails to make AI more sensitive and aware. If this work interests you, please feel free to reach out—I’d love to connect!
February 26, 2025 at 4:23 PM
🚀 I'm passionate about developing culturally contextual safety guardrails to make AI more sensitive and aware. If this work interests you, please feel free to reach out—I’d love to connect!
For more interesting findings, please check out our preprint 📜 arxiv.org/abs/2502.17710
Data 📚 github.com/Akhila-Yeruk...
Data 📚 github.com/Akhila-Yeruk...
Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures
Gestures are an integral part of non-verbal communication, with meanings that vary across cultures, and misinterpretations that can have serious social and diplomatic consequences. As AI systems becom...
arxiv.org
February 26, 2025 at 4:23 PM
For more interesting findings, please check out our preprint 📜 arxiv.org/abs/2502.17710
Data 📚 github.com/Akhila-Yeruk...
Data 📚 github.com/Akhila-Yeruk...
The cross-cultural safety risks aren’t theoretical – they’re already impacting several applications, such as:
✈️ AI-powered travel guides
🎭 AI-generated ad visuals
🤖 Automated content moderation
Culturally contextual safety guardrails are needed for AI systems!
✈️ AI-powered travel guides
🎭 AI-generated ad visuals
🤖 Automated content moderation
Culturally contextual safety guardrails are needed for AI systems!
February 26, 2025 at 4:23 PM
The cross-cultural safety risks aren’t theoretical – they’re already impacting several applications, such as:
✈️ AI-powered travel guides
🎭 AI-generated ad visuals
🤖 Automated content moderation
Culturally contextual safety guardrails are needed for AI systems!
✈️ AI-powered travel guides
🎭 AI-generated ad visuals
🤖 Automated content moderation
Culturally contextual safety guardrails are needed for AI systems!
🔬 Key Takeaway 🥉
All models—T2I, LLMs, and VLMs—exhibit US-centric biases, with higher accuracy in identifying offensive gestures in US contexts than in non-US ones (e.g., middle finger 🖕 in US vs UK)
All models—T2I, LLMs, and VLMs—exhibit US-centric biases, with higher accuracy in identifying offensive gestures in US contexts than in non-US ones (e.g., middle finger 🖕 in US vs UK)
February 26, 2025 at 4:23 PM
🔬 Key Takeaway 🥉
All models—T2I, LLMs, and VLMs—exhibit US-centric biases, with higher accuracy in identifying offensive gestures in US contexts than in non-US ones (e.g., middle finger 🖕 in US vs UK)
All models—T2I, LLMs, and VLMs—exhibit US-centric biases, with higher accuracy in identifying offensive gestures in US contexts than in non-US ones (e.g., middle finger 🖕 in US vs UK)
🔬 Key Takeaway 🥈
All models—T2I, LLMs, and VLMs—often default to US-centric interpretations of universal concepts (e.g., "good luck" → 🤞), overlooking the cultural variation in gestures used to express them
All models—T2I, LLMs, and VLMs—often default to US-centric interpretations of universal concepts (e.g., "good luck" → 🤞), overlooking the cultural variation in gestures used to express them
February 26, 2025 at 4:23 PM
🔬 Key Takeaway 🥈
All models—T2I, LLMs, and VLMs—often default to US-centric interpretations of universal concepts (e.g., "good luck" → 🤞), overlooking the cultural variation in gestures used to express them
All models—T2I, LLMs, and VLMs—often default to US-centric interpretations of universal concepts (e.g., "good luck" → 🤞), overlooking the cultural variation in gestures used to express them
🔬 Key Takeaway 🥇
(a) T2I models struggle to reject offensive gestures. LLMs tend to overflag gestures as offensive. VLMs show mixed results, with some performing near chance and others over-flagging
(b) Adding scene context doesnt affect LLMs but worsens T2I and VLM performance
(a) T2I models struggle to reject offensive gestures. LLMs tend to overflag gestures as offensive. VLMs show mixed results, with some performing near chance and others over-flagging
(b) Adding scene context doesnt affect LLMs but worsens T2I and VLM performance
February 26, 2025 at 4:23 PM
🔬 Key Takeaway 🥇
(a) T2I models struggle to reject offensive gestures. LLMs tend to overflag gestures as offensive. VLMs show mixed results, with some performing near chance and others over-flagging
(b) Adding scene context doesnt affect LLMs but worsens T2I and VLM performance
(a) T2I models struggle to reject offensive gestures. LLMs tend to overflag gestures as offensive. VLMs show mixed results, with some performing near chance and others over-flagging
(b) Adding scene context doesnt affect LLMs but worsens T2I and VLM performance
We assess how well T2I systems, LLMs, and VLMs understand cross-cultural gestures—revealing gaps in AI’s ability to navigate nonverbal communication safely. 💫
February 26, 2025 at 4:23 PM
We assess how well T2I systems, LLMs, and VLMs understand cross-cultural gestures—revealing gaps in AI’s ability to navigate nonverbal communication safely. 💫
🌍 Introducing MC-SIGNS — a testbed of 288 gesture-country pairs across 25 gestures & 85 countries, carefully annotated by cultural experts for:
1️⃣Offensiveness – how inappropriate a gesture is
2️⃣Confidence score
3️⃣Cultural meaning – associated gloss
4️⃣Contextual factors – when/where it may be risky
1️⃣Offensiveness – how inappropriate a gesture is
2️⃣Confidence score
3️⃣Cultural meaning – associated gloss
4️⃣Contextual factors – when/where it may be risky
February 26, 2025 at 4:23 PM
🌍 Introducing MC-SIGNS — a testbed of 288 gesture-country pairs across 25 gestures & 85 countries, carefully annotated by cultural experts for:
1️⃣Offensiveness – how inappropriate a gesture is
2️⃣Confidence score
3️⃣Cultural meaning – associated gloss
4️⃣Contextual factors – when/where it may be risky
1️⃣Offensiveness – how inappropriate a gesture is
2️⃣Confidence score
3️⃣Cultural meaning – associated gloss
4️⃣Contextual factors – when/where it may be risky
Why This Matters? 🤔
Humans can resolve such misunderstandings through social cues and context.
But AI? It generates STATIC content — ads 🎭, travel tips 🛫🏝️, and images 📸 — without accounting for the cross-cultural safety risks.
Humans can resolve such misunderstandings through social cues and context.
But AI? It generates STATIC content — ads 🎭, travel tips 🛫🏝️, and images 📸 — without accounting for the cross-cultural safety risks.
February 26, 2025 at 4:23 PM
Why This Matters? 🤔
Humans can resolve such misunderstandings through social cues and context.
But AI? It generates STATIC content — ads 🎭, travel tips 🛫🏝️, and images 📸 — without accounting for the cross-cultural safety risks.
Humans can resolve such misunderstandings through social cues and context.
But AI? It generates STATIC content — ads 🎭, travel tips 🛫🏝️, and images 📸 — without accounting for the cross-cultural safety risks.