LightNews LightNews
lee-messi.bsky.social
@lee-messi.bsky.social
16 followers 23 following 9 posts
Posts Replies Media Videos
lee-messi.bsky.social
We found that o3-mini requires more tokens when processing association-incompatible than association-compatible information in 9 of 10 RM-IATs, similar to how humans take more time to pair groups with attributes that don't match established associations.
Bar chart showing reasoning token counts comparing association-compatible (dark green bars) versus association-incompatible (orange bars) processing across 10 different RM-IATs. The chart is organized in a 2×5 grid, with each panel representing a different RM-IAT. In all tests except one (i.e., Mental/Physical Diseases + Temporary/Permanent), the orange bars (association-incompatible) are consistently taller than the blue bars (association-compatible). This visually demonstrates that the model requires more computational tokens when processing association-incompatible pairings than association-compatible ones.
March 17, 2025 at 2:23 PM
lee-messi.bsky.social
In the RM-IAT, o3-mini is instructed to categorize group words (e.g., he, she) in ways that are either compatible (e.g., men-career, women-family) or incompatible (e.g., men-family, women-career) with established associations.
Visualization illustrating the Reasoning Model Implicit Association Test (RM-IAT) methodology in four steps. Step 1 shows the presentation of stimuli with group categories (e.g., men and women) and attribute categories (e.g., career and family). Step 2 demonstrates two conditions: Association-Compatible (men related to career, women to family) and Association-Incompatible (men related to family, women to career). Step 3 shows identical writing prompts asking to categorize "Steve" in both conditions. Step 4 compares reasoning tokens used: 64 tokens for the compatible association versus 192 tokens for the incompatible association, showing the model requires significantly more computational effort when processing association-incompatible pairings than association-compatible ones.
March 17, 2025 at 2:23 PM
lee-messi.bsky.social
In a new paper w/ @calvinklai.bsky.social, I find that OpenAI’s latest reasoning model (o3-mini) exhibits implicit bias-like patterns. What’s exciting about reasoning models is the ability to unpack bias in how models *process* information, rather than just seeing bias in *outputs*. (1/10):
Academic paper header and abstract for "Implicit Bias-Like Patterns in Reasoning Models" by authors Messi H.J. Lee (Washington University in St. Louis) and Calvin K. Lai (Rutgers University), dated March 17, 2025. The abstract explains research on implicit bias-like patterns in AI reasoning models, introducing a method called the Reasoning Model Implicit Association Test (RM-IAT). Their findings show that reasoning models require more computational tokens when processing association-incompatible information versus association-compatible information, suggesting AI systems exhibit processing patterns analogous to human implicit bias.
March 17, 2025 at 2:23 PM