Aron Vallinder
LightNews LightNews
aronvallinder.bsky.social
Aron Vallinder
@aronvallinder.bsky.social
230 followers 370 following 19 posts
Posts Replies Media Videos
aronvallinder.bsky.social
We also find substantial variation in behavior across different runs of the same model, suggesting a sensitive dependence on initial conditions.
We plot the average final resources (y-axis) per generation (x-axis) for all five individual runs of each model. Note the different 𝑦 -axis scales. For Claude 3.5 Sonnet, average final resources vary substantially across runs, especially in later generations. All five runs of GPT-4o show average final resources declining across generations (although in absolute terms the change is tiny). Gemini 1.5 Flash behavior also varies substantially across runs, with several runs showing promising increases before a “cooperation crash”.
December 16, 2024 at 9:24 AM
aronvallinder.bsky.social
We find substantial divergence in the evolution of cooperation across the models examined, as seen here in the average final scores after each generation.
We plot the average final resources across all agents (y-axis) per generation (x-axis) for three different models (Claude 3.5 Sonnet, Gemini 1.5 Flash, GPT-4o). Each curve averages 5 runs with distinct random seeds for the language models, and the standard error of the mean is shown by shading. There is reliable cultural evolution of cooperation across generations for Claude 3.5 Sonnet but not for Gemini 1.5 Flash or GPT-4o with our prompting strategy.
December 16, 2024 at 9:24 AM