Excited to talk about our latest work, "AI as Humanity's Salieri," at Fireside Chat today at 7 PM PST! 🔥
app.ploutos.dev/streams/inno...
app.ploutos.dev/streams/inno...
December 22, 2024 at 11:33 PM
Excited to talk about our latest work, "AI as Humanity's Salieri," at Fireside Chat today at 7 PM PST! 🔥
app.ploutos.dev/streams/inno...
app.ploutos.dev/streams/inno...
Finally, the CREATIVITY INDEX proves to be a surprisingly effective criterion for zero-shot machine text detection, surpassing the strongest existing zero-shot system, DetectGPT, by 30.2%, and even outperforming the strongest supervised system, GhostBuster, in five out of six domains.
November 22, 2024 at 2:08 AM
Finally, the CREATIVITY INDEX proves to be a surprisingly effective criterion for zero-shot machine text detection, surpassing the strongest existing zero-shot system, DetectGPT, by 30.2%, and even outperforming the strongest supervised system, GhostBuster, in five out of six domains.
Furthermore, we explore creativity differences among various groups of humans. Despite in-group variance, famous authors of classic literature, like Hemingway and Dickens, exhibit the highest levels of creativity, consistent with their levels of renown.
November 22, 2024 at 2:08 AM
Furthermore, we explore creativity differences among various groups of humans. Despite in-group variance, famous authors of classic literature, like Hemingway and Dickens, exhibit the highest levels of creativity, consistent with their levels of renown.
Moreover, we found that RLHF dramatically reduces the CREATIVITY INDEX of LLMs, by an average of 30.1%. This reduction is more significant at the verbatim level than the semantic level, indicating that LLMs may have converged to certain linguistic style preferred by humans during alignment.
November 22, 2024 at 2:07 AM
Moreover, we found that RLHF dramatically reduces the CREATIVITY INDEX of LLMs, by an average of 30.1%. This reduction is more significant at the verbatim level than the semantic level, indicating that LLMs may have converged to certain linguistic style preferred by humans during alignment.
We found CREATIVITY INDEX of human authors—specifically professional writers and historical figures—is on average 66.2% higher than that of LLMs. This gap is consistent across various domains—novel snippets, modern poems, and speech transcripts—at both verbatim and semantic levels.
November 22, 2024 at 2:07 AM
We found CREATIVITY INDEX of human authors—specifically professional writers and historical figures—is on average 66.2% higher than that of LLMs. This gap is consistent across various domains—novel snippets, modern poems, and speech transcripts—at both verbatim and semantic levels.
To compute CREATIVITY INDEX efficiently, we introduce DJ SEARCH, a novel dynamic programming algorithm that can efficiently search for verbatim and near-verbatim matches of text snippets (i.e. n-grams) from a given document against a vast reference corpus in linear runtime.
November 22, 2024 at 2:06 AM
To compute CREATIVITY INDEX efficiently, we introduce DJ SEARCH, a novel dynamic programming algorithm that can efficiently search for verbatim and near-verbatim matches of text snippets (i.e. n-grams) from a given document against a vast reference corpus in linear runtime.
We define L-uniqueness for a text as the proportion of its words, outside of n-grams (n ≥ L), that appear in a vast reference corpus (e.g., RedPajama).
The CREATIVITY INDEX is then defined as the area under the L-uniqueness curve across a range of minimum n-gram lengths L.
The CREATIVITY INDEX is then defined as the area under the L-uniqueness curve across a range of minimum n-gram lengths L.
November 22, 2024 at 2:05 AM
We define L-uniqueness for a text as the proportion of its words, outside of n-grams (n ≥ L), that appear in a vast reference corpus (e.g., RedPajama).
The CREATIVITY INDEX is then defined as the area under the L-uniqueness curve across a range of minimum n-gram lengths L.
The CREATIVITY INDEX is then defined as the area under the L-uniqueness curve across a range of minimum n-gram lengths L.
TLDR: We found the seemingly remarkable creativity of LLMs 🤖can be attributable in large part to the creativity of human-written texts on the web. In contrast, works by distinguished human authors 👩🎓cannot be easily replicated by merely assembling snippets from other works.
November 22, 2024 at 2:04 AM
TLDR: We found the seemingly remarkable creativity of LLMs 🤖can be attributable in large part to the creativity of human-written texts on the web. In contrast, works by distinguished human authors 👩🎓cannot be easily replicated by merely assembling snippets from other works.
Are LLMs 🤖 as creative as humans 👩🎓? Not quite!
Introducing CREATIVITY INDEX: a metric that quantifies the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Spoiler: professional human writers like Hemingway are still far more creative than LLMs! 😲
Introducing CREATIVITY INDEX: a metric that quantifies the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Spoiler: professional human writers like Hemingway are still far more creative than LLMs! 😲
November 22, 2024 at 2:00 AM
Are LLMs 🤖 as creative as humans 👩🎓? Not quite!
Introducing CREATIVITY INDEX: a metric that quantifies the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Spoiler: professional human writers like Hemingway are still far more creative than LLMs! 😲
Introducing CREATIVITY INDEX: a metric that quantifies the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Spoiler: professional human writers like Hemingway are still far more creative than LLMs! 😲