Bao Pham
LightNews LightNews
baopham.bsky.social
Bao Pham
@baopham.bsky.social
26 followers 17 following 4 posts
PhD Student at RPI. Interested in Hopfield or Associative Memory models and Energy-based models.
Posts Replies Media Videos
Pinned
baopham.bsky.social
Bao Pham @baopham.bsky.social · Dec 5
Diffusion models create beautiful novel images, but they can also memorize samples from the training set. How does this blending of features allow creating novel patterns? Our new work in Sci4DL workshop #neurips2024 shows that diffusion models behave like Dense Associative Memory networks.
On the left figure, it showcases the behavior of Hopfield models. Given a query (the initial point of energy descent), a Hopfield model will retrieve the closest memory (local minimum) to that query such that it minimizes the energy function. A perfect Hopfield model is able to store patterns in distinct minima (or buckets). In contrast, the right figure illustrates a bad Associative Memory system, where stored patterns share a distinctive bucket. This enables the creation of spurious patterns, which appear like mixture of stored patterns. Spurious patterns will have lower energy than the memories due to this overlapping.
Reposted by Bao Pham
krotov.bsky.social
I am excited to announce the call for papers for the New Frontiers in Associative Memories workshop at ICLR 2025. New architectures and algorithms, memory-augmented LLMs, energy-based models, Hopfield nets, AM and diffusion, and many other topics.

Website: nfam.vizhub.ai

@iclr-conf.bsky.social
January 14, 2025 at 4:56 PM
Reposted by Bao Pham
krotov.bsky.social
Most of the work on Dense Associative Memory (DenseAM) thus far has focused on the regime when the amount of data (number of memories) is below the critical memory storage capacity. We are beginning to explore the opposite limit, when the data is large.
baopham.bsky.social Bao Pham @baopham.bsky.social · Dec 5
Diffusion models create beautiful novel images, but they can also memorize samples from the training set. How does this blending of features allow creating novel patterns? Our new work in Sci4DL workshop #neurips2024 shows that diffusion models behave like Dense Associative Memory networks.
On the left figure, it showcases the behavior of Hopfield models. Given a query (the initial point of energy descent), a Hopfield model will retrieve the closest memory (local minimum) to that query such that it minimizes the energy function. A perfect Hopfield model is able to store patterns in distinct minima (or buckets). In contrast, the right figure illustrates a bad Associative Memory system, where stored patterns share a distinctive bucket. This enables the creation of spurious patterns, which appear like mixture of stored patterns. Spurious patterns will have lower energy than the memories due to this overlapping.
December 5, 2024 at 6:19 PM
Reposted by Bao Pham
baopham.bsky.social
Diffusion models create beautiful novel images, but they can also memorize samples from the training set. How does this blending of features allow creating novel patterns? Our new work in Sci4DL workshop #neurips2024 shows that diffusion models behave like Dense Associative Memory networks.
On the left figure, it showcases the behavior of Hopfield models. Given a query (the initial point of energy descent), a Hopfield model will retrieve the closest memory (local minimum) to that query such that it minimizes the energy function. A perfect Hopfield model is able to store patterns in distinct minima (or buckets). In contrast, the right figure illustrates a bad Associative Memory system, where stored patterns share a distinctive bucket. This enables the creation of spurious patterns, which appear like mixture of stored patterns. Spurious patterns will have lower energy than the memories due to this overlapping.
December 5, 2024 at 5:29 PM
baopham.bsky.social
The work is done in collaboration with Gabriel Raya, Matteo Negri, Mohammed J. Zaki, @lucamb.bsky.social , @krotov.bsky.social

Lastly, join us at Sci4DL workshop at #NeurIPS2024 to learn more!

We will be giving an oral presentation there!
December 5, 2024 at 5:29 PM
baopham.bsky.social
This work enables a positive perspective of spurious patterns. Unlike their usual perception in Associative Memory, such patterns play a role in signaling generalization in deep generative models, like diffusion models.

Here is a link to the paper: openreview.net/pdf?id=zVMMa....
openreview.net
December 5, 2024 at 5:29 PM
baopham.bsky.social
In the low training data regime (number of memories), diffusion models memorize. As the data size increases, spurious states emerge, signaling the blending of stored features into new combinations which enables generalization. This is how such models create novel outputs in the high data regime.
This figure depicts some hand-selected examples of memorized, spurious, and generalized patterns in which our diffusion models generate, across various training data sizes. We show a target image along with its top-5 nearest neighbors from training set (top row) and synthetic set (bottom row). Memorized patterns are typically generated at low data sizes. Meanwhile, when the data size is large enough, but not substantially large enough for diffusion models to approximate the data distribution well, spurious patterns are typically generated from our models.
December 5, 2024 at 5:29 PM
baopham.bsky.social
Diffusion models create beautiful novel images, but they can also memorize samples from the training set. How does this blending of features allow creating novel patterns? Our new work in Sci4DL workshop #neurips2024 shows that diffusion models behave like Dense Associative Memory networks.
On the left figure, it showcases the behavior of Hopfield models. Given a query (the initial point of energy descent), a Hopfield model will retrieve the closest memory (local minimum) to that query such that it minimizes the energy function. A perfect Hopfield model is able to store patterns in distinct minima (or buckets). In contrast, the right figure illustrates a bad Associative Memory system, where stored patterns share a distinctive bucket. This enables the creation of spurious patterns, which appear like mixture of stored patterns. Spurious patterns will have lower energy than the memories due to this overlapping.
December 5, 2024 at 5:29 PM