basque neuron
@neurobasque.bsky.social
PhD Student in NeuroAI.
Researcher part time, beer full time.
Researcher part time, beer full time.
I was currently working following the work from Moraitis (arxiv.org/abs/2209.11883), since it appears to be the one with best results. I will check further into BTSP. My only doubt is that all papers i've seen on it are from the hippocampus, and it may not be appropiate for the visual system.
Hebbian Deep Learning Without Feedback
Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approxi...
arxiv.org
November 28, 2024 at 11:59 PM
I was currently working following the work from Moraitis (arxiv.org/abs/2209.11883), since it appears to be the one with best results. I will check further into BTSP. My only doubt is that all papers i've seen on it are from the hippocampus, and it may not be appropiate for the visual system.
yeah, my bad, I was informally referring to using "torch.SGD", so I rather meant that using a Hebbian rule with backprop on top is just backprop with more steps, my bad.
I totally agree that gradient descent naturally occurs during learning.
I totally agree that gradient descent naturally occurs during learning.
November 28, 2024 at 11:53 PM
yeah, my bad, I was informally referring to using "torch.SGD", so I rather meant that using a Hebbian rule with backprop on top is just backprop with more steps, my bad.
I totally agree that gradient descent naturally occurs during learning.
I totally agree that gradient descent naturally occurs during learning.
Mmm, I have to disagree. Hebbian methods have already been used to train networks to achieve 80% accuracy on datasets like CIFAR10 even without all the neurogeometry that our vision system has. We are missing a lot of things, but I doubt SGD is one.
November 28, 2024 at 9:06 PM
Mmm, I have to disagree. Hebbian methods have already been used to train networks to achieve 80% accuracy on datasets like CIFAR10 even without all the neurogeometry that our vision system has. We are missing a lot of things, but I doubt SGD is one.
They totally shouldn't
Using SGD with some extra Hebbian step does not really give insights into how Hebbian mechanism work
Using SGD with some extra Hebbian step does not really give insights into how Hebbian mechanism work
November 28, 2024 at 8:16 PM
They totally shouldn't
Using SGD with some extra Hebbian step does not really give insights into how Hebbian mechanism work
Using SGD with some extra Hebbian step does not really give insights into how Hebbian mechanism work