In hindsight it was obvious. A CNN-to-Regression model that takes two images and produces a 4-vector indicating their relative alignment has an extreme imbalance in the input vs output feature counts. Gradient descent initially pushes most of the hidden outputs to zero!

#ai

Just some screenshots of some experiments into improving observability of ML training - gradients across layers and the like.....more on this later.
#MLSky

Running some experiments to see how the fraction of active units in a network affects learning rates (ie: have a non-zero output activation), and the first result is very promising - looks like a clear indication that low activity rates badly affects learning rates.
#MLSky

I’m seeking feedback on a draft paper on a theory of consciousness: "Meta-management as a clearer explanation of conscious function". Reply here or tag me with your thoughts.
PDF: bit.ly/46r8DZB
Word: bit.ly/40RI4fd

#CompConsc #MLSky #compneuro #neuroscience