i also think learned optimization may end up being far more of a bottleneck in the long term compared to the architectural structure of neural networks in terms of sample efficient learning
arxiv.org/abs/1606.04474
arxiv.org/abs/1606.04474
November 27, 2024 at 11:49 PM
i also think learned optimization may end up being far more of a bottleneck in the long term compared to the architectural structure of neural networks in terms of sample efficient learning
arxiv.org/abs/1606.04474
arxiv.org/abs/1606.04474
clearly we can observe, that the deeper the network is, the better the heuristics that form in the network when it comes to generalizing to "like data".
so blanketly describing the solutions that dnns make as "poorly generalizable" is a little bizarre to me tbh
so blanketly describing the solutions that dnns make as "poorly generalizable" is a little bizarre to me tbh
November 27, 2024 at 11:37 PM
clearly we can observe, that the deeper the network is, the better the heuristics that form in the network when it comes to generalizing to "like data".
so blanketly describing the solutions that dnns make as "poorly generalizable" is a little bizarre to me tbh
so blanketly describing the solutions that dnns make as "poorly generalizable" is a little bizarre to me tbh
i mean, how can you coherently assess the formation of approximated functions that don't actually exist in the data as a form of "memorization" if the internal heuristics of the network look nothing like the data but are formed by an attempt to match it?
November 27, 2024 at 11:28 PM
i mean, how can you coherently assess the formation of approximated functions that don't actually exist in the data as a form of "memorization" if the internal heuristics of the network look nothing like the data but are formed by an attempt to match it?
