The universal approximation theorem states that a neural network of a specific structure can approximate a continuous function to any accuracy. While that means deeper neural nets are not needed in theory, later experiments showed practical benefits of increasing layers vs increasing hidden size.

so this is what being nerd-sniped feels like...

"ups the contrast" - small absolute differences are even smaller after squaring, while differences close to 1 remain close to 1.

This input data approach falls under the general body of data augmentation methods.

KL-Divergence isolates the extra information needed to encode those P-based messages as a result of wrongly assuming distribution Q.

Common sense reasoning about uncertainty means treating Bayesian probabilities as frequentist probabilities.