Yes. Every centred distribution in high dimension ends up being a hypersphere, most with thicker skin than a multi variate gaussian. That includes the latent of autoencoder and even images (more easily seen if normalised), assuming one wants to get some centred latent even for reg AE.

I know, but in high dimension the latent (and data) is on a hypersphere and drawing a hyperplan tangent to a 2D smooth manifold is not what happens. I don't know how to represent it, or imagine it, but this simplification is misleading I reckon without proper warning.

The bottom figure makes sense only if the data are 3D on a 2D manifold, and that the AE latent is 2D. In which case an AE is overkill (might as well use regression). AE works in high dimension (data and latent), in which case the bottom figure is wrong, or at least very misleading.

Finally, great to see chatGPT produce something useful.

Great post

I agree. The definition I like is “ability to solve a problem never seen before”

The image version is actually useful because the zooming on the gif does not work well.