Huzzah! I figured out how to preserve the maximum area when smooshing the polygon. The answer is just to make corners that are parallel with the incoming plane. I've worked out all the edge cases... Now I just need to figure out if this algorithm is actually useful for its intended purpose.

Wacky geometry experiment: concocting a thing called a "DEAP Constraint" for fast, stable collisions when a game character is squeezing through tight spaces. It smooshes down an "equi-angular" polygon based on distance from each wall. But the smooshing needs to preserve as much area as possible...