For example, in the cited paper each "task" comes with "checkpoints"; real tasks are more ambiguous and usually have multiple solutions.

Perhaps some people would wishfully and incorrectly interpret this same result as "a quarter of the work could already be automated!!", without realizing the huge gap between those "tasks" (which only approximate things) and real tasks.

I hope so too! Especially interested in learning more about technical advancements that can enable this.