It achieves this by reweighting samples according to the likelihood of state-action pairs under the agent’s state representation, effectively breaking the spurious correlations introduced by the policy.

Here, we show that the advantage function not only reduces the variance of gradient estimates but also helps mitigate the effects of policy confounding.

This paper builds on our work published last year at RLC, where we showed that agents can develop policies that exploit spurious correlations induced by their own policies, a phenomenon we call policy confounding.

🤔🤷
x.com/SuauMiguel/s...