Cool setup

Also, I'll be presenting this work at ICLR next month, please do come by!

Our benchmark code is already available for testing out new algorithms and I will be sharing additional instructions on using our code in the coming days. Stay tuned. I look forward to engaging and collaborating with anyone interested in advancing this new area of research! 🙂

There are numerous promising avenues for further exploration, particularly in adapting techniques and insights from single-objective RL generalization research to tackle this harder problem setting!

Ultimately, a priori scalarization of rewards in single-objective RL limits the agent's flexibility to adapt its behavior to environment changes and objective tradeoffs. Developing agents capable of generalizing across multiple environments AND objectives will become a crucial research direction.

Our baseline evaluations of current MORL algorithms uncover 2 key insights:

1) Current MORL algorithms struggle with generalization.
2) However, MORL demonstrate greater potential for learning adaptable behaviors for generalization compared to single-objective.

We also introduce a benchmark featuring diverse multi-objective domains with parameterized environment configurations to facilitate studies in this area.

Despite its importance, the intersection of generalization and multi-objectivity remains a significant gap in RL literature.

In this paper, we formalize generalization in Multi-Objective Reinforcement Learning (MORL) and how it can be evaluated.

Consider an autonomous vehicle, which must not only generalize across varied environmental conditions—different weather patterns, lighting, and road surfaces—but also learn optimal trade-offs between competing objectives such as travel time, passenger's comfort, and safety.

Real-world sequential decision-making tasks often involves balancing trade-offs among conflicting objectives and generalizing across diverse environments.