Online and Offline Learning of Player Objectives from Partial Observations in Dynamic Games

@inproceedings{peters2023ijrr,
    title     = {Online and Offline Learning of Player Objectives from Partial Observations in Dynamic Games
},
    author    = {Peters, Lasse and Rubies-Royo, Vicenc and Tomlin, Claire J. and Ferranti, Laura and Alonso-Mora, Javier and Stachniss, Cyrill and Fridovich-Keil, David },
    booktitle = {Intl.~Journal~of Robotics Research (IJRR)},
    year      = {2023},
    url       = {https://journals.sagepub.com/doi/pdf/10.1177/02783649231182453}
}

News

Abstract

Robots deployed to the real world must be able to interact with other agents in their environment. Dynamic game theory provides a powerful mathematical framework for modeling scenarios in which agents have individual objectives and interactions evolve over time. However, a key limitation of such techniques is that they require a-priori knowledge of all players’ objectives. In this work, we address this issue by proposing a novel method for learning players’ objectives in continuous dynamic games from noise-corrupted, partial state observations. Our approach learns objectives by coupling the estimation of unknown cost parameters of each player with inference of unobserved states and inputs through Nash equilibrium constraints. By coupling past state estimates with future state predictions, our approach is amenable to simultaneous online learning and prediction in receding horizon fashion. We demonstrate our method in several simulated traffic scenarios in which we recover players’ preferences for, e.g., desired travel speed and collision-avoidance behavior. Results show that our method reliably estimates game-theoretic models from noise-corrupted data that closely matches ground-truth objectives, consistently outperforming state-of-the-art approaches.

Code

The code for this project can be found at https://github.com/PRBonn/PartiallyObservedInverseGames.jl