Making Friends on the Fly: Advances in Ad Hoc Teamwork

This book is devoted to the encounter and interaction of agents such as robots with other agents and describes how they cooperate with their previously unknown teammates, forming an Ad Hoc team. It presents a new algorithm, PLASTIC, that allows agents to quickly adapt to new teammates by reusing knowledge learned from previous teammates. PLASTIC is instantiated in both a model-based approach, PLASTIC-Model and a policy-based approach, PLASTIC-Policy. In addition to reusing knowledge learned from previous teammates, PLASTIC also allows users to provide expert-knowledge and can use transfer learning (such as the new Two Stage Transfer algorithm) to quickly create models of new teammates when it has some information about its new teammates. The effectiveness of the algorithm is demonstrated on three domains, ranging from multi-armed bandits to simulated robot soccer games.