This paper presents an analytical characterization of the long run behaviors learned by algorithms that interact repeatedly. I show that these behaviors correspond to equilibria that are stable points of a tractable differential equation. As a running example, I consider a repeated Cournot game of quantity competition, for which the stage game Nash equilibrium serves as non-collusive benchmark. I give necessary and sufficient conditions for this Nash equilibrium not to be learned. These conditions are requirements on the information algorithms use to determine their actions, and the stage game. When algorithms determine actions based only on the past period's price, the Nash equilibrium can be learned. However, conditioning actions on a richer type of information precludes the Nash equilibrium from being learned. This type of information allows for the existence of a collusive equilibrium that will be learned with positive probability.
[go to paper]

This paper provides asymptotic results for a class of actor-critic batch-reinforcement learning algorithms in the multi-agent setting. At each period, each agent faces an estimation problem (the critic, e.g. estimating Q-value function), and a policy updating problem. The estimation step is done by parametric function estimation based on a batch of past observations. I give sufficient conditions on the environment, growth rate of the batch-size and speed of their stepsizes, so that each agent's parametric function estimator is consistent in the following sense: For large t, the optimal empirical parameter vector is close to a true optimal parameter vector, depending on t only through the current period's policy profile.
[go to paper]

Spillover of economic outcomes often arises over multiple networks, and distinguishing their separate roles is important in empirical research. For example, the direction of spillover between two groups (such as banks and industrial sectors linked in a bipartite graph) has important economic implications, and a researcher may want to learn which direction appears prominent in data. For this, we need to have an empirical methodology that allows for both directions of spillover simultaneously. In this paper, we develop a dynamic linear panel model and asymptotic inference with large n and small T, where both directions of spillover are accommodated through multiple networks. Using the methodology developed here, we perform an empirical study of spillovers between bank weakness and zombie-firm congestion in industrial sectors, using firm-bank matched data from Spain between 2005 and 2012. Overall, we find that there is positive spillover in both directions between banks and sectors.
[go to paper]