[jimfleming]

Hopefully this leads to better temporal priors for machine learning models. Sequence encoding frequently uses position coding[0] but the typical approaches (e.g. low frequency sin waves) are fairly naive. In reinforcement learning we have (somewhat obvious) works that show that providing explicit time representations help[1]. Simply throwing a recurrent model at a sequence may pick up simple ordering but improving the temporal representation would improve data efficiency.

[0] https://arxiv.org/abs/1706.03762

[1] https://arxiv.org/abs/1712.00378