[bravura]

bigvgan paper (https://arxiv.org/pdf/2206.04658.pdf)

"We introduce a proper inductive bias of periodicity to the generator by applying a recently proposed periodic activation called Snake function (Liu et al., 2020), defined as fα(x) = x + 1 α sin2 (αx), where α is a trainable parameter that controls the frequency of the periodic component of the signal and larger α gives higher frequency. The use of sin2 (x) ensures monotonicity and renders it amenable to easy optimization. Liu et al. (2020) demonstrates this periodic activation exhibits an improved extrapolation capability for temperature and financial data prediction."