Deep TabNine (announced today) uses transformers. TabNine (released last year) uses logistic regression.