[rmrfstar]

> at some point admit that we want machine learning algorithms to be able to interface with symbolic rules rather than pretending that everything is a data issue

That is a long way off. We find ourselves in a situation where we have models that do useful things, but are also horrendously biased.

If you want improvement now, the quickest way to get it is recasting benchmarks. Most NLP research groups just hill climb on GLUE or whatever. What gets measured gets managed, and bias is not being systematically measured.

Right now language models can't even compare ages that fall outside the 20-100 range, and it is really unlikely that we are going to quickly get to architectures with symbolic reasoning sophisticated enough to squelch bias.

To Google's credit, they are working to measure (and therefore manage) bias [2].

[1] https://arxiv.org/pdf/1912.13283.pdf

[2] https://www.youtube.com/watch?v=XR8YSRcuVLE