[robbedpeter]

Interesting - the modular idea is one of the most interesting to me. The recent hierarchical transformers papers hint that models can be made smaller and might open the door to modular approaches, which could mean highly nuanced customization of your domains of interest, and fitting the model size to the capacity of consumer hardware like phones.

Thanks for the effort you're putting into this!