HyenaDNA is one to look at wrt DNA.
And here are some other interesting links from Erik Garrison—a leader in pangenomics.
https://hazyresearch.stanford.edu/blog/2023-06-29-hyena-dna
https://github.com/instadeepai/nucleotide-transformer
https://dl.acm.org/doi/pdf/10.1145/3535508.3545512
https://github.com/dnbaker/bioseq
https://huggingface.co/AIRI-Institute/gena-lm-bert-base
https://discuss.huggingface.co/t/dna-long-sequence-tokenizat...