ESP32 is perfect for this application. RPI is far too beefy, unless you really want to do everything locally (without relying on an external server to package up the train data for you)
Sure, though 240Mhz, 32bit, optional gobs of PSRAM, etc, aren't usually what comes to mind when you hear MCU.
Edit: Yes, you could even use an ESP8266, though 32x128x3 plus X bits of PWM brightness would mean being limited to drawing from storage rather than in-memory manipulation. I'd jump right to the WROOM type devices with PSRAM.
If you need a microcontroller with networking (such as the train sign would), I think the ESP32 (or older/cheaper/less capable ESP8266) is one of the first that comes to mind for most engineers, especially those who aren't full-time firmware engineers.
You are quite right that the ESP is way over-specified for many tasks, but it's not like the uC is gonna quit and take a more fulfilling job elsewhere. You can get tiny boards for close to $5/ea, or under $5 if you don't need the USB connector. - and that's in single-digit quantities. The hardware configuration is very flexible, eg if power consumption matters you can run it a lot slower, switch off subsystems you don't need, use a watchdog timer to simplify exception handling etc.. There's a rich and constantly expanding SW ecosystem so you can get to a working prototype very fast.
Edit: Yes, you could even use an ESP8266, though 32x128x3 plus X bits of PWM brightness would mean being limited to drawing from storage rather than in-memory manipulation. I'd jump right to the WROOM type devices with PSRAM.