[godelski]

Yeah sorry I don't think I was clear. I don't exactly want to just drop in rich into the python source (for reasons you mention). But I do think they could take some of the ideas from there and place them in. Formatting is really the most important aspect here, especially around traces because these are the real work amplifiers. So much time is spent debugging that the better tools we have to debug better the more work __everyone__ does. But debugging is strangely a underappreciated area. I think you could do colors with just simple ansii encodings (same as you'd do in posix). I'm just using rich as an example of style.

r.e. pytorch: It's a love hate with me. I do think they should incorporate things that are extremely common and solve things that are daily issues. As a simple example, new users are often confused with loading and saving models when using distributed data parallel (DDP) because it creates this extra "module" name in the state_dict and so can require different usage for saving/loading models if you're distributed training or not. This can be quite annoying. Similarly there are no built in infinite samplers, which are common among generative modelers. People who don't iterate over epochs of data, but rather steps. There's of course many solutions to deal with this, but it does make sense with how prolific it is (and has been since 2015) that there just be a built in dataloader. I'd argue things like progress bars and loggers would also be highly beneficial, especially because pytorch's forte is generating research code.

But we're digressing. These are just opinions.

[extraduder_ire]

Would there be any way to inject such formatting in by messing with the interpreter at runtime? I assume it would be possible to get as far as "works most of the time" and "doesn't create too many additional problems".

[godelski]

Like how the rich logging handler works? Or something different? You can definitely write custom handlers for python loggers. Pytorch is a bit more of a pain though.