[heyitsguay]

How would you define "new best HW"? Seems challenging, particularly if it's for generation N+1 ML models that haven't been created yet. Also, while there is work underway to use ML models to guide HW design, it's not clear to me that the best ML models for that == the best ML models for more general purpose tasks in audio, visual, and natural language processing, i.e. is HW circuit design done using transformers? What are the inputs, latent space, and outputs?

GPU development seems driven by more general computational principles that might be summed up glibly as "We're hitting fundamental physics limits for single-core processors, what is the maximum amount of data we can move per second through the maximum number of cores?" Perhaps there is a way to extend this with a view toward the challenges of current ML model training and inference, but I imagine trying to approach it as a black-box optimization problem could be quite difficult.