I think I've seen either Intel or AMD explain hyperthreads as being useful in the case of a gamer who is also using a software encoder to stream their gameplay.
Also, HPC doesn't necessarily imply a uniform workload; it just implies that all workloads are trusted. Really, modern HPC clusters are still "multi-tenant"; they're just tenants who are all e.g. employees of the same company, or researchers at the same university. So you can describe them, from a security perspective, as one "tenant." None of them would have anything to gain by doing a side-channel attack against one-another.
Also, HPC doesn't necessarily imply a uniform workload; it just implies that all workloads are trusted. Really, modern HPC clusters are still "multi-tenant"; they're just tenants who are all e.g. employees of the same company, or researchers at the same university. So you can describe them, from a security perspective, as one "tenant." None of them would have anything to gain by doing a side-channel attack against one-another.