https://arxiv.org/pdf/1909.02803.pdf
Note that the “fine tuning” and “alignment” (RLHF) stages are much shorter than the early training so technically savvy people can already customize models.