I found speech and language processing to be one of the most interesting courses of my degree. Recently I decided to take a look at speech synthesis again and discovered a book by Paul Taylor on this subject (http://svr-www.eng.cam.ac.uk/~pat40/ and draft PDF at http://svr-www.eng.cam.ac.uk/~pat40/ttsbook_draft_2.pdf). It is more engineering focused than other books in this area.
Note that the book was released 2009 and the draft is from 2007. While most of the content is still very relevant for understanding the basics and challenges in building TTS systems, recent progress in DNN-based synthesis (including WaveNet, GANs, and end-to-end approaches like Tacotron) is obviously not covered.