GitHub is not managed by you and by hosting a static site on GH Pages, you really have no server. The CMS that the auhor refers to is a frontend client-based library that you include in your static site code and communicates directly with GitHub to manage your content. It's a UI for GitHub.
Exactly and this is the purpose of a static site generator. The content is not related to the actual site representation. In the case of the WP, instead of storing the content in GitHub and generating a site to be hosted on GH Pages, you are storing the content in a WP server and generating a site to be hosted somewhere else.