The source is available in https://github.com/hadley/r4ds, and works exactly as you describe when using the preview edition of RStudio (and indeed that's how I write the book)
I just saw this the other day! This wrapper around Pandoc to make life our lifes easier (tidyr)? I just went through the process of making a markdown file to an epub before I saw this, wish I saw this earlier.