If the Twitter API is down, then you'll have blank spots on your page where there are supposed to be tweets.
If Netlify goes down, you won't be able to publish, but your statically rendered site should still load just fine unless you are using it to make dynamic queries on the fly (which is, I think not generally done for Gatsby, but very possible to do)
A fair point. Also true for any front end that relies on an API to populate its data. Hacker News for instance will use a similar approach. If they concern is that sometimes API's may go down and data won't be displayed you can of course chose not to use API's in your front end. If you're not that bothered like me, that this is just. "one" way you "could" use Twitter as a CMS for you blog.
If Netlify goes down, you won't be able to publish, but your statically rendered site should still load just fine unless you are using it to make dynamic queries on the fly (which is, I think not generally done for Gatsby, but very possible to do)