| Hi larsga. Thanks for your interest. To begin, yes the concatenated input RNN is much better. A Markov chain could not do this feat (or make comprehensible text at the character level generally). Occasionally the reviews contradict themselves, but they're actually remarkably consistent regarding the conditioned upon attributes. I'd suggest you pay attention to the "temperature". This is a parameter that determines how stochastic the generation is. With low temperatures the reviews are less varied but "make more sense". With higher temperature they are more entropic. When you tell it to make an IPA review it stays on topic and talks about hoppy flavor. About stouts is consistently calls them black, with hints of chocolate (not to mention using the word "stout"). Here's an example of a review I just generated for "Fruit/Vegetable beer": "This brew pours a very clear golden color. The finger head is pretty small and fizzy and has a slightly pink color. The smell is really nice. The taste is fruity and sweet, but not overwhelming. The flavor is a little weak and is a bit sweeter than most beers but still very nice. I could drink this all day, but I would probably prefer the fruit beer to be a bit more pronounced. This beer is actually quite smooth and inviting. It has a strong taste of raspberries but is complimented by a nice tartness that comes through as well. The mouthfeel is smooth and creamy with a dry finish. This is a very drinkable beer and I could see myself enjoying to try this one again." Clearly the RNN learns to form words like "fruity" and "sweet" and "raspberries" to describe a fruit beer. It also says the flavor is a little "weak" and in the next sentence says it would prefer for the taste to be "more pronounced". Keep in mind, this neural network was given no a priori notion of words. A Markov chain cannot produce even remotely similar conditional text at the character level. For proof that it learns to differentiate the different types of beer, we demonstrate in the paper that the model can be run as a classifier and classify the type of beer form the review with 90% accuracy (on previously unseen test data). This is almost comparable to state of the art logistic regression tf-idf ngram model, despite the fact that we haven't even tuned the model especially carefully to be a classifier (with regularization or hyper-parameter search , for instance. Here's another example (for an IPA): "This is a fine IPA for sure, but not a beer I would love to drink a lot of. This is one of the better IPA's I have ever had. I can see why the beer is unlike any IPA should be and the best beer I've ever had. I could drink this all night, but it is very drinkable. I could easily drink a few of these without a problem. I don't know what the malt base but it is so faint and it is pretty tasty. I can see why the beer is a great hop bomb and the flavors are both subtle and superb. It's a great balance and can be a good supply of the style. I like it, but the hops are a bit off the more I drink them, but the hops are very pronounced. The finish is a little bitter, and the hop flavors are great." I'm not sure what else to say, if you don't believe that the net has learned to distinguish an IPA. Of course, it does contradict itself on sentiment. But with lower temperatures even this is not so common. It can also be addressed by setting extreme star ratings. (we can actually put in a 0 star rating, or as high as a 10 star rating to induce a review of more extreme sentiment). A comedic point that must be made here is that the source material (the reviews from BeerAdvocate) are themselves absurd, and occasionally contradictory. They English they contain is ungrammatical, ridiculous, and frequently misspelled. Nevertheless it's a fascinating dataset on account of how well-annotated and dense it is (the 190-core has over 250k reviews). |