Hacker News new | ask | show | jobs
by microtonal 4783 days ago
Rule-based parsers recover richer representations, but tend to have lower coverage over arbitrary data

This parser usually has a coverage around ~95-97%. In one experiment we also parsed Flemish (which uses some constructions that wouldn't be considered grammatical in Dutch) and obtained a coverage of ~94%.

It also has a robustness component that attempts to provide an analysis for as many constituents as possible if no fully spanning parse can be found.

Having worked on the CCG formalism through my PhD, I can say that even small differences in annotation scheme can make a big difference in which parsers come out ahead.

Certainly. But you are not mentioning the other elephant in the room: Dutch is a free word order language and also permits very liberal ordering in the middle field. The rule-based grammar may benefit from the detailed constraints in the lexical attribute-value structures.

1 comments

> Certainly. But you are not mentioning the other elephant in the room: Dutch is a free word order language and also permits very liberal ordering in the middle field. The rule-based grammar may benefit from the detailed constraints in the lexical attribute-value structures.

Mostly I didn't want to stick my neck out :p. It's easy to say something untrue about a language you don't know and haven't worked with.

I'd also be reluctant to assume which mechanisms were making a difference, because it's so hard to guess what cases are frequent and not easily inferred by a statistical model. One thing we can know is that the transition based parsers are best at producing projective dependency trees. The various techniques for non-projective shift-reduce parsing aren't very good.

Don't get me wrong --- I totally think the suggestion you offered makes sense, and it's a likely explanation. It's just that this stuff all very tricky.