Referring to the German language as Dutch (as in PA Dutch) is closer to the German word for the German language. The word "German" is actually the Anglicisation as far as I know.
High German is the language which was traditionally spoken in the southern 2/3 of Germany and in Austria; it evolved from the languages of the Allemani, Bavarians, and Franks. Low German is the language which was traditionally spoken in the northern 1/3 of Germany; it evolved from Saxon. For historical reasons (the big one being Martin Luther's choice of German dialect for translating the Bible), High German became the basis of the "standard" literary German language and is taught in schools, while Low German is seen as a weird dialect. Although if objective criteria were used, Low German and High German would be classified as different languages, not dialects.
I don't know, I just know "is it dutch or is it german?" is a common question and that's the answer. I think high/low is a geographic indicator of where it's spoken in germany.