No, it can be an operator on any kind of (usually) ring or more general algebraic structure. What you refer to is the "usual" derivative of functions of one variable, just one of many derivatives one can define.
I implemented Brzozowski's regex derivatives to build a regex implementation back-end. That back-end is used whenever exotic constructs (negation, intersection) appear in the abstract syntax of the regex; in their absence, the implementation falls back on the NFA-graph-based back end.
Yes, this seems to build on the idea of regex derivatives. If regex derivatives can be used to transform a regular expression into a recognizer for strings, why not transform a more general grammar into a recognizer of strings.
Yes, it is completely arbitrary. However, this arbitrary definition follows certain rules and properties and can therefore be used for certain types of mathematical reasoning.
This is why, when we talk about rings and fields and such we say "multiplication-like" or "addition-like" operators. The operators defined for the algebraic structure may not be exactly like "standard" operators, but they still follow rules and you can still do cool things with them.
http://matt.might.net/papers/might2011derivatives.pdf