[calebh]

Mathematica can definitely compute the derivatives if you fix the size of the matrix. This isn't very useful if you're trying to compute the derivative of an expression with arbitrary sized matrices.

[improbable22]

You can do general matrices too, what do you have in mind?

aa[x_] = {{1, 2}, {3, 4}} x

bb[x_] = {x^2, x^3}

D[ a[x].b[x] , x, x] (* for any suitable tensors *)

% /. {a -> aa, b -> bb}