A primary key is the same thing as a unique index (any index which enforces unique entries) + a not null constraint.
It might make a difference in the heuristic for finding functional dependencies for group by though. This feature was introduced in 9.1 and allows one to not always have to list all columns in the GROUP BY clause.
postgresql doesn't automatically create indexes for foreign keys.
> Is there a fundamental difference to the default pkey index versus an explicitly created index?
i don't know of any in practice.