Variance in metabolism at the extremes is less than the energy in a Snickers bar. It matters, but worrying about your metabolism without first sorting your overall intake is a classic case of premature optimization.
No doubt, but "I have a fast/slow metabolism" is cited as an excuse by a lot of people in regards to weight changes (or lack thereof), and it's an excuse that is more or less irrelevant compared to intake management.