It's probably a smart initial response to cut back on the metoprolol to see. It's not like your life depends on it, and it's not an anti-arrhythmic effect you're looking for...just to keep heart rate lower and the heart a bit calmer. And, it is good for incipient hypertension.
It would be good for a thorough checkup to ensure it is only the 'unneeded' dosage of metoprolol that can be changed, and not some other issue that has crept up on both you and your physician. So, sure, go easier on the metoprolol, but do keep an open mind on other potential conditions/developments that need addressing that have nothing to do with metoprolol.