This is my stock answer as recently replied to a PM (and keep in mind, it's my brain with my experience at work here; I don't mean to speak for all cases and for all people):
It's strictly your call. Obviously, I don't know a thing about you, and can't do more than to point out the possible outcomes of not controlling it better. If you're okay with that, then here is my counsel, which you must have seen in other replies to people by now:

a. AF is a progressive disorder. The more you have, the worse it gets. The worse it gets, the more problems will happen as it evolves;

b. Thirty eight percent of your time, your oxygenation is poor(er) than when your heart is in NSR. As you age, all sorts of things deteriorate. Now you're adding a deficit, even if mild, of oxygen to the mix;

c. Many people feel that, if it isn't so bad, or not the least bit intrusive, what's to worry about? I feel fine, and I have the two meds, sssoooooo........ That's good, but it isn't stopping the progression of your disordered heart toward more intractable forms of the arrhythmia, and it might go on to result in mitral valve fibrosis and prolapse, and it could end in 'heart failure';

d. Did your various blood tests happen to show any insufficiency of electrolytes, and how about troponin? If the electrolytes were lowish, and if troponin is higher than normal, you have options with the electrolytes (supplementation daily), and the troponin is a marker of myocyte death, probably due to the stresses of the arrhythmia on heart muscle;

e. An ablation, in the right hands, is now considered to be the 'gold standard' of care for AF. It has the best outcomes and the least morbidity between other options, including just controlling it with medication. Your burden suggests that, at the very least, you should be on an anti-arrhythmic drug (to be determined by a competent physician based on your genotype and phenotype), and if it were up to me, you'd be in line with an electrophysiologist now. The idea is to control the onset of AF, to prevent it, and to stay out of it for as much as is possible without making your life truly miserable. For me, being in AF was indeed truly miserable. It was bloody awful. So, soon after my diagnosis, and after reading about things like amiodarone, I knew an ablation was my goal. It took two rounds of it to work, which is the case for about 25% of all first-time ablatees, but I am three days away from my second anniversary completely free of AF. If you'd like to be where I am, you have two options....get on an anti-arrhythmic or get an ablation.