If you had the OncotypeDX test and the report from Oncotype listed your result as 4% 'risk of recurrence (loco-regional) within 9 years', that's a very low risk. Assuming, using the statistic accepted by oncologists, that aromatase inhibitors 'may' (that verb is important) reduce your risk of recurrence by less than 50%, the drugs would theoretically reduce your risk from recurrence risk from 4% to about 2.4%.
Put it another way. If you do NOT take the drugs, you have a 96% chance that the cancer will NOT recur 'within 9 years.' And the drugs 'may' only improve that to about 97.6%.
I know three women with similar OncotypeDX scores who declined anti-hormone therapy with the agreement of their oncologists. They are all over 65 years of age and tried the drugs, had side effects that, they felt, were quality of life issues for only a small increase in risk of recurrence.
These are personal decisions. But one way to carve out a path forward would be to get a second opinion from a different oncologist. You can also always try the drugs and might have no side effects or discomfort of any consequence.
I tried anastrozole and had side effects that were a problem. But I also had the OncotypeDX that gave me a very low risk of recurrence as well so stopped taking the drug as the risk/reward ratio wasn't persuasive. I did so though after consulting with two oncologists, both of whom concurred.
It is your body and your life and doctors can advise but they don't live with the consequences. And they frequently don't agree with each other either. So I hope you find the decision that gives you the most peace of mind...
May I ask where you obtain an Oncotype DX test?