What you can do to offset these issues is to make sure your doctor uses a MRIdian SBRT machine (or equivalent). The MRI guided SBRT uses a much finer beam that affects less tissue near where Radiation is needed. You can also get a SpaceOAR or equivalent to protect your rectum.

I have not really heard of people complaining about long term after effects of SBRT, at least there’s not been a major discussion from people that feel they are affected, like you describe. I had seven weeks IMRT radiation and the only issues I have after 11 years is some incontinence.