You should speak to multiple doctors across multiple adjacent specialities.
For me, at my age of 54, my doctors recommended removal over radiation since the long term impact of radiation can impact me down the road - plus radiation is always a backup-option. The way it was conveyed to me was that for someone older they might recommend radiation first because it is easier for an older patient than surgery and their natural lifespan was thus that they are less likely to encounter problems down the road. The way they phrased it was "those men are more likely to die WITH prostate cancer than FROM it".
I spoke to multiples in all the key fields: urology, medical oncology and radiation oncology - each said the same thing, without exception. Their expert advice put me on the path I'm on now. Radiation might have spared me some of the side effects but could have also caused more now or later and those were just simply risks I was not willing to take, and since I still have that as a backup option then it was logical to me.
This was MY experience, so I cannot say what you should or should not do, that is for you and your doctors to decide, strangers on a forum are no replacement for that.
Thank you for the insights. I appreciate the time you took. I agree with all that you said. Ron