You are in charge!
My experience may be analogous to your situation.
I was being treated by a well regarded gastroenterologist for radiation proctitis. My RO deferred to him. After a long course of treatment with sucralfate, I thought that hyperbaric oxygen treatment should be my next step. My gastro doctor did not agree. My urologist was very supportive of getting hyperbaric oxygen therapy. While I wanted the HBOT treatment my personal calendar was not quite conducive to six weeks of treatment (perhaps not my best decision to prioritize deer hunting season and other events on my calendar but I decided that I could wait several months). I took the approach of repeatedly lobbying for a referral from my gastro.
Eventually he referred me to the HBOT treatment staff in the same hospital. I received the treatment and my symptoms of radiation proctitis receded significantly.

BTW, I quit Orgovyx after one year against my urologist's recommendation. I was ready to look for another urologist if he had refused to see me after I quit the ADT. (I had severe joint pain at that time.) That was 5 years ago, and my PSA continues to hover around 0.1 and my urologist says I am doing much better than he expected.

If my decisions for my care had resulted in a poor outcome, I was ready to accept responsibility for those decisions. I'm glad that I "rolled the dice".