I will be glad to share my experience but please be aware that I am not your physician and it would be inappropriate for me to give you medical advice. So, no medical advice.
A few considerations:

You said your PSA went from 1.6 to 3.98 (1/15 to 1/24). You didn't state over what time. A year? Is this a one year rise? In general, urologists don't look at the absolute PSA as much as they look at the rate of rise. A 20% or more increase in PSA over a one year period is concerning. My PSA went up more than 20% in a year and that was a red flag.

An MRI with contrast is a very good test when performed on a good machine and interpreted by a qualified, experienced Radiologist. It would be unusual to have a high grade prostate cancer with a Pi-Rads 2 reading. But cancers have to start small and there could be a small cancer it doesn't detect. It may also miss a cancer that is not as aggressive (growing as quickly). All tests, CT, MRI, PET, bone scan, US etc, are very good individually at what they do and the limits of resolution have improves greatly over the years, but they are not infallible. They are not 100% accurate (sensitive and specific). Having said that, consecutive years negative MRIs are certainly a good sign.

My experience with prostate biopsy (and having talked to others ) is that it is now a relatively painless procedure when performed under anesthesia. I was given propofol and the procedure took 15 minutes. I had no pain afterwards. I had a little a little blood in my urine for a week or so and blood in my semen for 6 weeks or so. As with any invasive procedure, complications (infection etc) can occur but are unusual. It seems you are concerned about the possible pain associated with the biopsy. I would not let that one factor keep you from having a biopsy. Does your urologist use anesthesia? Can you consult one that does?

I did somewhat ignore a rising PSA for 2 years. In early 2020 during my annual physical exam my PSA had gone from around 4 to 5.5 (more than 20% rise). My Internist referred me to a urologist (a colleague and friend for decades) who did a digital exam and felt nothing. I had a negative MRI. He said we could do a biopsy or repeat the PSA in 6 months. I elected to repeat the PSA. It went down a little and gave me a false sense of security. Also, should have consulted a urologist that didn't know me and would have treated me as just another patient. At my next physical exam in 2021 my PSA jumped to 7.5. Went to see a different urologist who said he was going to biopsy no matter the MRI results. However, he did want a repeat MRI because if it showed a specific suspicion area, he would be sure to get samples from there. The MRI did show a new, enhancing 1 cm nodule. On biopsy it was a Gleason 9 cancer. A RP ensued several months late. Followed by rise in PSA with PSMA PET revealing solitary T8 met. Radiated with SBRT. Four months later PSA again rising with rapid doubling time, repeat PET showed positive node in the pelvis wit presumed additional micrometers which weren't visible. Sought treatment at Johns Hopkins, ADT, Darolutamide, (second generation androgen receptor blocker), Chemo with Taxotere. PSA went rapidly undetectable. After chemo had full pelvis radiation. Now almost a year later I am still PSA undetectable and off all medication.

I guess the point of my story is I pretty much ignored a rising PSA (more than 20% year over year) and I wish I had gotten a biopsy a year earlier. There is no test (Imaging or blood test) that can definitively exclude cancer. In my career I biopsied thousands of organs and body parts. Even did prostate biopsies in the 90s before the urologists took them over. The significant majority of times the biopsies weren't painful for the patient (they said). No matter what the Imaging (CT, MRI etc) shows, you need tissue to make a definitive diagnosis. There were many times I thought the biopsy was going to be cancer form the Imaging and turned out to be benign and vice versa. Got to have tissue to know for sure.

You need to have complete confidence in your physician. Your physician needs to be well trained and up to date. If you are unsure, seek another consultation from someone at a large, respected institution. Prostate cancer, in particular, is a very complicated. I practiced medicine for over 40 years and thought I knew a lot about it. After I was diagnosed I have extensively read the literature and talked to numerous colleagues. I still know only a fraction of what my Medical Oncologist at Johns Hopkins knows. There is a saying "a little knowledge is a dangerous thing". I believe that. My daughter is a physician/Professor in Infectious Disease at a major medical center/teaching hospital. One of her colleagues gave her a coffee cup that said "Don't confuse your Google search with my medical degree". Of course, it was meant to be funny but there is truth in that. I am all for people being informed, being part of their treatment decision making and having the ultimate say so, as it is their (our) bodies. But I believe it can be foolish if laymen begin thinking they know more about the disease and treatment than their doctors. If that is the case, they need to find a new physician. I know I don't know as much and I rely on the advice of my physician because I trust him/her/them.

Good luck!