The PSA is like a check-engine light. Something is wrong; it could be minor or a major problem. I’d suggest to have a biopsy. In my situation, my PSA was 4.2. My biopsy showed cancer in 6 of 12 samples. Ugh.
Good luck with your decision. It a tuff one; however, you’re on the right track!