Tamoxifen prescribed after ADH (Atypical Ductal Hyperplasia) diagnosis

I had a lumpectomy which revealed ADH. My doc started me on Tamoxifen. I was on it for 6 months, then had back to back surgeries for other issues so I had to go off of it. She wants me to go back on the drug but I'm hesitant. In the 6 months I was on it (at half dose) I gained a fatty liver, hot flashes, lost hair. Six months after going off of it, a test revealed a spot on my uterus and and an enlarged ovary. I had a hysterectomy that showed a polyp on my uterus that was hyperplasia. I apologize if this sounds whiney. I don't have the BRCA genes but my mother, aunt, grandmother, and cousin (all on the maternal side) have had breast cancer. My cousin took full dose Tamoxifen for 5 years and now has macular degeneration (unsure if Tamoxifen was the cause of course, but vision issues are another side effect). I am really stuck as to whether or not to take the drug. Again, I know many on here have had breast cancer so my apologies if this seems trivial in comparison. Would love to know if others are in a similar situation though, with taking it after ADH diagnosis. Thank you