I was also told that my lump (golf ball size) was nothing to worry about by radiology. My surgeon felt differently and removed it, grade 3 cancer.
My point is this, never accept an answer that hasn’t been tested. If there is a hard lump and he says it’s a tumor, I would not rest until it is under a microscope. This is my 20/20 hindsight from my own situation.
It is your body and no one will protect it like you will. Please make sure you are comfortable with your second opinion, especially since I think I read you have breast cancer in your family.
When is your appointment for a second?