HRT Intolerance or Side Effects?

Very hopeful that others may share reassuring feedback. Im 63, 7 years post meno, started HRT about 7 months ago (Estradiol patch 0.5 and Progesterone oral 100mg). First 3-4 months were great, and then it changed - tons of weird side effects including brain fog, lightheadedness, anxiety/panic, visual changes, and more. I read about Progesterone Intolerance and my doc lowered it to 35mg through compounding pharmacy. Note, Im extremely petite (93 pounds) and may have high sensitivities to things. I wake up from HEAVY sleep, feel like Im in a fog for hours. Don't know what my next step is... lower Estradiol? Lower Progesterone? Stop HRT? This is very scary. Im otherwise VERY HEALTHY and active and youthful and productive, but this is interfering with my life almost continually. Are there women who cant do HRT? Anyone struggle with this and what did you do? Thank you!