Allow me to share my experience since I started wearing hearing aids in 2007 because I was increasingly facing difficulties with word recognitions, such as understanding my team members during project review meetings. In noisy restaurants it was even more noticeable. I was fitted with a pair of Oticon Epoq which cost me at that time a whooping $6,600. Initially, I faced the same issue as you: I could not hear clearly and I had to make more efforts to understand others, so when the meetings were over, I would take the hearing aids out. I never wore them at home. That was a mistake. Complaining to my audiologist, she repeated again to wear them all the time during waking hours in order to allow the brain to gradually adapt to recognizing again the sounds as it remembered it. Having "gotten the message", wearing them all day became a habit and my word recognition in any environment was no issue. Whenever I felt an adjustment was needed, I would get a hearing test, the HAs would be adjusted, again requiring my brain to adapt to it, sometimes it would take a few hours, sometimes 1-3 days maximum. As my hearing gradually deteriorated, I was fitted in 2017 with a pair of Oticon miniRITE S1. It took my brain about 2 weeks to adapt to them and I wouldn’t even notice I was wearing them.

The key point I want to make is that apparently many licensed audiologists (they have a doctor's degree in audiology), as well as the Licensed Hearing Aid Fitters (like at COSTCO) apparently don’t explain it well enough, or emphasize, or make it clear to HA users that there is a need for the brain to adjust to the sounds provided by the hearing aids. Whether the patient is new to hearing aids, or a longtime user seeking an adjustment, or needing HA replacement, the brain simply needs time to adjust to it. It requires patience and sometime determination. There is no way around it.
On the online Hearing Aids Forum I found other HA users making similar comments.