This all sound very similar to my condition.
After a bunch tests and things being eliminated I was put on 20 mg of prednisone and the symptoms all went away within days. This was effectively a diagnosis of PMR.
4 mg of prednisone isn’t enough initially to give significant relief.
I started at 20 mg and have now tapered down to 6 mg and I am totally pain free.
Good luck.