Cwal, I have the same problem you do in winter. I used to get sinus headaches in winter and sometimes even a tiny bit of blood from dry irritated sinuses. Humidifiers didn’t even work — I didn’t know at the time they weren’t safe.

What I found finally helped is to spray Simply Saline nasal mist in my nose before I go to bed, and I keep the spray bottle on my nightstand if I wake up at night I do it again.
You need to really spray a lot and saturate your sinuses, not just a small spray.