Your occasional AHI of greater than even 3, let alone 5, is affecting your sleep. Over at apneaboard.com, we routinely get people finding us every day and saying they feel exhausted, depressed, and are about to quit their therapy now six months along. We coach them to learn more about their condition, about the machine they've been given, even to the extent of learning that it's the wrong machine (!!) once we learn of the (redacted) diagnosis and prescription for therapy, and how the machine is not responding to their needs if it's set improperly. All these happen regularly across the medical system. If you ask me, it's a crying shame.
The main problem with affected sleep is interruptions. They cause arousals, which the best machines will duly detect and note. With the freeware software we have developed at apneaboard (it's all strictly voluntary, no paid staff, and we immediately cull ALL commercial links found in any posts by members), members can see much more information at their fingertips by inserting their machine's SD card into a reader an importing the night's data into the freeware, called OSCAR. I use it myself. I can see in digital form, and in analog/graphic form, my various events, what type they were, their duration, my tidal volume, my minute vent, the maximum pressure the machine used, and so on and so on... Once a person understands their prescription, and then their machine's settings and therapy type, it becomes obvious what is incorrect or needs refinement.
One data point is called 'arousal's. We become aroused when we begin to sense an impediment or a threat. When we stop breathing due to central apnea, or are prevented by obstructions, we become aroused. Each arousal, often several times each hour, interrupts the sleep pattern progression. With that fact in mind, it should easily be seen why people feel awful, unrested, and even cheated the next day, and why we begin to doubt the veridical nature of the instruments we are using, including the smart watches (I also have a Galaxy watch which I feel is close to what the OSCAR software tells me next day), and we even suspect the PAP machine itself!!!
You must...you simply MUST.,..get your AHI reliably, and consistently, below 3, or you will continue to suffer unduly. Or, you have the wrong therapy, or the wrong diagnosis. One of those three.